Azure ML Documentation

Contents
Azure Machine Learning service Documentation

Overview
What is Azure Machine Learning service?
Architecture & terms
Tutorials
Python SDK
Create first ML experiment
1. Set up workspace & dev environment
2. Train your first model
Image classification (MNIST data)
1. Train a model
2. Deploy a model
Regression (NYC Taxi data)
Auto-train an ML model
Visual interface
1. Train regression model
2. Deploy regression model
Automated machine learning
Classification (Bank data)
Samples
Notebooks
Visual interface experiments
Predict price
Predict price (advanced)
Predict credit risk
Predict credit risk (advanced)
Predict churn
Predict flight delays
Azure Open Datasets
Concepts
Deep learning
Workspace
Compute target
Visual interface
Model management (MLOps)
ML pipelines
Model interpretability
ONNX
Security
Enterprise security overview
Manage users and roles
Use virtual networks
Secure web services with SSL
Regenerate storage access keys
How-to guides
Manage workspaces
Use portal for workspaces
Use a Resource Manager template
Set up development environment
Configure dev environments
Manage software environment
Enable logging
Set input & output directories
Use VS Code extension
Work with data
Get data from a datastore
Add & register datasets
Train models
Set up training environments
Create estimators in training
Use Scikit-learn
Use TensorFlow
Use Keras
Use PyTorch
Use Chainer
Tune hyperparameters
Use Key Vault when training
Track & monitor experiments
Start, monitor or cancel runs
Log metrics for training runs
Track experiments with MLflow
Visualize runs with TensorBoard
Automate machine learning
Use automated ML (Python)
Use automated ML (Azure portal)
Use remote compute targets
Auto-train a forecast model
Understand charts and metrics
Deploy & serve models
Where and how to deploy
Deploy to Azure Kubernetes Service
Deploy for GPU inference
Deploy to Azure App Service
Deploy to Azure Container Instances
Deploy to Notebook VMs
Use custom Docker image
Deploy existing models
Deploy to IoT Edge
Deploy for FPGAs inference
Troubleshoot & debug deployments
Consume web services
Consume in real-time
Run batch predictions
Monitor models
Collect & evaluate model data
Detect data drift
Monitor with Application Insights
Create reusable workflows
Create ML pipelines in Python
Azure Pipelines for CI/CD
Manage resource quotas
Export and delete data
Reference
Machine learning SDK
Machine Learning CLI
Visual interface modules
Module reference overview
Data format conversions
Convert to CSV
Data input & output
Import Data
Export Data
Enter Data Manually
Data transformations
Select Columns in Dataset
Edit Metadata
Clean Missing Data
Feature Hashing
Extract N-Gram Features from Text
Add Columns
Add Rows
Remove Duplicate Rows
Preprocess Text
Join Data
Split Data
Normalize Data
Partition and Sample
Machine learning
Score Model
Assign Data To Clusters
Train Model
Train Clustering Model
Evaluate Model
Apply Transformation
Linear Regression
Neural Network Regression
Decision Forest Regression
Boosted Decision Tree Regression
Two-class Boosted Decision Tree
Two-Class Logistic Regression
Multiclass Logistic Regression
Multiclass Neural Network
Multiclass Decision Forest
Multiclass Boosted Decision Tree
Two-Class Averaged Perceptron
Two-Class Decision Forest
Two-Class Neural Network
Two-Class Support Vector Machine
K-Means Clustering
Python and R language modules
Execute Python Script
Create Python Model
Execute R Script
Error messages & codes
Resources
Release notes
Azure roadmap
Pricing
Regional availability
Known issues
User forum
Stack Overflow
Compare our ML products
What happened to Workbench
8/26/2019 • 4 minutes to read • Edit Online
Azure Machine Learning service is a cloud service that you use to train, deploy, automate, and manage machine
learning models, all at the broad scale that the cloud provides.
What is machine learning?

Machine learning is a data science technique that allows computers to use existing data to forecast future
behaviors, outcomes, and trends. By using machine learning, computers learn without being explicitly
programmed.
Forecasts or predictions from machine learning can make apps and devices smarter. For example, when you shop
online, machine learning helps recommend other products you might want based on what you've bought. Or
when your credit card is swiped, machine learning compares the transaction to a database of transactions and
helps detect fraud. And when your robot vacuum cleaner vacuums a room, machine learning helps it decide
whether the job is done.

Azure Machine Learning service provides a cloud-based environment you can use to prep data, train, test, deploy,
manage, and track machine learning models. Start training on your local machine and then scale out to the cloud.
The service fully supports open-source technologies such as PyTorch, TensorFlow, and scikit-learn and can be
used for any kind of machine learning, from classical ml to deep learning, supervised and unsupervised learning.
Explore and prepare data, train and test models, and deploy them using rich tools such as:
A visual interface in which you can drag-n-drop modules to build your experiments and then deploy models
Jupyter notebooks in which you use the SDKs to write your own code, such as these sample notebooks
Visual Studio Code extension
What can I do with Azure Machine Learning service?

Use the Azure Machine Learning Python SDK with open-source Python packages, or use the visual interface
(preview ) to build and train highly accurate machine learning and deep-learning models yourself in an Azure
Machine Learning service Workspace.
You can choose from many machine learning components available in open-source Python packages, such as
Scikit-learn, Tensorflow, PyTorch, and MXNet.
Whether you write code or use the visual interface, you can track multiple runs as you experiment to find the best
solution as well as manage the deployed models.
Code -first experience
Start training on your local machine using the Azure Machine Learning Python SDK and then scale out to the
cloud. With many available compute targets, like Azure Machine Learning Compute and Azure Databricks, and
with advanced hyperparameter tuning services, you can build better models faster by using the power of the
cloud.
You can also automate model training and tuning using the SDK.
UI -based, low-code experience
For code-free training, try:
Creating automated ML experiments in the easy-to-use interface.
The drag-n-drop experimenting in the visual interface.
Operationalization (MLOps)
When you have the right model, you can easily use it in a web service, on an IoT device, or from Power BI. For
more information, see the article on how to deploy and where.
Then you can manage your deployed models by using the Azure Machine Learning SDK for Python or the Azure
portal.
These models can be consumed and return predictions in real time or asynchronously on large quantities of data.
And with advanced machine learning pipelines, you can collaborate on each step from data preparation, model
training and evaluation, through deployment. Pipelines allow you to:
Automate the end-to-end machine learning process in the cloud
Reuse components and only re-run steps when needed
Use different compute resources in each step
Run batch scoring tasks
To get started using Azure Machine Learning service, see Next steps.
How does Azure Machine Learning service differ from Studio?

Machine Learning Studio is a collaborative, drag-and-drop visual workspace where you can build, test, and deploy
machine learning solutions without needing to write code. It uses prebuilt and preconfigured machine learning
algorithms and data-handling modules as well as a proprietary compute platform.
Azure Machine Learning service provides both SDKs -and- a visual interface(preview ), to quickly prep data, train
and deploy machine learning models. This visual interface (preview ) provides a similar drag-and-drop experience
to Studio. However, unlike the proprietary compute platform of Studio, the visual interface uses your own
compute resources and is fully integrated into Azure Machine Learning service.
Here is a quick comparison.
AZURE MACHINE LEARNING SERVICE:

MACHINE LEARNING STUDIO VISUAL INTERFACE
Generally available (GA) In preview
Modules for interface Many Initial set of popular modules
Training compute targets Proprietary compute target, CPU Supports Azure Machine Learning
support only compute, GPU or CPU.
(Other computes supported in SDK)
Deployment compute targets Proprietary web service format, not Enterprise security options & Azure
customizable Kubernetes Service.
(Other computes supported in SDK)
Automated model training and No Not yet in visual interface.

hyperparameter tuning (Supported in the SDK and Azure
portal.)
Try out the visual interface (preview ) with Tutorial: Predict automobile price with the visual interface.
NOTE
Models created in Studio can't be deployed or managed by Azure Machine Learning service. However, models created and
deployed in the service visual interface can be managed through the Azure Machine Learning service workspace.
Free trial
If you don’t have an Azure subscription, create a free account before you begin. Try the free or paid version of
Azure Machine Learning service today.
You get credits to spend on Azure services. After they're used up, you can keep the account and use free Azure
services. Your credit card is never charged unless you explicitly change your settings and ask to be charged. Or
activate MSDN subscriber benefits, which give you credits every month that you can use for paid Azure services.
Next steps
Create a Machine Learning service workspace to get started.
Follow the full-length tutorials:
Create a workspace and train your first ML model
Train an image classification model with Azure Machine Learning service
Learn about machine learning pipelines to build, optimize, and manage your machine learning scenarios.
Read the in-depth Azure Machine Learning service architecture and concepts article.
For more information, see other machine learning products from Microsoft.
How Azure Machine Learning service works:
Architecture and concepts
Learn about the architecture, concepts, and workflow for Azure Machine Learning service. The major components
of the service and the general workflow for using the service are shown in the following diagram:
Workflow
The machine learning model workflow generally follows this sequence:
1. Train
Develop machine learning training scripts in Python or with the visual interface.
Create and configure a compute target.
Submit the scripts to the configured compute target to run in that environment. During training, the
scripts can read from or write to datastore. And the records of execution are saved as runs in the
workspace and grouped under experiments.
2. Package - After a satisfactory run is found, register the persisted model in the model registry.
3. Validate - Query the experiment for logged metrics from the current and past runs. If the metrics don't
indicate a desired outcome, loop back to step 1 and iterate on your scripts.
4. Deploy - Develop a scoring script that uses the model and Deploy the model as a web service in Azure,
or to an IoT Edge device.
5. Monitor - Monitor for data drift between the training dataset and inference data of a deployed model.
When necessary, loop back to step 1 to retrain the model with new training data.
Tools for Azure Machine Learning

Use these tools for Azure Machine Learning:
Interact with the service in any Python environment with the Azure Machine Learning SDK for Python.
Automate your machine learning activities with the Azure Machine Learning CLI.
Write code in Visual Studio Code with Azure Machine Learning VS Code extension
Use the visual interface (preview ) for Azure Machine Learning service to perform the workflow steps without
writing code.
NOTE
Although this article defines terms and concepts used by Azure Machine Learning service, it does not define terms and
concepts for the Azure platform. For more information about Azure platform terminology, see the Microsoft Azure glossary.
Glossary
Workspace
Experiments
Models
Run Configuration
Estimators
Dataset & datastores
Compute targets
Training script
Run
Git tracking
Snapshot
Activity
Image
Deployment
Web services
IoT modules
ML pipelines
Logging
Workspaces
The workspace is the top-level resource for Azure Machine Learning service. It provides a centralized place to
work with all the artifacts you create when you use Azure Machine Learning service. You can share a workspace
with others. For a detailed description of workspaces, see What is an Azure Machine Learning workspace?.
Experiments
An experiment is a grouping of many runs from a specified script. It always belongs to a workspace. When you
submit a run, you provide an experiment name. Information for the run is stored under that experiment. If you
submit a run and specify an experiment name that doesn't exist, a new experiment with that newly specified name
is automatically created.
For an example of using an experiment, see Tutorial: Train your first model.
Models
At its simplest, a model is a piece of code that takes an input and produces output. Creating a machine learning
model involves selecting an algorithm, providing it with data, and tuning hyperparameters. Training is an iterative
process that produces a trained model, which encapsulates what the model learned during the training process.
A model is produced by a run in Azure Machine Learning. You can also use a model that's trained outside of
Azure Machine Learning. You can register a model in an Azure Machine Learning service workspace.
Azure Machine Learning service is framework agnostic. When you create a model, you can use any popular
machine learning framework, such as Scikit-learn, XGBoost, PyTorch, TensorFlow, and Chainer.
For an example of training a model using Scikit-learn and an estimator, see Tutorial: Train an image classification
model with Azure Machine Learning service.
The model registry keeps track of all the models in your Azure Machine Learning service workspace.
Models are identified by name and version. Each time you register a model with the same name as an existing
one, the registry assumes that it's a new version. The version is incremented, and the new model is registered
under the same name.
When you register the model, you can provide additional metadata tags and then use the tags when you search
for models.
TIP
A registered model is a logical container for one or more files that make up your model. For example, if you have a model
that is stored in multiple files, you can register them as a single model in your Azure Machine Learning workspace. After
registration, you can then download or deploy the registered model and receive all the files that were registered.
You can't delete a registered model that is being used by an active deployment.
For an example of registering a model, see Train an image classification model with Azure Machine Learning.
Run configurations
A run configuration is a set of instructions that defines how a script should be run in a specified compute target.
The configuration includes a wide set of behavior definitions, such as whether to use an existing Python
environment or to use a Conda environment that's built from a specification.
A run configuration can be persisted into a file inside the directory that contains your training script, or it can be
constructed as an in-memory object and used to submit a run.
For example run configurations, see Select and use a compute target to train your model.
Estimators
To facilitate model training with popular frameworks, the estimator class allows you to easily construct run
configurations. You can create and use a generic Estimator to submit training scripts that use any learning
framework you choose (such as scikit-learn).
For PyTorch, TensorFlow, and Chainer tasks, Azure Machine Learning also provides respective PyTorch,
TensorFlow, and Chainer estimators to simplify using these frameworks.
For more information, see the following articles:
Train ML Models with estimators.
Train Pytorch deep learning models at scale with Azure Machine Learning.
Train and register TensorFlow models at scale with Azure Machine Learning service.
Train and register Chainer models at scale with Azure Machine Learning service.
Datasets and datastores
Azure Machine Learning Datasets (preview ) make it easier to access and work with your data. Datasets
manage data in various scenarios such as model training and pipeline creation. Using the Azure Machine
Learning SDK, you can access underlying storage, explore and prepare data, manage the life cycle of different
Dataset definitions, and compare between Datasets used in training and in production.
Datasets provide methods for working with data in popular formats, such as using from_delimited_files() or
to_pandas_dataframe() .
For more information, see Create and register Azure Machine Learning Datasets. For more examples using
Datasets, see the sample notebooks.
A datastore is a storage abstraction over an Azure storage account. The datastore can use either an Azure blob
container or an Azure file share as the back-end storage. Each workspace has a default datastore, and you can
register additional datastores. Use the Python SDK API or the Azure Machine Learning CLI to store and retrieve
files from the datastore.
Compute targets
A compute target lets you specify the compute resource where you run your training script or host your service
deployment. This location may be your local machine or a cloud-based compute resource. Compute targets make
it easy to change your compute environment without changing your code.
Learn more about the available compute targets for training and deployment.
Training scripts
To train a model, you specify the directory that contains the training script and associated files. You also specify an
experiment name, which is used to store information that's gathered during training. During training, the entire
directory is copied to the training environment (compute target), and the script that's specified by the run
configuration is started. A snapshot of the directory is also stored under the experiment in the workspace.
For an example, see Tutorial: Train an image classification model with Azure Machine Learning service.
Runs
A run is a single execution of a training script. Azure Machine Learning records all runs and stores the following
information:
Metadata about the run (timestamp, duration, and so on)
Metrics that are logged by your script
Output files that are autocollected by the experiment or explicitly uploaded by you
A snapshot of the directory that contains your scripts, prior to the run
You produce a run when you submit a script to train a model. A run can have zero or more child runs. For
example, the top-level run might have two child runs, each of which might have its own child run.
GitHub tracking and integration
When you start a training run where the source directory is a local Git repository, information about the
repository is stored in the run history. For example, the current commit ID for the repository is logged as part of
the history. This works with runs submitted using an estimator, ML pipeline, or script run. It also works for runs
submitted from the SDK or Machine Learning CLI.
Snapshots
When you submit a run, Azure Machine Learning compresses the directory that contains the script as a zip file
and sends it to the compute target. The zip file is then extracted, and the script is run there. Azure Machine
Learning also stores the zip file as a snapshot as part of the run record. Anyone with access to the workspace can
browse a run record and download the snapshot.
NOTE
To prevent unnecessary files from being included in the snapshot, make an ignore file (.gitignore or .amlignore). Place this
file in the Snapshot directory and add the filenames to ignore in it. The .amlignore file uses the same syntax and patterns as
the .gitignore file. If both files exist, the .amlignore file takes precedence.
Activities
An activity represents a long running operation. The following operations are examples of activities:
Creating or deleting a compute target
Running a script on a compute target
Activities can provide notifications through the SDK or the web UI so that you can easily monitor the progress of
these operations.
Images
Images provide a way to reliably deploy a model, along with all components you need to use the model. An image
contains the following items:
A model.
A scoring script or application. You use the script to pass input to the model and return the output of the
model.
The dependencies that are needed by the model or scoring script or application. For example, you might
include a Conda environment file that lists Python package dependencies.
Azure Machine Learning can create two types of images:
FPGA image: Used when you deploy to a field-programmable gate array in Azure.
Docker image: Used when you deploy to compute targets other than FPGA. Examples are Azure Container
Instances and Azure Kubernetes Service.
The Azure Machine Learning service provides a base image, which is used by default. You can also provide your
own custom images.
Image registry
Images are cataloged in the image registry in your workspace. You can provide additional metadata tags when
you create the image, so that you can query them to find your image later.
For an example of creating an image, see Deploy an image classification model in Azure Container Instances.
For an example of deploying a model using a custom image, see How to deploy a model using a custom Docker
image.
Deployment
A deployment is an instantiation of your model into either a web service that can be hosted in the cloud or an IoT
module for integrated device deployments.
Web service deployments
A deployed web service can use Azure Container Instances, Azure Kubernetes Service, or FPGAs. You create the
service from your model, script, and associated files. These are encapsulated in an image, which provides the run
time environment for the web service. The image has a load-balanced, HTTP endpoint that receives scoring
requests that are sent to the web service.
Azure helps you monitor your web service deployment by collecting Application Insights telemetry or model
telemetry, if you've chosen to enable this feature. The telemetry data is accessible only to you, and it's stored in
your Application Insights and storage account instances.
If you've enabled automatic scaling, Azure automatically scales your deployment.
For an example of deploying a model as a web service, see Deploy an image classification model in Azure
Container Instances.
IoT module deployments
A deployed IoT module is a Docker container that includes your model and associated script or application and
any additional dependencies. You deploy these modules by using Azure IoT Edge on edge devices.
If you've enabled monitoring, Azure collects telemetry data from the model inside the Azure IoT Edge module.
The telemetry data is accessible only to you, and it's stored in your storage account instance.
Azure IoT Edge ensures that your module is running, and it monitors the device that's hosting it.
ML Pipelines
You use machine learning pipelines to create and manage workflows that stitch together machine learning phases.
For example, a pipeline might include data preparation, model training, model deployment, and inference/scoring
phases. Each phase can encompass multiple steps, each of which can run unattended in various compute targets.
Pipeline steps are reusable, and can be run without rerunning subsequent steps if the output of that step hasn't
changed. For example, you can retrain a model without rerunning costly data preparation steps if the data hasn't
changed. Pipelines also allow data scientists to collaborate while working on separate areas of a machine learning
workflow.
For more information about machine learning pipelines with this service, see Pipelines and Azure Machine
Learning.
Logging
When you develop your solution, use the Azure Machine Learning Python SDK in your Python script to log
arbitrary metrics. After the run, query the metrics to determine whether the run has produced the model you want
to deploy.
Next steps
To get started with Azure Machine Learning service, see:
Create an Azure Machine Learning service workspace
Tutorial (part 1): Train a model
Tutorial: Get started creating your first ML
experiment
In this tutorial, you complete the end-to-end steps to get started with the Azure Machine Learning Python SDK
running in Jupyter notebooks. This tutorial is part one of a two-part tutorial series, and covers Python
environment setup and configuration, as well as creating a workspace to manage your experiments and machine
learning models. Part two builds on this to train multiple machine learning models and introduce the model
management process using both the Azure portal and the SDK.
In this tutorial, you:
Create a machine learning Workspace to use in the next tutorial.
Create a cloud-based Jupyter notebook VM with Azure Machine Learning Python SDK installed and pre-
configured.
Prerequisites
The only prerequisite for this tutorial is an Azure subscription. If you don’t have an Azure subscription, create a
free account before you begin. Try the free or paid version of Azure Machine Learning service today.
Create a workspace
A workspace is a foundational resource in the cloud that you use to experiment, train, and deploy machine
learning models. It ties your Azure subscription and resource group to an easily consumed object in the SDK. If
you already have an Azure Machine Learning service workspace, skip to the next section. Otherwise, create one
now.
1. Sign in to the Azure portal by using the credentials for the Azure subscription you use.
2. In the upper-left corner of Azure portal, select + Create a resource.
3. Use the search bar to find Machine Learning service workspace.
4. Select Machine Learning service workspace.
5. In the Machine Learning service workspace pane, select Create to begin.
6. Configure your new workspace by providing the workspace name, subscription, resource group, and
location.
FIELD DESCRIPTION
Workspace name Enter a unique name that identifies your workspace. In

this example, we use docs-ws. Names must be unique
across the resource group. Use a name that's easy to
recall and to differentiate from workspaces created by
others.
Subscription Select the Azure subscription that you want to use.
Resource group Use an existing resource group in your subscription or

enter a name to create a new resource group. A resource
group holds related resources for an Azure solution. In
this example, we use docs-aml.
Location Select the location closest to your users and the data
resources to create your workspace.
7. After you are finished configuring the workspace, select Create.
WARNING
It can take a several minutes to create your workspace in the cloud.
When the process is finished, a deployment success message appears.

8. To view the new workspace, select Go to resource.
Create a cloud notebook server
This example uses the cloud notebook server in your workspace for an install-free and pre-configured
experience. Use your own environment if you prefer to have control over your environment, packages and
dependencies.
From your workspace, you create a cloud resource to get started using Jupyter notebooks. This resource is a
cloud-based Linux virtual machine pre-configured with everything you need to run Azure Machine Learning
service.
1. Open your workspace in the Azure portal. If you're not sure how to locate your workspace in the portal,
see how to find your workspace.
2. On your workspace page in the Azure portal, select Notebook VMs on the left.
3. Select +New to create a notebook VM.
4. Provide a name for your VM.

Your Notebook VM name must be between 2 to 16 characters. Valid characters are letters, digits, and
the - character.
The name must also be unique across your Azure subscription.
5. Then select Create. It can take a moment to set up your VM.
6. Wait until the status changes to Running.
Launch Jupyter web interface
After your VM is running, use the Notebook VMs section to launch the Jupyter web interface.
1. Select Jupyter in the URI column for your VM.
The link starts your notebook server and opens the Jupyter notebook webpage in a new browser tab. This
link will only work for the person who creates the VM. Each user of the workspace must create their own
VM.
2. On the Jupyter notebook webpage, select the top foldername, which has your username.
This folder exists in the workspace storage account rather than on the notebook VM itself. If you delete
the notebook VM, you'll still keep all your work. When you create a new notebook VM later, it will load
this same folder. If you share your workspace with others, they will see your folder and you will see theirs.
3. Open the samples-* subdirectory, then open the Jupyter notebook
tutorials/tutorial-1st-experiment-sdk-train.ipynb
WARNING
Ensure you open the tutorial-1st-experiment-sdk-train.ipynb file, not the .yml file of the > same name.
Next steps
In this tutorial, you completed these tasks:
Created an Azure Machine Learning service workspace.
Created and configured a cloud notebook server in your workspace.
In part two of the tutorial you run the code in tutorial-1st-experiment-sdk-train.ipynb to train a machine
learning model.
Tutorial: Train your first model
IMPORTANT
If you do not plan on following part 2 of this tutorial or any other tutorials, you should stop the cloud notebook server
VM when you are not using it to reduce cost.
Tutorial: Train your first ML model
This tutorial is part two of a two-part tutorial series. In the previous tutorial, you created a workspace and
chose a development environment. In this tutorial, you learn the foundational design patterns in Azure Machine
Learning service, and train a simple scikit-learn model based on the diabetes data set. After completing this
tutorial, you will have the practical knowledge of the SDK to scale up to developing more-complex experiments
and workflows.
In this tutorial, you learn the following tasks:
Connect your workspace and create an experiment
Load data and train scikit-learn models
View training results in the portal
Retrieve the best model
Prerequisites
The only prerequisite is to run part one of this tutorial, Setup environment and workspace.
In this part of the tutorial, you run the code in the sample Jupyter notebook
tutorials/tutorial-1st-experiment-sdk-train.ipynb opened at the end of part one. This article walks through the
same code that is in the notebook.
Connect workspace and create experiment

Import the Workspace class, and load your subscription information from the file config.json using the function
from_config(). This looks for the JSON file in the current directory by default, but you can also specify a path
parameter to point to the file using from_config(path="your/file/path") . In a cloud notebook server, the file is
automatically in the root directory.
If the following code asks for additional authentication, simply paste the link in a browser and enter the
authentication token.
TIP
If you're new to Jupyter notebooks, run code by clicking on a code cell and hitting Shift+Enter to run the cells one at a time.
Alternatively, to run the entire notebook at once, click Cell on the top menu bar and then click Run All.
from azureml.core import Workspace

ws = Workspace.from_config()
Now create an experiment in your workspace. An experiment is another foundational cloud resource that
represents a collection of trials (individual model runs). In this tutorial you use the experiment to create runs and
track your model training in the Azure portal. Parameters include your workspace reference, and a string name for
the experiment.
from azureml.core import Experiment

experiment = Experiment(workspace=ws, name="diabetes-experiment")
Load data and prepare for training
For this tutorial, you use the diabetes data set, which is a pre-normalized data set included in scikit-learn. This data
set uses features like age, gender, and BMI to predict diabetes disease progression. Load the data from the
load_diabetes() static function, and split it into training and test sets using train_test_split() . This function
segregates the data so the model has unseen data to use for testing following training.
from sklearn.datasets import load_diabetes

from sklearn.model_selection import train_test_split
X, y = load_diabetes(return_X_y = True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=66)
Train a model
Training a simple scikit-learn model can easily be done locally for small-scale training, but when training many
iterations with dozens of different feature permutations and hyperparameter settings, it is easy to lose track of
what models you've trained and how you trained them. The following design pattern shows how to leverage the
SDK to easily keep track of your training in the cloud.
Build a script that trains ridge models in a loop through different hyperparameter alpha values.
from sklearn.linear_model import Ridge

from sklearn.metrics import mean_squared_error
from sklearn.externals import joblib
import math
alphas = [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]
for alpha in alphas:

run = experiment.start_logging()
run.log("alpha_value", alpha)
model = Ridge(alpha=alpha)
model.fit(X=X_train, y=y_train)
y_pred = model.predict(X=X_test)
rmse = math.sqrt(mean_squared_error(y_true=y_test, y_pred=y_pred))
run.log("rmse", rmse)
model_name = "model_alpha_" + str(alpha) + ".pkl"

filename = "outputs/" + model_name
joblib.dump(value=model, filename=filename)
run.upload_file(name=model_name, path_or_stream=filename)
run.complete()
The above code accomplishes the following:

1. For each alpha hyperparameter value in the alphas array, a new run is created within the experiment. The
alpha value is logged to differentiate between each run.
2. In each run, a Ridge model is instantiated, trained, and used to run predictions. The root-mean-squared-error is
calculated for the actual versus predicted values, and then logged to the run. At this point the run has metadata
attached for both the alpha value and the rmse accuracy.
3. Next, the model for each run is serialized and uploaded to the run. This allows you to download the model file
from the run in the portal.
4. At the end of each iteration the run is completed by calling run.complete() .
After the training has completed, call the experiment variable to fetch a link to the experiment in the portal.
experiment
NAME WORKSPACE REPORT PAGE DOCS PAGE
diabetes-experiment your-workspace-name Link to Azure Portal Link to Documentation
View training results in portal

Following the Link to Azure Portal takes you to the main experiment page. Here you see all the individual runs
in the experiment. Any custom-logged values ( alpha_value and rmse , in this case) become fields for each run,
and also become available for the charts and tiles at the top of the experiment page. To add a logged metric to a
chart or tile, hover over it, click the edit button, and find your custom-logged metric.
When training models at scale over hundreds and thousands of separate runs, this page makes it easy to see
every model you trained, specifically how they were trained, and how your unique metrics have changed over
time.
Clicking on a run number link in the RUN NUMBER column takes you to the page for each individual run. The default
tab Details shows you more-detailed information on each run. Navigate to the Outputs tab, and you see the
.pkl file for the model that was uploaded to the run during each training iteration. Here you can download the
model file, rather than having to retrain it manually.
Get the best model
In addition to being able to download model files from the experiment in the portal, you can also download them
programmatically. The following code iterates through each run in the experiment, and accesses both the logged
run metrics and the run details (which contains the run_id). This keeps track of the best run, in this case the run
with the lowest root-mean-squared-error.
minimum_rmse_runid = None
minimum_rmse = None
for run in experiment.get_runs():

run_metrics = run.get_metrics()
run_details = run.get_details()
# each logged metric becomes a key in this returned dict
run_rmse = run_metrics["rmse"]
run_id = run_details["runId"]
if minimum_rmse is None:
minimum_rmse = run_rmse
minimum_rmse_runid = run_id
else:
if run_rmse < minimum_rmse:
minimum_rmse = run_rmse
minimum_rmse_runid = run_id
print("Best run_id: " + minimum_rmse_runid)

print("Best run_id rmse: " + str(minimum_rmse))
Best run_id: 864f5ce7-6729-405d-b457-83250da99c80

Best run_id rmse: 57.234760283951765
Use the best run id to fetch the individual run using the Run constructor along with the experiment object. Then
call get_file_names() to see all the files available for download from this run. In this case, you only uploaded one
file for each run during training.
from azureml.core import Run
best_run = Run(experiment=experiment, run_id=minimum_rmse_runid)
print(best_run.get_file_names())
['model_alpha_0.1.pkl']
Call download() on the run object, specifying the model file name to download. By default this function
downloads to the current directory.
best_run.download_file(name="model_alpha_0.1.pkl")
Clean up resources
Do not complete this section if you plan on running other Azure Machine Learning service tutorials.
Stop the notebook VM
If you used a cloud notebook server, stop the VM when you are not using it to reduce cost.
1. In your workspace, select Notebook VMs.
2. From the list, select the VM.

3. Select Stop.
4. When you're ready to use the server again, select Start.
Delete everything
IMPORTANT
The resources you created can be used as prerequisites to other Azure Machine Learning service tutorials and how-to
articles.
If you don't plan to use the resources you created, delete them, so you don't incur any charges:
1. In the Azure portal, select Resource groups on the far left.
2. From the list, select the resource group you created.
3. Select Delete resource group.
4. Enter the resource group name. Then select Delete.
You can also keep the resource group but delete a single workspace. Display the workspace properties and select
Delete.
Next steps
In this tutorial, you did the following tasks:
Connected your workspace and created an experiment
Loaded data and trained scikit-learn models
Viewed training results in the portal and retrieved models
Deploy your model with Azure Machine Learning. Learn how to develop automated machine learning
experiments.
Tutorial: Train image classification models with
MNIST data and scikit-learn using Azure Machine
Learning
In this tutorial, you train a machine learning model on remote compute resources. You'll use the training and
deployment workflow for Azure Machine Learning service in a Python Jupyter notebook. You can then use the
notebook as a template to train your own machine learning model with your own data. This tutorial is part one
of a two-part tutorial series.
This tutorial trains a simple logistic regression by using the MNIST dataset and scikit-learn with Azure Machine
Learning service. MNIST is a popular dataset consisting of 70,000 grayscale images. Each image is a
handwritten digit of 28 x 28 pixels, representing a number from zero to nine. The goal is to create a multiclass
classifier to identify the digit a given image represents.
Learn how to take the following actions:
Set up your development environment.
Access and examine the data.
Train a simple logistic regression model on a remote cluster.
Review training results and register the best model.
You learn how to select a model and deploy it in part two of this tutorial.
NOTE
Code in this article was tested with Azure Machine Learning SDK version 1.0.57.
Prerequisites
Complete the Tutorial: Get started creating your first ML experiment to:
Create a workspace
Create a cloud notebook server
Launch the Jupyter notebook dashboard
After you launch the Jupyter notebook dashboard, open the tutorials/img-classification-part1-
training.ipynb notebook.
The tutorial and accompanying utils.py file is also available on GitHub if you wish to use it on your own local
environment. Make sure you have installed matplotlib and scikit-learn in your environment.
Set up your development environment

All the setup for your development work can be accomplished in a Python notebook. Setup includes the
following actions:
Import Python packages.
Connect to a workspace, so that your local computer can communicate with remote resources.
Create an experiment to track all your runs.
Create a remote compute target to use for training.
Import packages
Import Python packages you need in this session. Also display the Azure Machine Learning SDK version:
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import azureml.core
# check core SDK version number

print("Azure ML SDK Version: ", azureml.core.VERSION)
Connect to a workspace
Create a workspace object from the existing workspace. Workspace.from_config() reads the file config.json
and loads the details into an object named ws :
# load workspace configuration from the config.json file in the current folder.
print(ws.name, ws.location, ws.resource_group, sep='\t')
Create an experiment
Create an experiment to track the runs in your workspace. A workspace can have multiple experiments:

experiment_name = 'sklearn-mnist'
exp = Experiment(workspace=ws, name=experiment_name)
Create or attach an existing compute target

By using Azure Machine Learning Compute, a managed service, data scientists can train machine learning
models on clusters of Azure virtual machines. Examples include VMs with GPU support. In this tutorial, you
create Azure Machine Learning Compute as your training environment. The code below creates the compute
clusters for you if they don't already exist in your workspace.
Creation of the compute target takes about five minutes. If the compute resource is already in the
workspace, the code uses it and skips the creation process.
from azureml.core.compute import AmlCompute
from azureml.core.compute import ComputeTarget
import os
# choose a name for your cluster

compute_name = os.environ.get("AML_COMPUTE_CLUSTER_NAME", "cpucluster")
compute_min_nodes = os.environ.get("AML_COMPUTE_CLUSTER_MIN_NODES", 0)
compute_max_nodes = os.environ.get("AML_COMPUTE_CLUSTER_MAX_NODES", 4)
# This example uses CPU VM. For using GPU VM, set SKU to STANDARD_NC6
vm_size = os.environ.get("AML_COMPUTE_CLUSTER_SKU", "STANDARD_D2_V2")
if compute_name in ws.compute_targets:
compute_target = ws.compute_targets[compute_name]
if compute_target and type(compute_target) is AmlCompute:
print('found compute target. just use it. ' + compute_name)
else:
print('creating a new compute target...')
provisioning_config = AmlCompute.provisioning_configuration(vm_size=vm_size,
min_nodes=compute_min_nodes,
max_nodes=compute_max_nodes)
# create the cluster

compute_target = ComputeTarget.create(
ws, compute_name, provisioning_config)
# can poll for a minimum number of nodes and for a specific timeout.
# if no min node count is provided it will use the scale settings for the cluster
compute_target.wait_for_completion(
show_output=True, min_node_count=None, timeout_in_minutes=20)
# For a more detailed view of current AmlCompute status, use get_status()

print(compute_target.get_status().serialize())
You now have the necessary packages and compute resources to train a model in the cloud.
Explore data
Before you train a model, you need to understand the data that you use to train it. You also need to copy the
data into the cloud. Then it can be accessed by your cloud training environment. In this section, you learn how
to take the following actions:
Download the MNIST dataset.
Display some sample images.
Upload data to the cloud.
Download the MNIST dataset
Download the MNIST dataset and save the files into a data directory locally. Images and labels for both
training and testing are downloaded:
import urllib.request
import os
data_folder = os.path.join(os.getcwd(), 'data')

os.makedirs(data_folder, exist_ok=True)
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz',
filename=os.path.join(data_folder, 'train-images.gz'))
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz',
filename=os.path.join(data_folder, 'train-labels.gz'))
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz',
filename=os.path.join(data_folder, 'test-images.gz'))
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz',
filename=os.path.join(data_folder, 'test-labels.gz'))
You will see output similar to this: ('./data/test-labels.gz', <http.client.HTTPMessage at 0x7f40864c77b8>)
Display some sample images

Load the compressed files into numpy arrays. Then use matplotlib to plot 30 random images from the dataset
with their labels above them. This step requires a load_data function that's included in an util.py file. This
file is included in the sample folder. Make sure it's placed in the same folder as this notebook. The load_data
function simply parses the compressed files into numpy arrays:
# make sure utils.py is in the same directory as this code

from utils import load_data
# note we also shrink the intensity values (X) from 0-255 to 0-1. This helps the model converge faster.
X_train = load_data(os.path.join(
data_folder, 'train-images.gz'), False) / 255.0
X_test = load_data(os.path.join(data_folder, 'test-images.gz'), False) / 255.0
y_train = load_data(os.path.join(
data_folder, 'train-labels.gz'), True).reshape(-1)
y_test = load_data(os.path.join(
data_folder, 'test-labels.gz'), True).reshape(-1)
# now let's show some randomly chosen images from the traininng set.
count = 0
sample_size = 30
plt.figure(figsize=(16, 6))
for i in np.random.permutation(X_train.shape[0])[:sample_size]:
count = count + 1
plt.subplot(1, sample_size, count)
plt.axhline('')
plt.axvline('')
plt.text(x=10, y=-10, s=y_train[i], fontsize=18)
plt.imshow(X_train[i].reshape(28, 28), cmap=plt.cm.Greys)
plt.show()
A random sample of images displays:
Now you have an idea of what these images look like and the expected prediction outcome.
Upload data to the cloud
You downloaded and used the training data on the computer your notebook is running on. In the next section,
you will train a model on the remote Azure Machine Learning Compute. The remote compute resource will
also need access to your data. To provide access, upload your data to a centralized datastore associated with
your workspace. This datastore provides fast access to your data when using remote compute targets in the
cloud, as it is in the Azure data center.
Upload the MNIST files into a directory named mnist at the root of the datastore. See access data from your
datastores for more information.
ds = ws.get_default_datastore()
print(ds.datastore_type, ds.account_name, ds.container_name)
ds.upload(src_dir=data_folder, target_path='mnist',
overwrite=True, show_progress=True)
You now have everything you need to start training a model.
Train on a remote cluster

For this task, submit the job to the remote training cluster you set up earlier. To submit a job you:
Create a directory
Create a training script
Create an estimator object
Submit the job
Create a directory
Create a directory to deliver the necessary code from your computer to the remote resource.
import os
script_folder = os.path.join(os.getcwd(), "sklearn-mnist")
os.makedirs(script_folder, exist_ok=True)
Create a training script

To submit the job to the cluster, first create a training script. Run the following code to create the training script
called train.py in the directory you just created.
%%writefile $script_folder/train.py
import argparse
import os
import numpy as np
from sklearn.linear_model import LogisticRegression


# let user feed in 2 parameters, the location of the data files (from datastore), and the regularization
rate of the logistic regression model
parser = argparse.ArgumentParser()
parser.add_argument('--data-folder', type=str, dest='data_folder', help='data folder mounting point')
parser.add_argument('--regularization', type=float, dest='reg', default=0.01, help='regularization rate')
args = parser.parse_args()
data_folder = args.data_folder
print('Data folder:', data_folder)
# load train and test set into numpy arrays

# note we scale the pixel intensity values to 0-1 (by dividing it with 255.0) so the model can converge
faster.
X_train = load_data(os.path.join(data_folder, 'train-images.gz'), False) / 255.0
y_train = load_data(os.path.join(data_folder, 'train-labels.gz'), True).reshape(-1)
y_test = load_data(os.path.join(data_folder, 'test-labels.gz'), True).reshape(-1)
print(X_train.shape, y_train.shape, X_test.shape, y_test.shape, sep = '\n')
# get hold of the current run

run = Run.get_context()
print('Train a logistic regression model with regularization rate of', args.reg)

clf = LogisticRegression(C=1.0/args.reg, solver="liblinear", multi_class="auto", random_state=42)
clf.fit(X_train, y_train)
print('Predict the test set')

y_hat = clf.predict(X_test)
# calculate accuracy on the prediction

acc = np.average(y_hat == y_test)
print('Accuracy is', acc)
run.log('regularization rate', np.float(args.reg))

run.log('accuracy', np.float(acc))
os.makedirs('outputs', exist_ok=True)
# note file saved in the outputs folder is automatically uploaded into experiment record
joblib.dump(value=clf, filename='outputs/sklearn_mnist_model.pkl')
Notice how the script gets data and saves models:

The training script reads an argument to find the directory that contains the data. When you submit the
job later, you point to the datastore for this argument:
parser.add_argument('--data-folder', type=str, dest='data_folder', help='data directory mounting
point')
The training script saves your model into a directory named outputs. Anything written in this directory
is automatically uploaded into your workspace. You access your model from this directory later in the
tutorial. joblib.dump(value=clf, filename='outputs/sklearn_mnist_model.pkl')
The training script requires the file utils.py to load the dataset correctly. The following code copies
utils.py into script_folder so that the file can be accessed along with the training script on the
remote resource.
import shutil
shutil.copy('utils.py', script_folder)
Create an estimator
An SKLearn estimator object is used to submit the run. Create your estimator by running the following code to
define these items:
The name of the estimator object, est .
The directory that contains your scripts. All the files in this directory are uploaded into the cluster nodes for
execution.
The compute target. In this case, you use the Azure Machine Learning compute cluster you created.
The training script name, train.py.
Parameters required from the training script.
In this tutorial, this target is AmlCompute. All files in the script folder are uploaded into the cluster nodes for
run. The data_folder is set to use the datastore, ds.path('mnist').as_mount() :
from azureml.train.sklearn import SKLearn
script_params = {
'--data-folder': ds.path('mnist').as_mount(),
'--regularization': 0.5
}
est = SKLearn(source_directory=script_folder,
script_params=script_params,
compute_target=compute_target,
entry_script='train.py')
Submit the job to the cluster

Run the experiment by submitting the estimator object:
run = exp.submit(config=est)
run
Because the call is asynchronous, it returns a Preparing or Running state as soon as the job is started.
Monitor a remote run

In total, the first run takes about 10 minutes. But for subsequent runs, as long as the script dependencies don't
change, the same image is reused. So the container startup time is much faster.
What happens while you wait:
Image creation: A Docker image is created that matches the Python environment specified by the
estimator. The image is uploaded to the workspace. Image creation and uploading takes about five
minutes.
This stage happens once for each Python environment because the container is cached for subsequent
runs. During image creation, logs are streamed to the run history. You can monitor the image creation
progress by using these logs.
Scaling: If the remote cluster requires more nodes to do the run than currently available, additional
nodes are added automatically. Scaling typically takes about five minutes.
Running: In this stage, the necessary scripts and files are sent to the compute target. Then datastores
are mounted or copied. And then the entry_script is run. While the job is running, stdout and the
./logs directory are streamed to the run history. You can monitor the run's progress by using these logs.
Post-processing: The ./outputs directory of the run is copied over to the run history in your
workspace, so you can access these results.
You can check the progress of a running job in several ways. This tutorial uses a Jupyter widget and a
wait_for_completion method.
Jupyter widget
Watch the progress of the run with a Jupyter widget. Like the run submission, the widget is asynchronous and
provides live updates every 10 to 15 seconds until the job finishes:
from azureml.widgets import RunDetails

RunDetails(run).show()
The widget will look like the following at the end of training:
If you need to cancel a run, you can follow these instructions.

Get log results upon completion
Model training and monitoring happen in the background. Wait until the model has finished training before
you run more code. Use wait_for_completion to show when the model training is finished:
run.wait_for_completion(show_output=False) # specify True for a verbose log
Display run results

You now have a model trained on a remote cluster. Retrieve the accuracy of the model:
print(run.get_metrics())
The output shows the remote model has accuracy of 0.9204:

{'regularization rate': 0.8, 'accuracy': 0.9204}
In the next tutorial, you explore this model in more detail.
Register model
The last step in the training script wrote the file outputs/sklearn_mnist_model.pkl in a directory named
outputs in the VM of the cluster where the job is run. outputs is a special directory in that all content in this
directory is automatically uploaded to your workspace. This content appears in the run record in the
experiment under your workspace. So the model file is now also available in your workspace.
You can see files associated with that run:
print(run.get_file_names())
Register the model in the workspace, so that you or other collaborators can later query, examine, and deploy
this model:
# register model
model = run.register_model(model_name='sklearn_mnist',
model_path='outputs/sklearn_mnist_model.pkl')
print(model.name, model.id, model.version, sep='\t')
Clean up resources
IMPORTANT
articles.

You can also delete just the Azure Machine Learning Compute cluster. However, autoscale is turned on, and the
cluster minimum is zero. So this particular resource won't incur additional compute charges when not in use:
# optionally, delete the Azure Machine Learning Compute cluster

compute_target.delete()
Next steps
In this Azure Machine Learning service tutorial, you used Python for the following tasks:
Set up your development environment.
Access and examine the data.
Train multiple models on a remote cluster using the popular scikit-learn machine learning library
Review training details and register the best model.
You're ready to deploy this registered model by using the instructions in the next part of the tutorial series:
Tutorial 2 - Deploy models
Tutorial: Deploy an image classification model in
Azure Container Instances
This tutorial is part two of a two-part tutorial series. In the previous tutorial, you trained machine learning
models and then registered a model in your workspace on the cloud.
Now you're ready to deploy the model as a web service in Azure Container Instances. A web service is an image,
in this case a Docker image. It encapsulates the scoring logic and the model itself.
In this part of the tutorial, you use Azure Machine Learning service for the following tasks:
Set up your testing environment.
Retrieve the model from your workspace.
Test the model locally.
Deploy the model to Container Instances.
Test the deployed model.
Container Instances is a great solution for testing and understanding the workflow. For scalable production
deployments, consider using Azure Kubernetes Service. For more information, see how to deploy and where.
NOTE
Prerequisites
To run the notebook, first complete the model training in Tutorial (part 1): Train an image classification model.
Then open the tutorials/img-classification-part2-deploy.ipynb notebook using the same notebook server.
This tutorial is also available on GitHub if you wish to use it on your own local environment. Make sure you have
installed matplotlib and scikit-learn in your environment.
Set up the environment

Start by setting up a testing environment.
Import packages
Import the Python packages needed for this tutorial:
%matplotlib inline
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import azureml
from azureml.core import Workspace, Run
# display the core SDK version number

print("Azure ML SDK Version: ", azureml.core.VERSION)
Retrieve the model
You registered a model in your workspace in the previous tutorial. Now load this workspace and download the
model to your local directory:

from azureml.core.model import Model
import os
model = Model(ws, 'sklearn_mnist')
model.download(target_dir=os.getcwd(), exist_ok=True)
# verify the downloaded model file

file_path = os.path.join(os.getcwd(), "sklearn_mnist_model.pkl")
os.stat(file_path)
Test the model locally

Before you deploy, make sure your model is working locally:
Load test data.
Predict test data.
Examine the confusion matrix.
Load test data
Load the test data from the ./data/ directory created during the training tutorial:

import os
data_folder = os.path.join(os.getcwd(), 'data')

# note we also shrink the intensity values (X) from 0-255 to 0-1. This helps the neural network converge
faster
y_test = load_data(os.path.join(
data_folder, 'test-labels.gz'), True).reshape(-1)
Predict test data

To get predictions, feed the test dataset to the model:
import pickle
clf = joblib.load(os.path.join(os.getcwd(), 'sklearn_mnist_model.pkl'))

y_hat = clf.predict(X_test)
Examine the confusion matrix

Generate a confusion matrix to see how many samples from the test set are classified correctly. Notice the
misclassified value for the incorrect predictions:
from sklearn.metrics import confusion_matrix
conf_mx = confusion_matrix(y_test, y_hat)

print(conf_mx)
print('Overall accuracy:', np.average(y_hat == y_test))
The output shows the confusion matrix:
[[ 960 0 1 2 1 5 6 3 1 1]
[ 0 1112 3 1 0 1 5 1 12 0]
[ 9 8 920 20 10 4 10 11 37 3]
[ 4 0 17 921 2 21 4 12 20 9]
[ 1 2 5 3 915 0 10 2 6 38]
[ 10 2 0 41 10 770 17 7 28 7]
[ 9 3 7 2 6 20 907 1 3 0]
[ 2 7 22 5 8 1 1 950 5 27]
[ 10 15 5 21 15 27 7 11 851 12]
[ 7 8 2 13 32 13 0 24 12 898]]
Overall accuracy: 0.9204
Use matplotlib to display the confusion matrix as a graph. In this graph, the x-axis shows the actual values, and
the y-axis shows the predicted values. The color in each grid shows the error rate. The lighter the color, the higher
the error rate is. For example, many 5's are misclassified as 3's. So you see a bright grid at (5,3):
# normalize the diagonal cells so that they don't overpower the rest of the cells when visualized
row_sums = conf_mx.sum(axis=1, keepdims=True)
norm_conf_mx = conf_mx / row_sums
np.fill_diagonal(norm_conf_mx, 0)
fig = plt.figure(figsize=(8, 5))

ax = fig.add_subplot(111)
cax = ax.matshow(norm_conf_mx, cmap=plt.cm.bone)
ticks = np.arange(0, 10, 1)
ax.set_xticks(ticks)
ax.set_yticks(ticks)
ax.set_xticklabels(ticks)
ax.set_yticklabels(ticks)
fig.colorbar(cax)
plt.ylabel('true labels', fontsize=14)
plt.xlabel('predicted values', fontsize=14)
plt.savefig('conf.png')
plt.show()
Deploy as a web service

After you tested the model and you're satisfied with the results, deploy the model as a web service hosted in
Container Instances.
To build the correct environment for Container Instances, provide the following components:
A scoring script to show how to use the model.
An environment file to show what packages need to be installed.
A configuration file to build the container instance.
The model you trained previously.
Create scoring script
Create the scoring script, called score.py. The web service call uses this script to show how to use the model.
Include these two required functions in the scoring script:
The init() function, which typically loads the model into a global object. This function is run only once
when the Docker container is started.
The run(input_data) function uses the model to predict a value based on the input data. Inputs and
outputs to the run typically use JSON for serialization and de-serialization, but other formats are
supported.
%%writefile score.py
import json
import numpy as np
import os
import pickle
def init():
global model
# retrieve the path to the model file using the model name
model_path = Model.get_model_path('sklearn_mnist')
model = joblib.load(model_path)
def run(raw_data):
data = np.array(json.loads(raw_data)['data'])
# make prediction
y_hat = model.predict(data)
# you can return any data type as long as it is JSON-serializable
return y_hat.tolist()
Create environment file

Next create an environment file, called myenv.yml, that specifies all of the script's package dependencies. This
file is used to make sure that all of those dependencies are installed in the Docker image. This model needs
scikit-learn and azureml-sdk :
from azureml.core.conda_dependencies import CondaDependencies
myenv = CondaDependencies()
myenv.add_conda_package("scikit-learn")
with open("myenv.yml", "w") as f:

f.write(myenv.serialize_to_string())
Review the content of the myenv.yml file:
with open("myenv.yml", "r") as f:

print(f.read())
Create a configuration file

Create a deployment configuration file. Specify the number of CPUs and gigabytes of RAM needed for your
Container Instances container. Although it depends on your model, the default of one core and 1 gigabyte of
RAM is sufficient for many models. If you need more later, you have to re-create the image and redeploy the
service.
from azureml.core.webservice import AciWebservice
aciconfig = AciWebservice.deploy_configuration(cpu_cores=1,
memory_gb=1,
tags={"data": "MNIST",
"method": "sklearn"},
description='Predict MNIST with sklearn')
Deploy in Container Instances

The estimated time to finish deployment is about seven to eight minutes.
Configure the image and deploy. The following code goes through these steps:
1. Build an image by using these files:
The scoring file, score.py .
The environment file, myenv.yml .
The model file.
2. Register the image under the workspace.
3. Send the image to the Container Instances container.
4. Start up a container in Container Instances by using the image.
5. Get the web service HTTP endpoint.
%%time
from azureml.core.webservice import Webservice
from azureml.core.model import InferenceConfig
inference_config = InferenceConfig(runtime= "python",

entry_script="score.py",
conda_file="myenv.yml")
service = Model.deploy(workspace=ws,
name='sklearn-mnist-svc',
models=[model],
inference_config=inference_config,
deployment_config=aciconfig)
service.wait_for_deployment(show_output=True)
Get the scoring web service's HTTP endpoint, which accepts REST client calls. You can share this endpoint with
anyone who wants to test the web service or integrate it into an application:
print(service.scoring_uri)
Test the deployed service

Earlier, you scored all the test data with the local version of the model. Now you can test the deployed model with
a random sample of 30 images from the test data.
The following code goes through these steps:
1. Send the data as a JSON array to the web service hosted in Container Instances.
2. Use the SDK's run API to invoke the service. You can also make raw calls by using any HTTP tool such as
curl.
3. Print the returned predictions and plot them along with the input images. Red font and inverse image,
white on black, is used to highlight the misclassified samples.
Because the model accuracy is high, you might have to run the following code a few times before you can see a
misclassified sample:
import json
# find 30 random samples from test set

n = 30
sample_indices = np.random.permutation(X_test.shape[0])[0:n]
test_samples = json.dumps({"data": X_test[sample_indices].tolist()})

test_samples = bytes(test_samples, encoding='utf8')
# predict using the deployed model

result = service.run(input_data=test_samples)
# compare actual value vs. the predicted values:

i = 0
plt.figure(figsize=(20, 1))
for s in sample_indices:
plt.subplot(1, n, i + 1)
plt.axhline('')
plt.axvline('')
# use different color for misclassified sample

font_color = 'red' if y_test[s] != result[i] else 'black'
clr_map = plt.cm.gray if y_test[s] != result[i] else plt.cm.Greys
plt.text(x=10, y=-10, s=result[i], fontsize=18, color=font_color)

plt.imshow(X_test[s].reshape(28, 28), cmap=clr_map)
i = i + 1
plt.show()
This result is from one random sample of test images:
You can also send a raw HTTP request to test the web service:
import requests
# send a random row from the test set to score

random_index = np.random.randint(0, len(X_test)-1)
input_data = "{\"data\": [" + str(list(X_test[random_index])) + "]}"
headers = {'Content-Type': 'application/json'}
# for AKS deployment you'd need to the service key in the header as well
# api_key = service.get_key()
# headers = {'Content-Type':'application/json', 'Authorization':('Bearer '+ api_key)}
resp = requests.post(service.scoring_uri, input_data, headers=headers)
print("POST to url", service.scoring_uri)

#print("input data:", input_data)
print("label:", y_test[random_index])
print("prediction:", resp.text)
Clean up resources
To keep the resource group and workspace for other tutorials and exploration, you can delete only the Container
Instances deployment by using this API call:
service.delete()
IMPORTANT
articles.
Next steps
Learn about all of the deployment options for Azure Machine Learning service.
Learn how to create clients for the web service.
Make predictions on large quantities of data asynchronously.
Monitor your Azure Machine Learning models with Application Insights.
Try out the automatic algorithm selection tutorial.
Tutorial: Use automated machine learning to predict
taxi fares
In this tutorial, you use automated machine learning in Azure Machine Learning service to create a regression
model to predict NYC taxi fare prices. This process accepts training data and configuration settings, and
automatically iterates through combinations of different feature normalization/standardization methods, models,
and hyperparameter settings to arrive at the best model.
In this tutorial you learn the following tasks:

Download, transform, and clean data using Azure Open Datasets
Train an automated machine learning regression model
Calculate model accuracy
Prerequisites
Complete the setup tutorial if you don't already have an Azure Machine Learning service workspace or
notebook virtual machine.
After you complete the setup tutorial, open the tutorials/regression-automated-ml.ipynb notebook using
the same notebook server.
This tutorial is also available on GitHub if you wish to run it in your own local environment. Run
pip install azureml-sdk[automl] azureml-opendatasets azureml-widgets to get the required packages.
Download and prepare data

Import the necessary packages. The Open Datasets package contains a class representing each data source (
NycTlcGreen for example) to easily filter date parameters before downloading.
from azureml.opendatasets import NycTlcGreen

import pandas as pd
from datetime import datetime
from dateutil.relativedelta import relativedelta
Begin by creating a dataframe to hold the taxi data. When working in a non-Spark environment, Open Datasets
only allows downloading one month of data at a time with certain classes to avoid MemoryError with large
datasets.
To download taxi data, iteratively fetch one month at a time, and before appending it to green_taxi_df randomly
sample 2,000 records from each month to avoid bloating the dataframe. Then preview the data.
green_taxi_df = pd.DataFrame([])
start = datetime.strptime("1/1/2015","%m/%d/%Y")
end = datetime.strptime("1/31/2015","%m/%d/%Y")
for sample_month in range(12):

temp_df_green = NycTlcGreen(start + relativedelta(months=sample_month), end +
relativedelta(months=sample_month)) \
.to_pandas_dataframe()
green_taxi_df = green_taxi_df.append(temp_df_green.sample(2000))
green_taxi_df.head(10)
I
L M
L P P
P E R
E P D O
P D P R V
P R P I P O E
I O A C I P M
C P S T P D K C O E
K O S R U O U K F P N T T
U F E I L L P U F A F T O O
P F N P O O L P L Y A S T L T
V D D G D C C O L O M R U I L E A T
E A A E I A A N A N E E R P S H L R
N T T R S T T G T G N A M C A A A A I
D E E C T I I I I I T M E T H M M I M P
O T T O A O O T T T T O X A A O O L O T
R I I U N N N U U U Y U T T R U U F U Y
I M M N C I I D D D P N R A G N N E N P
D E E T E D D E E E ... E T A X E T T E T E
2 2 2 3 4 N N - 4 - ... 2 1 0 0 0 0 0 n 1 1
0 0 . o o 7 0 7 5 . . . . . a 6 .
1 1 8 n n 3 . 3 . 5 5 3 0 0 n . 0
5 5 4 e e . 8 . 0 0 0 0 0 3 0
- - 8 4 9 0 0
0 0 8 4
1
1 1
3 - -
1 1 1
9 1 1
6 0 0
9
5 5
: :
3 4
4 5
: :
4 0
4 3
I
L M
L P P
P E R
E P D O
P D P R V
P R P I P O E
I O A C I P M
C P S T P D K C O E
2 2 2 1 0 N N - 4 - ... 2 4 1 0 0 0 0 n 6 1
0 0 . o o 7 0 7 . . . . . . a . .
1 1 6 n n 3 . 3 5 0 5 3 0 0 n 3 0
5 5 9 e e . 8 . 0 0 0 0 0 0 0
- - 9 1 9
0 0 6 6
1 1 1
1 - -
2 2 2
9
8 0 0
1 1 1
7 6 6
: :
2 3
6 0
: :
2 2
9 6
2 2 2 1 0 N N - 4 - ... 2 4 0 0 0 0 0 n 4 1
0 0 . o o 7 0 7 . . . . . . a . .
1 1 4 n n 3 . 3 0 0 5 3 0 0 n 8 0
5 5 5 e e . 7 . 0 0 0 0 0 0 0
- - 9 6 9
0 0 2 1
1 1 1
2 - -
7 0 0
8
6 1 1
2 0 0
0 5 6
: :
5 0
8 0
: :
1 5
0 5
I
L M
L P P
P E R
E P D O
P D P R V
P R P I P O E
I O A C I P M
C P S T P D K C O E
2 2 2 1 0 N N - 4 - ... 2 1 0 0 0 0 0 n 1 1
0 0 . o o 7 0 7 2 . . . . . a 3 .
1 1 0 n n 3 . 3 . 5 5 3 0 0 n . 0
5 5 0 e e . 7 . 5 0 0 0 0 8 0
- - 8 0 8 0 0
0 0 1 2
3
1 1
4 - -
8 1 1
4 7 7
3 0 0
0
2 2
: :
2 4
0 1
: :
5 3
0 8
1 2 2 1 0 N N - 4 - ... 2 4 0 0 0 0 0 n 5 1
0 0 . o o 7 0 7 . . . . . a . .
1 1 5 n n 3 . 3 0 5 5 0 0 n 0 0
5 5 0 e e . 7 . 0 0 0 0 0 0 0
- - 9 6 9
0 0 2 2
1 1 1
2 - -
6 0 0
9
6 1 1
2 0 0
7 5 5
: :
0 0
4 6
: :
1 2
0 3
I
L M
L P P
P E R
E P D O
P D P R V
P R P I P O E
I O A C I P M
C P S T P D K C O E
1 2 2 2 1 N N - 4 - ... 2 6 0 0 0 0 0 n 7 1
0 0 . o o 7 0 7 . . . . . . a . .
1 1 1 n n 3 . 3 5 5 5 3 0 0 n 8 0
5 5 0 e e . 7 . 0 0 0 0 0 0 0
- - 9 2 9
0 0 6 5
8
1 1
1 - -
1 0 0
7 4 4
5 1 2
5
9 0
: :
5 0
7 5
: :
5 4
1 5
1 2 2 1 0 N N - 4 - ... 2 6 0 0 0 0 0 n 6 1
0 0 . o o 7 0 7 . . . . . . a . .
1 1 9 n n 3 . 3 0 0 5 3 0 0 n 8 0
5 5 0 e e . 7 . 0 0 0 0 0 0 0
- - 8 6 8
0 0 8 7
7
1 1
3 - -
7 0 0
2 3 3
8 1 1
1
2 2
: :
2 3
7 3
: :
3 5
1 2
I
L M
L P P
P E R
E P D O
P D P R V
P R P I P O E
I O A C I P M
C P S T P D K C O E
1 2 2 1 3 N N - 4 - ... 2 1 0 0 0 0 0 n 1 1
0 0 . o o 7 0 7 2 . . . . . a 3 .
1 1 3 n n 3 . 3 . 5 5 3 0 0 n . 0
5 5 0 e e . 7 . 5 0 0 0 0 8 0
- - 9 2 9 0 0
0 0 6 1
1
1 1
1 - -
3 0 0
9 9 9
5 2 2
1
3 3
: :
2 3
5 9
: :
5 5
1 2
2 2 2 1 1 N N - 4 - ... 1 7 0 0 0 1 0 n 9 1
0 0 . o o 7 0 7 . . . . . . a . .
1 1 1 n n 3 . 3 0 0 5 3 7 0 n 5 0
5 5 9 e e . 7 . 0 0 0 5 0 5 0
- - 9 1 9
0 0 4 5
1
1 1
5 - -
0 1 1
4 1 1
3 1 1
6
7 7
: :
1 2
5 2
: :
1 5
4 7
I
L M
L P P
P E R
E P D O
P D P R V
P R P I P O E
I O A C I P M
C P S T P D K C O E
2 2 2 1 0 N N - 4 - ... 2 5 0 0 0 0 0 n 6 1
0 0 . o o 7 0 7 . . . . . . a . .
1 1 6 n n 3 . 3 0 5 5 3 0 0 n 3 0
5 5 5 e e . 7 . 0 0 0 0 0 0 0
- - 9 1 9
0 0 4 4
4
1 1
3 - -
2 2 2
1 2 2
3 2 2
6
3 3
: :
1 2
6 0
: :
3 1
3 3
10 rows × 23 columns
Now that the initial data is loaded, define a function to create various time-based features from the pickup
datetime field. This will create new fields for the month number, day of month, day of week, and hour of day, and
will allow the model to factor in time-based seasonality. Use the apply() function on the dataframe to iteratively
apply the build_time_features() function to each row in the taxi data.
def build_time_features(vector):
pickup_datetime = vector[0]
month_num = pickup_datetime.month
day_of_month = pickup_datetime.day
day_of_week = pickup_datetime.weekday()
hour_of_day = pickup_datetime.hour
return pd.Series((month_num, day_of_month, day_of_week, hour_of_day))
green_taxi_df[["month_num", "day_of_month","day_of_week", "hour_of_day"]] =

green_taxi_df[["lpepPickupDatetime"]].apply(build_time_features, axis=1)
I
M
L P
L P R
P E D O
E P P R V
P D I P O E
P R P C I P M
I O A K C O E
C P S T P D U K F N D
K O S R U O P U F T T T A D H
U F E I L L L P L S O O Y A O
P F N P O O O L O U T L T M _ Y U
V D D G D C C N A N R I L E A T O O _ R
E A A E I A A G T G C P S H L R N F O _
N T T R S T T I I I H A A A A I T _ F O
D E E C T I I T T T A M M I M P H M _ F
O T T O A O O U U U R O O L O T _ O W _
R I I U N N N D D D G U U F U Y N N E D
I M M N C I I E E E E N N E N P U T E A
D E E T E D D ... T T E T E M H K Y
2 2 2 3 4 N N - 4 - ... 0 0 0 n 1 1 1 1 6 5
0 0 . o o 7 0 7 . . . a 6 . 1
1 1 8 n n 3 . 3 3 0 0 n . 0
5 5 4 e e . 8 . 0 0 3 0
- - 8 4 9 0
0 0 8 4
1
1 1
3 - -
1 1 1
9 1 1
6 0 0
9
5 5
: :
3 4
4 5
: :
4 0
4 3
2 2 2 1 0 N N - 4 - ... 0 0 0 n 6 1 1 2 1 1
0 0 . o o 7 0 7 . . . a . . 0 6
1 1 6 n n 3 . 3 3 0 0 n 3 0
5 5 9 e e . 8 . 0 0 0 0
- - 9 1 9
0 0 6 6
1 1 1
1 - -
2 2 2
9
8 0 0
1 1 1
7 6 6
: :
2 3
6 0
: :
2 2
9 6
I
M
L P
L P R
P E D O
E P P R V
P D I P O E
P R P C I P M
I O A K C O E
2 2 2 1 0 N N - 4 - ... 0 0 0 n 4 1 1 1 3 5
0 0 . o o 7 0 7 . . . a . .
1 1 4 n n 3 . 3 3 0 0 n 8 0
5 5 5 e e . 7 . 0 0 0 0
- - 9 6 9
0 0 2 1
1 1 1
2 - -
7 0 0
8
6 1 1
2 0 0
0 5 6
: :
5 0
8 0
: :
1 5
0 5
2 2 2 1 0 N N - 4 - ... 0 0 0 n 1 1 1 1 5 2
0 0 . o o 7 0 7 . . . a 3 . 7
1 1 0 n n 3 . 3 3 0 0 n . 0
5 5 0 e e . 7 . 0 0 8 0
- - 8 0 8 0
0 0 1 2
3
1 1
4 - -
8 1 1
4 7 7
3 0 0
0
2 2
: :
2 4
0 1
: :
5 3
0 8
I
M
L P
L P R
P E D O
E P P R V
P D I P O E
P R P C I P M
I O A K C O E
1 2 2 1 0 N N - 4 - ... 0 0 0 n 5 1 1 1 3 5
0 0 . o o 7 0 7 . . a . .
1 1 5 n n 3 . 3 0 0 n 0 0
5 5 0 e e . 7 . 0 0 0 0
- - 9 6 9
0 0 2 2
1 1 1
2 - -
6 0 0
9
6 1 1
2 0 0
7 5 5
: :
0 0
4 6
: :
1 2
0 3
1 2 2 2 1 N N - 4 - ... 0 0 0 n 7 1 1 4 6 1
0 0 . o o 7 0 7 . . . a . . 9
1 1 1 n n 3 . 3 3 0 0 n 8 0
5 5 0 e e . 7 . 0 0 0 0
- - 9 2 9
0 0 6 5
8
1 1
1 - -
1 0 0
7 4 4
5 1 2
5
9 0
: :
5 0
7 5
: :
5 4
1 5
I
M
L P
L P R
P E D O
E P P R V
P D I P O E
P R P C I P M
I O A K C O E
1 2 2 1 0 N N - 4 - ... 0 0 0 n 6 1 1 3 5 1
0 0 . o o 7 0 7 . . . a . . 2
1 1 9 n n 3 . 3 3 0 0 n 8 0
5 5 0 e e . 7 . 0 0 0 0
- - 8 6 8
0 0 8 7
7
1 1
3 - -
7 0 0
2 3 3
8 1 1
1
2 2
: :
2 3
7 3
: :
3 5
1 2
1 2 2 1 3 N N - 4 - ... 0 0 0 n 1 1 1 9 4 2
0 0 . o o 7 0 7 . . . a 3 . 3
1 1 3 n n 3 . 3 3 0 0 n . 0
5 5 0 e e . 7 . 0 0 8 0
- - 9 2 9 0
0 0 6 1
1
1 1
1 - -
3 0 0
9 9 9
5 2 2
1
3 3
: :
2 3
5 9
: :
5 5
1 2
I
M
L P
L P R
P E D O
E P P R V
P D I P O E
P R P C I P M
I O A K C O E
2 2 2 1 1 N N - 4 - ... 0 1 0 n 9 1 1 1 6 1
0 0 . o o 7 0 7 . . . a . . 1 7
1 1 1 n n 3 . 3 3 7 0 n 5 0
5 5 9 e e . 7 . 5 0 5 0
- - 9 1 9
0 0 4 5
1
1 1
5 - -
0 1 1
4 1 1
3 1 1
6
7 7
: :
1 2
5 2
: :
1 5
4 7
2 2 2 1 0 N N - 4 - ... 0 0 0 n 6 1 1 2 3 2
0 0 . o o 7 0 7 . . . a . . 2 3
1 1 6 n n 3 . 3 3 0 0 n 3 0
5 5 5 e e . 7 . 0 0 0 0
- - 9 1 9
0 0 4 4
4
1 1
3 - -
2 2 2
1 2 2
3 2 2
6
3 3
: :
1 2
6 0
: :
3 1
3 3
10 rows × 27 columns
Remove some of the columns that you won't need for training or additional feature building.
columns_to_remove = ["lpepPickupDatetime", "lpepDropoffDatetime", "puLocationId", "doLocationId", "extra",
"mtaTax",
"improvementSurcharge", "tollsAmount", "ehailFee", "tripType", "rateCodeID",
"storeAndFwdFlag", "paymentType", "fareAmount", "tipAmount"
]
for col in columns_to_remove:
green_taxi_df.pop(col)
Cleanse data
Run the describe() function on the new dataframe to see summary statistics for each field.
green_taxi_df.describe()
PASS PICK PICK DROP DROP

ENGE TRIP UPLO UPLA OFFL OFFL TOTA MON DAY_ DAY_ HOUR
VEND RCOU DIST NGIT TITU ONGI ATIT LAMO TH_N OF_M OF_W _OF_D
ORID NT ANCE UDE DE TUDE UDE UNT UM ONTH EEK AY
COUN 4800 4800 4800 4800 4800 4800 4800 4800 4800 4800 4800 4800
T 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
1.78 1.37 2.87 - 40.6 - 40.7 14.7 6.50 15.1 3.27 13.52
MEAN 73.8 9 73.8 0 5 3
3 4
STD
0.41 1.04 2.93 2.76 1.52 2.61 1.44 12.0 3.45 8.45 1.95 6.83
8
1.00 0.00 0.00 - 0.00 - 0.00 - 1.00 1.00 0.00 0.00

MIN 74.6 74.6 300.
6 6 00
2.00 1.00 1.06 - 40.7 - 40.7 7.80 3.75 8.00 2.00 9.00
25% 73.9 0 73.9 0
6 7
2.00 1.00 1.90 - 40.7 - 40.7 11.3 6.50 15.0 3.00 15.00
50% 73.9 5 73.9 5 0 0
4 4
2.00 1.00 3.60 - 40.8 - 40.7 17.8 9.25 22.0 5.00 19.00
75% 73.9 0 73.9 9 0 0
2 1
MAX
2.00 9.00 97.5 0.00 41.9 0.00 41.9 450. 12.0 30.0 6.00 23.00
7 3 4 00 0 0
From the summary statistics, you see that there are several fields that have outliers or values that will reduce
model accuracy. First filter the lat/long fields to be within the bounds of the Manhattan area. This will filter out
longer taxi trips or trips that are outliers in respect to their relationship with other features.
Additionally filter the tripDistance field to be greater than zero but less than 31 miles (the haversine distance
between the two lat/long pairs). This eliminates long outlier trips that have inconsistent trip cost.
Lastly, the totalAmount field has negative values for the taxi fares, which don't make sense in the context of our
model, and the passengerCount field has bad data with the minimum values being zero.
Filter out these anomalies using query functions, and then remove the last few columns unnecessary for training.
final_df = green_taxi_df.query("pickupLatitude>=40.53 and pickupLatitude<=40.88")

final_df = final_df.query("pickupLongitude>=-74.09 and pickupLongitude<=-73.72")
final_df = final_df.query("tripDistance>=0.25 and tripDistance<31")
final_df = final_df.query("passengerCount>0 and totalAmount>0")
columns_to_remove_for_training = ["pickupLongitude", "pickupLatitude", "dropoffLongitude", "dropoffLatitude"]

for col in columns_to_remove_for_training:
final_df.pop(col)
Call describe() again on the data to ensure cleansing worked as expected. You now have a prepared and
cleansed set of taxi, holiday, and weather data to use for machine learning model training.
final_df.describe()
Configure workspace
Create a workspace object from the existing workspace. A Workspace is a class that accepts your Azure
subscription and resource information. It also creates a cloud resource to monitor and track your model runs.
Workspace.from_config() reads the file config.json and loads the authentication details into an object named ws
. ws is used throughout the rest of the code in this tutorial.
from azureml.core.workspace import Workspace

Split the data into train and test sets

Split the data into training and test sets by using the train_test_split function in the scikit-learn library. This
function segregates the data into the x (features) data set for model training and the y (values to predict) data
set for testing.
The test_size parameter determines the percentage of data to allocate to testing. The random_state parameter
sets a seed to the random generator, so that your train-test splits are deterministic.
y_df = final_df.pop("totalAmount")
x_df = final_df
x_train, x_test, y_train, y_test = train_test_split(x_df, y_df, test_size=0.2, random_state=223)
The purpose of this step is to have data points to test the finished model that haven't been used to train the
model, in order to measure true accuracy.
In other words, a well-trained model should be able to accurately make predictions from data it hasn't already
seen. You now have data prepared for auto-training a machine learning model.
Automatically train a model

To automatically train a model, take the following steps:
1. Define settings for the experiment run. Attach your training data to the configuration, and modify settings that
control the training process.
2. Submit the experiment for model tuning. After submitting the experiment, the process iterates through
different machine learning algorithms and hyperparameter settings, adhering to your defined constraints. It
chooses the best-fit model by optimizing an accuracy metric.
Define training settings
Define the experiment parameter and model settings for training. View the full list of settings. Submitting the
experiment with these default settings will take approximately 5-10 min, but if you want a shorter run time,
reduce the iterations parameter.
PROPERTY VALUE IN THIS TUTORIAL DESCRIPTION
iteration_timeout_minutes 2 Time limit in minutes for each iteration.

Reduce this value to decrease total
runtime.
iterations 20 Number of iterations. In each iteration,

a new machine learning model is
trained with your data. This is the
primary value that affects total run
time.
primary_metric spearman_correlation Metric that you want to optimize. The

best-fit model will be chosen based on
this metric.
preprocess True By using True, the experiment can

preprocess the input data (handling
missing data, converting text to
numeric, etc.)
verbosity logging.INFO Controls the level of logging.
n_cross_validations 5 Number of cross-validation splits to

perform when validation data is not
specified.
import logging
automl_settings = {
"iteration_timeout_minutes": 2,
"iterations": 20,
"primary_metric": 'spearman_correlation',
"preprocess": True,
"verbosity": logging.INFO,
"n_cross_validations": 5
}
Use your defined training settings as a **kwargs parameter to an AutoMLConfig object. Additionally, specify your
training data and the type of model, which is regression in this case.
from azureml.train.automl import AutoMLConfig
automl_config = AutoMLConfig(task='regression',
debug_log='automated_ml_errors.log',
X=x_train.values,
y=y_train.values.flatten(),
**automl_settings)
NOTE
Automated machine learning pre-processing steps (feature normalization, handling missing data, converting text to
numeric, etc.) become part of the underlying model. When using the model for predictions, the same pre-processing steps
applied during training are applied to your input data automatically.
Train the automatic regression model

Create an experiment object in your workspace. An experiment acts as a container for your individual runs. Pass
the defined automl_config object to the experiment, and set the output to True to view progress during the run.
After starting the experiment, the output shown updates live as the experiment runs. For each iteration, you see
the model type, the run duration, and the training accuracy. The field BEST tracks the best running training score
based on your metric type.
from azureml.core.experiment import Experiment

experiment = Experiment(ws, "taxi-experiment")
local_run = experiment.submit(automl_config, show_output=True)
Running on local machine
Parent Run ID: AutoML_1766cdf7-56cf-4b28-a340-c4aeee15b12b
Current status: DatasetFeaturization. Beginning to featurize the dataset.
Current status: DatasetEvaluation. Gathering dataset statistics.
Current status: FeaturesGeneration. Generating features for the dataset.
Current status: DatasetFeaturizationCompleted. Completed featurizing the dataset.
Current status: DatasetCrossValidationSplit. Generating individually featurized CV splits.
Current status: ModelSelection. Beginning model selection.
****************************************************************************************************
ITERATION: The iteration being evaluated.
PIPELINE: A summary description of the pipeline being evaluated.
DURATION: Time taken for the current iteration.
METRIC: The result of computing score on the fitted pipeline.
BEST: The best observed score thus far.
****************************************************************************************************
ITERATION PIPELINE DURATION METRIC BEST

0 StandardScalerWrapper RandomForest 0:00:16 0.8746 0.8746
1 MinMaxScaler RandomForest 0:00:15 0.9468 0.9468
2 StandardScalerWrapper ExtremeRandomTrees 0:00:09 0.9303 0.9468
3 StandardScalerWrapper LightGBM 0:00:10 0.9424 0.9468
4 RobustScaler DecisionTree 0:00:09 0.9449 0.9468
5 StandardScalerWrapper LassoLars 0:00:09 0.9440 0.9468
6 StandardScalerWrapper LightGBM 0:00:10 0.9282 0.9468
9 MinMaxScaler ExtremeRandomTrees 0:00:35 0.9199 0.9468
10 RobustScaler ExtremeRandomTrees 0:00:19 0.9411 0.9468
13 MinMaxScaler ExtremeRandomTrees 0:00:14 0.9186 0.9468
14 RobustScaler RandomForest 0:00:10 0.8810 0.9468
18 VotingEnsemble 0:00:23 0.9471 0.9471
19 StackEnsemble 0:00:27 0.9463 0.9471
Explore the results

Explore the results of automatic training with a Jupyter widget. The widget allows you to see a graph and table of
all individual run iterations, along with training accuracy metrics and metadata. Additionally, you can filter on
different accuracy metrics than your primary metric with the dropdown selector.

RunDetails(local_run).show()
Retrieve the best model
Select the best model from your iterations. The get_output function returns the best run and the fitted model for
the last fit invocation. By using the overloads on get_output , you can retrieve the best run and fitted model for
any logged metric or a particular iteration.
best_run, fitted_model = local_run.get_output()

print(best_run)
print(fitted_model)
Test the best model accuracy

Use the best model to run predictions on the test data set to predict taxi fares. The function predict uses the
best model and predicts the values of y, trip cost, from the x_test data set. Print the first 10 predicted cost
values from y_predict .
y_predict = fitted_model.predict(x_test.values)
print(y_predict[:10])
Calculate the root mean squared error of the results. Convert the y_test dataframe to a list to compare to the
predicted values. The function mean_squared_error takes two arrays of values and calculates the average squared
error between them. Taking the square root of the result gives an error in the same units as the y variable, cost. It
indicates roughly how far the taxi fare predictions are from the actual fares.
from math import sqrt
y_actual = y_test.values.flatten().tolist()
rmse = sqrt(mean_squared_error(y_actual, y_predict))
rmse
Run the following code to calculate mean absolute percent error (MAPE ) by using the full y_actual and
y_predict data sets. This metric calculates an absolute difference between each predicted and actual value and
sums all the differences. Then it expresses that sum as a percent of the total of the actual values.
sum_actuals = sum_errors = 0
for actual_val, predict_val in zip(y_actual, y_predict):

abs_error = actual_val - predict_val
if abs_error < 0:
abs_error = abs_error * -1
sum_errors = sum_errors + abs_error

sum_actuals = sum_actuals + actual_val
mean_abs_percent_error = sum_errors / sum_actuals

print("Model MAPE:")
print(mean_abs_percent_error)
print()
print("Model Accuracy:")
print(1 - mean_abs_percent_error)
Model MAPE:
0.14353867606052823
Model Accuracy:
0.8564613239394718
From the two prediction accuracy metrics, you see that the model is fairly good at predicting taxi fares from the
data set's features, typically within +- $4.00, and approximately 15% error.
The traditional machine learning model development process is highly resource-intensive, and requires
significant domain knowledge and time investment to run and compare the results of dozens of models. Using
automated machine learning is a great way to rapidly test many different models for your scenario.
Clean up resources
Do not complete this section if you plan on running other Azure Machine Learning service tutorials.
Stop the notebook VM
If you used a cloud notebook server, stop the VM when you are not using it to reduce cost.
1. In your workspace, select Notebook VMs.
2. From the list, select the VM.
3. Select Stop.
4. When you're ready to use the server again, select Start.
Delete everything
If you don't plan to use the resources you created, delete them, so you don't incur any charges.
You can also keep the resource group but delete a single workspace. Display the workspace properties and select
Delete.
Next steps
In this automated machine learning tutorial, you did the following tasks:
Configured a workspace and prepared data for an experiment.
Trained by using an automated regression model locally with custom parameters.
Explored and reviewed training results.
Deploy your model with Azure Machine Learning service.
Tutorial: Predict automobile price with the visual
interface
In this two-part tutorial, you learn how to use the Azure Machine Learning service visual interface to develop and
deploy a predictive analytic solution that predicts the price of any car.
In part one, you'll set up your environment, drag-and-drop datasets and analysis modules onto an interactive
canvas, and connect them together to create an experiment.
In part one of the tutorial you learn how to:
Create a new experiment
Import data
Prepare data
Train a machine learning model
Evaluate a machine learning model
In part two of the tutorial, you'll learn how to deploy your predictive model as an Azure web service so you can
use it to predict the price of any car based on technical specifications you send it.
A completed version of this tutorial is available as a sample experiment.
To find it, from the Experiments page, select Add New, then select the Sample 1 - Regression: Automobile
Price Prediction(Basic) experiment.
Create a new experiment

To create a visual interface experiment, you first need an Azure Machine Learnings service workspace. In this
section you learn how to create both these resources.
Create a new workspace
If you have an Azure Machine Learning service workspace, skip to the next section.
location.
FIELD DESCRIPTION

others.

WARNING

1. Open your workspace in the Azure portal.
2. In your workspace, select Visual interface. Then select Launch visual interface.
3. Create a new experiment by selecting +New at the bottom of the visual interface window.
4. Select Blank Experiment.
5. Select the default experiment name "Experiment created on ..." at the top of the canvas and rename it to
something meaningful. For example, "Automobile price prediction". The name doesn't need to be
unique.
Import data
Machine learning depends on data. Luckily, there are several sample datasets included in this interface available
for you to experiment with. For this tutorial, use the sample dataset Automobile price data (Raw).
1. To the left of the experiment canvas is a palette of datasets and modules. Select Saved Datasets then
select Samples to view the available sample datasets.
2. Select the dataset, Automobile price data (raw), and drag it onto the canvas.
3. Select which columns of data to work with. Type Select in the Search box at the top of the palette to find
the Select Columns in Dataset module.
4. Click and drag the Select Columns in Dataset module onto the canvas. Drop the module below the
dataset module.
5. Connect the dataset you added earlier to the Select Columns in Dataset module by clicking and
dragging. Drag from the dataset's output port, which is the small circle at the bottom of the dataset on the
canvas, all the way to the input port of Select Columns in Dataset, which is the small circle at the top of
the module.
TIP
You create a flow of data through your experiment when you connect the output port of one module to an input
port of another.
The red exclamation mark indicates that you haven't set the properties for the module yet.
6. Select the Select Columns in Dataset module.
7. In the Properties pane to the right of the canvas, select Edit columns.
In the Select columns dialog, select ALL COLUMNS and include all features. The dialog should look
like this:
8. On the lower right, select OK to close the column selector.

Run the experiment
At any time, click the output port of a dataset or module to see what the data looks like at that point in the data
flow. If the Visualize option is disabled, you first need to run the experiment.
An experiment runs on a compute target, a compute resource that is attached to your workspace. Once you create
a compute target, you can reuse it for future runs.
1. Select Run at the bottom to run the experiment.
2. When the Setup Compute Targets dialog appears, if your workspace already has a compute resource,
you can select it now. Otherwise, select Create new.
NOTE
The visual interface can only run experiments on Machine Learning Compute targets. Other compute targets will
not be shown.
3. Provide a name for the compute resource.

4. Select Run.
The compute resource will now be created. View the status in the top-right corner of the experiment.
NOTE
It takes approximately 5 minutes to create a compute resource. After the resource is created, you can reuse it and
skip this wait time for future runs.
The compute resource will autoscale to 0 nodes when it is idle to save cost. When you use it again after a delay, you
may again experience approximately 5 minutes of wait time while it scales back up.
After the compute target is available, the experiment runs. When the run is complete, a green check mark appears
on each module.
Visualize the data
Now that you have run your initial experiment, you can visualize the data to understand more about the dataset
you have.
1. Select the output port at the bottom of the Select Columns in Dataset then select Visualize.
2. Click on different columns in the data window to view information about that column.
In this dataset, each row represents an automobile, and the variables associated with each automobile
appear as columns. There are 205 rows and 26 columns in this dataset.
Each time you click a column of data, the Statistics information and Visualization image of that column
appears on the left.
3. Click each column to understand more about your dataset, and think about whether these columns will be
useful to predict the price of an automobile.
Prepare data
Typically, a dataset requires some preprocessing before it can be analyzed. You might have noticed some missing
values when visualizing the dataset. These missing values need to be cleaned so the model can analyze the data
correctly. You'll remove any rows that have missing values. Also, the normalized-losses column has a large
proportion of missing values, so you'll exclude that column from the model altogether.
TIP
Cleaning the missing values from input data is a prerequisite for using most of the modules.
Remove column
First, remove the normalized-losses column completely.
1. Select the Select Columns in Dataset module.
2. In the Properties pane to the right of the canvas, select Edit columns.
Leave With rules and ALL COLUMNS selected.
From the drop-downs, select Exclude and column names, and then click inside the text box. Type
normalized-losses.
On the lower right, select OK to close the column selector.
Now the properties pane for Select Columns in Dataset indicates that it will pass through all columns from
the dataset except normalized-losses.
The properties pane shows that the normalized-losses column is excluded.
3. Double-click the Select Columns in Dataset module and type the comment "Exclude normalized losses."
After you type the comment, click outside the module. A down-arrow appears to show that the module
contains a comment.
4. Click on the down-arrow to display the comment.
The module now shows an up-arrow to hide the comment.
Clean missing data

When you train a model, you have to do something about the data that is missing. In this case, you'll add a
module to remove any remaining row that has missing data.
1. Type Clean in the Search box to find the Clean Missing Data module.
2. Drag the Clean Missing Data module to the experiment canvas and connect it to the Select Columns in
Dataset module.
3. In the Properties pane, select Remove entire row under Cleaning mode.
4. Double-click the module and type the comment "Remove missing value rows."
Your experiment should now look something like this:
Train a machine learning model
Now that the data is ready, you can construct a predictive model. You'll use your data to train the model. Then
you'll test the model to see how closely it's able to predict prices.
Select an algorithm
Classification and regression are two types of supervised machine learning algorithms. Classification predicts
an answer from a defined set of categories, such as a color (red, blue, or green). Regression is used to predict a
number.
Because you want to predict price, which is a number, you can use a regression algorithm. For this example, you'll
use a linear regression model.
Split the data
Use your data for both training the model and testing it by splitting the data into separate training and testing
datasets.
1. Type split data in the search box to find the Split Data module and connect it to the left port of the Clean
Missing Data module.
2. Select the Split Data module. In the Properties pane, set the Fraction of rows in the first output dataset to
0.7. This way, we'll use 70 percent of the data to train the model, and hold back 30 percent for testing.
3. Double-click the Split Data and type the comment "Split the dataset into training set(0.7) and test
set(0.3)"
Train the model
Train the model by giving it a set of data that includes the price. The model scans the data and looks for
correlations between a car's features and its price.
1. To select the learning algorithm, clear your module palette search box.
2. Expand the Machine Learning then expand Initialize Model. This displays several categories of
modules that can be used to initialize machine learning algorithms.
3. For this experiment, select Regression > Linear Regression and drag it to the experiment canvas.
4. Find and drag the Train Model module to the experiment canvas. Connect the output of the Linear
Regression module to the left input of the Train Model module, and connect the training data output (left
port) of the Split Data module to the right input of the Train Model module.
5. Select the Train Model module. In the Properties pane, Select Launch column selector and then type
price next to Include column names. Price is the value that your model is going to predict
Your experiment should look like this:

Evaluate a machine learning model
Now that you've trained the model using 70 percent of your data, you can use it to score the other 30 percent of
the data to see how well your model functions.
1. Type score model in the search box to find the Score Model module and drag the module to the
experiment canvas. Connect the output of the Train Model module to the left input port of Score Model.
Connect the test data output (right port) of the Split Data module to the right input port of Score Model.
2. Type evaluate in the search box to find the Evaluate Model and drag the module to the experiment
canvas. Connect the output of the Score Model module to the left input of Evaluate Model. The final
experiment should look something like this:
3. Run the experiment using the compute resource you created earlier.
4. View the output from the Score Model module by selecting the output port of Score Model and select
Visualize. The output shows the predicted values for price and the known values from the test data.
5. To view the output from the Evaluate Model module, select the output port, and then select Visualize.
The following statistics are shown for your model:

Mean Absolute Error (MAE ): The average of absolute errors (an error is the difference between the
predicted value and the actual value).
Root Mean Squared Error (RMSE ): The square root of the average of squared errors of predictions made
on the test dataset.
Relative Absolute Error: The average of absolute errors relative to the absolute difference between actual
values and the average of all actual values.
Relative Squared Error: The average of squared errors relative to the squared difference between the actual
values and the average of all actual values.
Coefficient of Determination: Also known as the R squared value, this is a statistical metric indicating how
well a model fits the data.
For each of the error statistics, smaller is better. A smaller value indicates that the predictions more closely match
the actual values. For Coefficient of Determination, the closer its value is to one (1.0), the better the predictions.
Clean up resources
IMPORTANT
You can use the resources that you created as prerequisites for other Azure Machine Learning service tutorials and how-to
articles.
Delete everything
If you don't plan to use anything that you created, delete the entire resource group so you don't incur any
charges:
1. In the Azure portal, select Resource groups on the left side of the window.
2. In the list, select the resource group that you created.

3. On the right side of the window, select the ellipsis button (...).
Deleting the resource group also deletes all resources that you created in the visual interface.
Delete only the compute target
The compute target that you created here automatically autoscales to zero nodes when it's not being used. This is
to minimize charges. If you want to delete the compute target, take these steps:
1. In the Azure portal, open your workspace.
2. In the Compute section of your workspace, select the resource.
3. Select Delete.
Delete individual assets
In the visual interface where you created your experiment, delete individual assets by selecting them and then
selecting the Delete button.
Next steps
In part one of this tutorial, you completed these steps:
Created an experiment
Prepare the data
Train the model
Score and evaluate the model
In part two, you'll learn how to deploy your model as an Azure web service.
Continue to deploying models
Tutorial: Deploy a machine learning model with the
visual interface
To give others a chance to use the predictive model developed in part one of the tutorial, you can deploy it as an
Azure web service. So far, you've been experimenting with training your model. Now, it's time to generate new
predictions based on user input. In this part of the tutorial, you:
Prepare a model for deployment
Deploy a web service
Test a web service
Manage a web service
Consume the web service
Prerequisites
Complete part one of the tutorial to learn how to train and score a machine learning model in the visual interface.
Prepare for deployment

Before you deploy your experiment as a web service, you first have to convert your training experiment into a
predictive experiment.
1. Select Create Predictive Experiment* at the bottom of the experiment canvas.
When you select Create Predictive Experiment, several things happen:

The trained model is stored as a Trained Model module in the module palette. You can find it under
Trained Models.
Modules that were used for training are removed; specifically:
Train Model
Split Data
Evaluate Model
The saved trained model is added back into the experiment.
Web service input and Web service output modules are added. These modules identify where the
user data will enter the model, and where data is returned.
The training experiment is still saved under the new tabs at the top of the experiment canvas.
2. Run the experiment.
3. Select the output of the Score Model module and select View Results to verify the model is still working.
You can see the original data is displayed, along with the predicted price ("Scored Labels").
Your experiment should now look like this:
Deploy the web service

1. Select Deploy Web Service below the canvas.
2. Select the Compute Target that you'd like to run your web service.
Currently, the visual interface only supports deployment to Azure Kubernetes Service (AKS ) compute
targets. You can choose from available AKS compute targets in your machine learning service workspace or
configure a new AKS environment using the steps in the dialogue that appears.
3. Select Deploy Web Service. You'll see the following notification when deployment completes.
Deployment may take a few minutes.
Test the web service

You can test and manage your visual interface web services by navigating to the Web Services tab.
1. Go to the web service section. You'll see the web service you deployed with the name Tutorial - Predict
Automobile Price[Predictive Exp].
2. Select the web service name to view additional details.

3. Select Test.
4. Input testing data or use the autofilled sample data and select Test.
The test request is submitted to the web service and the results are shown on page. Although a price value
is generated for the input data, it is not used to generate the prediction value.
Consume the web service

Users can now send API requests to your Azure web service and receive results to predict the price of their new
automobiles.
Request/Response - The user sends one or more rows of automobile data to the service by using an HTTP
protocol. The service responds with one or more sets of results.
You can find sample REST calls in the Consume tab of the web service details page.
Navigate to the API Doc tab, to find more API details.
Manage models and deployments

The models and web service deployments you create in the visual interface can also be managed from the Azure
Machine Learning service workspace.
1. Open your workspace in the Azure portal.
2. In your workspace, select Models. Then select the experiment you created.
On this page, you'll see additional details about the model.
3. Select Deployments, it will list any web services that use the model. Select the web service name, it will go
to web service detail page. In this page, you can get more detailed information of the web service.
Clean up resources
IMPORTANT
articles.
Delete everything
If you don't plan to use anything that you created, delete the entire resource group so you don't incur any charges:

3. Select Delete.
Next steps
In this tutorial, you learned the key steps in creating, deploying, and consuming a machine learning model in the
visual interface. To learn more about how you can use the visual interface to solve other types of problems, see out
our other sample experiments.
Credit risk classification sample
Tutorial: Create your first classification model with
automated machine learning
In this tutorial, you learn how to create your first automated machine learning experiment in the Azure portal
(preview ) without writing a single line of code. This example creates a classification model to predict if a client will
subscribe to a fixed term deposit with a financial institution.
With automated machine learning, you can automate away time intensive tasks. Automated machine learning
rapidly iterates over many combinations of algorithms and hyperparameters to help you find the best model based
on a success metric of your choosing.
In this tutorial, you learn how to do the following tasks:
Create an Azure Machine Learning service workspace.
Run an automated machine learning experiment.
View experiment details.
Deploy the model.
Prerequisites
An Azure subscription. If you don’t have an Azure subscription, create a free account.
Download the bankmarketing_train.csv data file. The y column indicates if a customer subscribed to a
fixed term deposit, which is later identified as the target column for predictions in this tutorial.
Create a workspace
location.
FIELD DESCRIPTION

across the resource group. Use a name that's easy to recall
and to differentiate from workspaces created by others.

WARNING

Create and run the experiment
These steps walk you through experiment set up from data selection to choosing your primary metric and model
type.
1. Go to the left pane of your workspace. Select Automated machine learning under the Authoring
(Preview) section. You'll see the Welcome to Automated Machine Learning screen, since this is your
first experiment with Automated Machine Learning.
2. Select Create experiment. Then enter my-1st-automl-experiment as the experiment name.

3. Select Create a new compute and configure your compute context for this experiment.
FIELD VALUE
Compute name Enter a unique name that identifies your compute context.
For this example, we use automl-compute.
Virtual machine size Select the virtual machine size for your compute. We use
Standard_DS12_V2.
Additional settings Min node: 1. To enable data profiling, you must have one
or more nodes.
Max node: 6.
To create your new compute, select Create. This takes a few moments.
When creation is complete, select your new compute from the drop-down list, and then select Next.
NOTE
For this tutorial, we use the default storage account and container created with your new compute. They
automatically populate in the form.
4. Select Upload and choose the bankmarketing_train.csv file from your local computer to upload it to the
default container. Public preview supports only local file uploads and Azure Blob storage accounts. When
the upload is complete, select the file from the list.
5. The Preview tab allows us to further configure our data for this experiment.
On the Preview tab, indicate that the data includes headers. The service defaults to include all of the
features (columns) for training. For this example, scroll to the right and Ignore the day_of_week feature.
NOTE
Data profiling is not available with computes that have zero minimum nodes.
6. Select Classification as the prediction task.

7. Select y as the target column, where we want to do predictions. This column indicates whether the client
subscribed to a term deposit or not.
8. Expand Advanced Settings and populate the fields as follows.
ADVANCED SETTINGS VALUE
Primary metric AUC_weighted
Exit criteria When any of these criteria are met, the training job ends
before full completion:
Training job time (minutes ): 5
Max number of iterations: 10
Preprocessing Enables preprocessing done by automated machine

learning. This includes automatic data cleansing, preparing,
and transformation to generate synthetic features.
Validation Select K-fold cross-validation and 2 for the number of

cross-validations.
Concurrency Select 5 for the number of max concurrent iterations.

NOTE
For this experiment, we don't set a metric or max cores per iterations threshold. We also don't block algorithms from
being tested.
9. Select Start to run the experiment.

When the experiment starts, you see a blank Run Detail screen with the following status at the top.
The experiment preparation process takes a couple of minutes. When the process finishes, the status message
changes to Run is Running.
View experiment details

As the experiment progresses, the Run Detail screen updates the iteration chart and list with the different
iterations (models) that are run. The iterations list is in order by metric score. By default, the model that scores the
highest based on our AUC_weighted metric is at the top of the list.
TIP
Training jobs take several minutes for each pipeline to finish running.
Deploy the model

By using automated machine learning in the Azure portal, we can deploy the best model as a web service to
predict on new data and identify potential areas of opportunity. For this experiment, deployment means that the
financial institution now has an iterative and scalable solution for identifying potential fixed term deposit
customers.
In this experiment context, VotingEnsemble is considered the best model, based on the AUC_weighted metric.
We deploy this model, but be advised, deployment takes about 20 minutes to complete.
1. On the Run Detail page, select the Deploy Best Model button.
2. Populate the Deploy Best Model pane as follows:
FIELD VALUE
Deployment name my-automl-deploy
Deployment description My first automated machine learning experiment

deployment
Scoring script Autogenerate
Environment script Autogenerate
3. Select Deploy.
A deployment complete message appears when deployment successfully finishes.
Now you have an operational web service to generate predictions.
Clean up resources
Deployment files are larger than data and experiment files, so they cost more to store. Delete only the deployment
files to minimize costs to your account, or if you want to keep your workspace and experiment files. Otherwise,
delete the entire resource group, if you don't plan to use any of the files.
Delete the deployment instance
Delete just the deployment instance from the Azure portal, if you want to keep the resource group and workspace
for other tutorials and exploration.
1. Go to the Assets pane on the left and select Deployments.
2. Select the deployment you want to delete and select Delete.
3. Select Proceed.
Delete the resource group
IMPORTANT
articles.
Next steps
In this automated machine learning tutorial, you used the Azure portal to create and deploy a classification model.
See these articles for more information and next steps:
Consume a web service
Learn more about preprocessing.
Learn more about data profiling.
Learn more about automated machine learning.
NOTE
This Bank Marketing dataset is made available under the Creative Commons (CCO: Public Domain) License. Any rights in
individual contents of the database are licensed under the Database Contents License and available on Kaggle. This dataset
was originally available within the UCI Machine Learning Database.
Please cite the following work:

[Moro et al., 2014] S. Moro, P. Cortez and P. Rita. A Data-Driven Approach to Predict the Success of Bank Telemarketing.
Decision Support Systems, Elsevier, 62:22-31, June 2014.
Explore Azure Machine Learning service with Jupyter
notebooks
The example Azure Machine Learning Notebooks repository includes the latest Azure Machine Learning Python
SDK samples. These Juypter notebooks are designed to help you explore the SDK and serve as models for your
own machine learning projects.
This article shows you how to access the repository from the following environments:
Azure Machine Learning Notebook VM
Bring your own notebook server
Data Science Virtual Machine
Azure Notebooks
NOTE
Once you've cloned the repository, you'll find tutorial notebooks in the tutorials folder and feature-specific notebooks in
the how-to-use-azureml folder.
Get samples on Azure Machine Learning Notebook VM

The easiest way to get started with the samples is to complete the Tutorial: Setup environment and workspace.
Once completed, you'll have a dedicated notebook server pre-loaded with the SDK and the sample repository. No
downloads or installation necessary.
Get samples on your notebook server

If you'd like to bring your own notebook server for local development, follow these steps:
The Azure Machine Learning SDK for Python installed. Use the instructions at Azure Machine Learning SDK
to do the following:
1. Use the instructions at Azure Machine Learning SDK to do the following:
Create a Miniconda environment[Create and manage Azure Machine Learning service workspaces]
Install the Azure Machine Learning SDK for Python
2. Create an Azure Machine Learning service workspace.
3. Write a configuration file file (aml_config/config.json).
4. Clone the GitHub repository.
git clone https://github.com/Azure/MachineLearningNotebooks.git
5. Start the notebook server from your cloned directory.
jupyter notebook
These instructions install the base SDK packages necessary for the quickstart and tutorial notebooks. Other
sample notebooks may require you to install extra components. For more information, see Install the Azure
Machine Learning SDK for Python.
Get samples on DSVM

The Data Science Virtual Machine (DSVM ) is a customized VM image built specifically for doing data science. If
you create a DSVM, the SDK and notebook server are installed and configured for you. However, you'll still need
to create a workspace and clone the sample repository.
1. Create an Azure Machine Learning service workspace.
2. Clone the GitHub repository.
3. Add a workspace configuration file to the cloned directory using either of these methods:
In the Azure portal, select Download config.json from the Overview section of your workspace.
Create a new workspace using code in the configuration.ipynb notebook in your cloned directory.
4. Start the notebook server from your cloned directory.
jupyter notebook
Get samples on Azure Notebooks

On Azure Notebooks, the SDK and notebook server are installed and configured for you. Azure Notebooks
provides a fully-managed, lightweight notebook environment for you to explore.
To access the sample repository on Azure Notebooks, navigate to your Azure Machine Learning workspace
through the Azure portal. From the Overview section, select Get Started in Azure Notebooks.
Next steps
Explore the sample notebooks to discover what Azure Machine Learning service can do, or try these tutorials:
Train and deploy an image classification model with MNIST
Prepare data and use automated machine learning to train a regression model with the NYC taxi data set
Sample 1 - Regression: Predict price
Learn how to build a machine learning regression model without writing a single line of code using the visual
interface.
This experiment trains a decision forest regressor to predict a car's price based on technical features such as
make, model, horsepower, and size. Because we're trying to answer the question "How much?" this is called a
regression problem. However, you can apply the same fundamental steps in this experiment to tackle any type of
machine learning problem whether it be regression, classification, clustering, and so on.
The fundamental steps of a training machine learning model are:
1. Get the data
2. Pre-process the data
3. Train the model
4. Evaluate the model
Here's the final, completed graph of the experiment we'll be working on. We'll provide the rationale for all the
modules so you can make similar decisions on your own.
Prerequisites
1. Create an Azure Machine Learning service workspace if you don't have one.
2. Open your workspace in the Azure portal. If you're not sure how to locate your workspace in the portal, see
how to find your workspace.
The interface webpage opens in a new browser page.
4. Select the Open button for the Sample 1 experiment:
Get the data

In this experiment, we use the Automobile price data (Raw) dataset, which is from the UCI Machine Learning
Repository. The dataset contains 26 columns that contain information about automobiles, including make, model,
price, vehicle features (like the number of cylinders), MPG, and an insurance risk score. The goal of this experiment
is to predict the price of the car.
Pre-process the data

The main data preparation tasks include data cleaning, integration, transformation, reduction, and discretization or
quantization. In the visual interface, you can find modules to perform these operations and other data pre-
processing tasks in the Data Transformation group in the left panel.
We use the Select Columns in Dataset module to exclude normalized-losses that have many missing values. We
then use Clean Missing Data to remove the rows that have missing values. This helps to create a clean set of
training data.
Train the model

Machine learning problems vary. Common machine learning tasks include classification, clustering, regression,
and recommender systems, each of which might require a different algorithm. Your choice of algorithm often
depends on the requirements of the use case. After you pick an algorithm, you need to tune its parameters to train
a more accurate model. You then need to evaluate all models based on metrics like accuracy, intelligibility, and
efficiency.
Because the goal of this experiment is to predict automobile prices, and because the label column (price) contains
real numbers, a regression model is a good choice. Considering that the number of features is relatively small (less
than 100) and these features aren't sparse, the decision boundary is likely to be nonlinear. So we use Decision
Forest Regression for this experiment.
We use the Split Data module to randomly divide the input data so that the training dataset contains 70% of the
original data and the testing dataset contains 30% of the original data.
Test, evaluate, and compare

We split the dataset and use different datasets to train and test the model to make the evaluation of the model
more objective.
After the model is trained, we use the Score Model and Evaluate Model modules to generate predicted results
and evaluate the models.
Score Model generates predictions for the test dataset by using the trained model. To check the result, select the
output port of Score Model and then select Visualize.
We then pass the scores to the Evaluate Model module to generate evaluation metrics. To check the result, select
the output port of the Evaluate Model and then select Visualize.
Clean up resources
IMPORTANT
articles.
Delete everything

3. Select Delete.
Next steps
Explore the other samples available for the visual interface:
Sample 2 - Regression: Compare algorithms for automobile price prediction
Sample 3 - Classification: Predict credit risk
Sample 4 - Classification: Predict credit risk (cost sensitive)
Sample 5 - Classification: Predict churn
Sample 6 - Classification: Predict flight delays
Sample 2 - Regression: Predict price and compare
algorithms
Learn how to build a complex machine learning experiment without writing a single line of code using the visual
interface. This sample trains and compares multiple regression models to predict a car's price based on its
technical features. We'll provide the rationale for the choices made in this experiment so you can tackle your own
machine learning problems.
If you're just getting started with machine learning, you can take a look at the basic version of this experiment to
see a basic regression experiment.
Here's the completed graph for this experiment:
Prerequisites
Experiment summary
We use these steps to build the experiment:
1. Get the data.
2. Pre-process the data.
3. Train the model.
4. Test, evaluate, and compare the models.
Get the data

In this experiment, we use the Automobile price data (Raw) dataset, which is from the UCI Machine Learning
Repository. This dataset contains 26 columns that contain information about automobiles, including make, model,
price, vehicle features (like the number of cylinders), MPG, and an insurance risk score. The goal of this experiment
is to predict the price of a car.

The main data preparation tasks include data cleaning, integration, transformation, reduction, and discretization or
quantization. In visual interface, you can find modules to perform these operations and other data pre-processing
tasks in the Data Transformation group in the left panel.
In this experiment, we use the Select Columns in Dataset module to exclude normalized-losses that have many
missing values. We then use Clean Missing Data to remove the rows that have missing values. This helps to
create a clean set of training data.
Train the model

Machine learning problems vary. Common machine learning tasks include classification, clustering, regression,
and recommender systems, each of which might require a different algorithm. Your choice of algorithm often
depends on the requirements of the use case. After you pick an algorithm, you need to tune its parameters to train
a more accurate model. You then need to evaluate all models based on metrics like accuracy, intelligibility, and
efficiency.
Because the goal of this experiment is to predict automobile prices, and because the label column (price) contains
real numbers, a regression model is a good choice. Considering that the number of features is relatively small (less
than 100) and these features aren't sparse, the decision boundary is likely to be nonlinear.
To compare the performance of different algorithms, we use two nonlinear algorithms, Boosted Decision Tree
Regression and Decision Forest Regression, to build models. Both algorithms have parameters that you can
change, but we use the default values for this experiment.
We use the Split Data module to randomly divide the input data so that the training dataset contains 70% of the
original data and the testing dataset contains 30% of the original data.
Test, evaluate, and compare the models

We use two different sets of randomly chosen data to train and then test the model, as described in the previous
section. We split the dataset and use different datasets to train and test the model to make the evaluation of the
model more objective.
After the model is trained, we use the Score Model and Evaluate Model modules to generate predicted results
and evaluate the models. Score Model generates predictions for the test dataset by using the trained model. We
then pass the scores to Evaluate Model to generate evaluation metrics.
In this experiment, we use two instances of Evaluate Model to compare two pairs of models.
First, we compare two algorithms on the training dataset. Second, we compare two algorithms on the testing
dataset. Here are the results:
These results show that the model built with Boosted Decision Tree Regression has a lower root mean squared
error than the model built on Decision Forest Regression.
Both algorithms have a lower error on the training dataset than on the unseen testing dataset.
Clean up resources
IMPORTANT
articles.
Delete everything

3. Select Delete.
Next steps
Sample 1 - Regression: Predict an automobile's price
Learn how to build a machine learning classifier without writing a single line of code using the visual interface.
This sample trains a two-class boosted decision tree to predict credit risk (high or low ) based on credit
application information such as credit history, age, and number of credit cards.
Because we're trying to answer the question "Which one?" this is called a classification problem. However, you can
apply the same fundamental process to tackle any type of machine learning problem whether it be regression,
classification, clustering, and so on.
Prerequisites
Related sample
Sample 4 - Classification: Credit Risk Prediction (Cost Sensitive) provides an advanced experiment that solves the
same problem as this experiment. It shows how to perform cost sensitive classification by using an Execute
Python Script module and compare the performance of two binary classification algorithms. Refer to it if you
want to learn more about how to build classification experiments.
Data
We use the German Credit Card dataset from the UC Irvine repository. The dataset contains 1,000 samples with
20 features and 1 label. Each sample represents a person. The features include numerical and categorical features.
See the UCI website for the meaning of the categorical features. The last column is the label, which denotes the
credit risk and has only two possible values: high credit risk = 2, and low credit risk = 1.
Experiment summary
We follow these steps to create the experiment:
1. Drag the German Credit Card UCI Data dataset module into the experiment's canvas.
2. Add an Edit Metadata module so we can add meaningful names for each column.
3. Add a Split Data module to create the training and test sets. Set the fraction of rows in the first output dataset
to 0.7. This setting specifies that 70% of the data will be output to the left port of the module and the rest to the
right port. We use the left dataset for training and the right one for testing.
4. Add a Two-Class Boosted Decision Tree module to initialize a boosted decision tree classifier.
5. Add a Train Model module. Connect the classifier from the previous step to the left input port of the Train
Model. Add the training set (the left output port of the Split Data) to the right input port of the Train Model.
The Train Model will train the classifier.
6. Add a Score Model module and connect the Train Model module to it. Then add the test set (the right port
of the Split Data) to the Score Model. The Score Model will make the predictions. You can select its output
port to see the predictions and the positive class probabilities.
7. Add an Evaluate Model module and connect the scored dataset to its left input port. To see the evaluation
results, select the output port of the Evaluate Model module and select Visualize.
Here's the complete experiment graph:
Results
In the evaluation results, you can see that the AUC of the model is 0.776. At threshold 0.5, the precision is 0.621,
the recall is 0.456, and the F1 score is 0.526.
Clean up resources
IMPORTANT
articles.
Delete everything


3. Select Delete.
Next steps
Sample 4 - Classification: Predict credit risk (cost
sensitive)
This article shows you how to build a complex machine learning experiment using the visual interface. You'll learn
how to implement custom logic using Python scripts and compare multiple models to choose the best option.
This sample trains a classifier to predict credit risk using credit application information such as credit history, age,
and number of credit cards. However, you can apply the concepts in this article to tackle your own machine
learning problems.
If you're just getting started with machine learning, you can take a look at the basic classifier sample first.
Prerequisites
2. Open your workspace in the Azure portal. If you're not sure how to locate your workspace in the portal,
see how to find your workspace.

Data
We use the German Credit Card dataset from the UC Irvine repository. This dataset contains 1,000 samples with
20 features and 1 label. Each sample represents a person. The 20 features include numerical and categorical
features. See the UCI website for more information about the dataset. The last column is the label, which denotes
the credit risk and has only two possible values: high credit risk = 2, and low credit risk = 1.
Experiment summary
In this experiment, we compare two different approaches for generating models to solve this problem:
Training with the original dataset.
Training with a replicated dataset.
With both approaches, we evaluate the models by using the test dataset with replication to ensure that results are
aligned with the cost function. We test two classifiers with both approaches: Two-Class Support Vector
Machine and Two-Class Boosted Decision Tree.
The cost of misclassifying a low -risk example as high is 1, and the cost of misclassifying a high-risk example as
low is 5. We use an Execute Python Script module to account for this misclassification cost.
Here's the graph of the experiment:
Data processing
We start by using the Metadata Editor module to add column names to replace the default column names with
more meaningful names, obtained from the dataset description on the UCI site. We provide the new column
names as comma-separated values in the New column name field of the Metadata Editor.
Next, we generate the training and test sets used to develop the risk prediction model. We split the original
dataset into training and test sets of the same size by using the Split Data module. To create sets of equal size,
we set the Fraction of rows in the first output dataset option to 0.5.
Generate the new dataset
Because the cost of underestimating risk is high, we set the cost of misclassification like this:
For high-risk cases misclassified as low risk: 5
For low -risk cases misclassified as high risk: 1
To reflect this cost function, we generate a new dataset. In the new dataset, each high-risk example is replicated
five times, but the number of low -risk examples doesn't change. We split the data into training and test datasets
before replication to prevent the same row from being in both sets.
To replicate the high-risk data, we put this Python code into an Execute Python Script module:
import pandas as pd
def azureml_main(dataframe1 = None, dataframe2 = None):
df_label_1 = dataframe1[dataframe1.iloc[:, 20] == 1]

df_label_2 = dataframe1[dataframe1.iloc[:, 20] == 2]
result = df_label_1.append([df_label_2] * 5, ignore_index=True)

return result,
The Execute Python Script module replicates both the training and test datasets.
Feature engineering
The Two-Class Support Vector Machine algorithm requires normalized data. So we use the Normalize Data
module to normalize the ranges of all numeric features with a tanh transformation. A tanh transformation
converts all numeric features to values within a range of 0 and 1 while preserving the overall distribution of
values.
The Two-Class Support Vector Machine module handles string features, converting them to categorical
features and then to binary features with a value of 0 or 1. So we don't need to normalize these features.
Models
Because we apply two classifiers, Two-Class Support Vector Machine (SVM ) and Two-Class Boosted
Decision Tree, and also use two datasets, we generate a total of four models:
SVM trained with original data.
SVM trained with replicated data.
Boosted Decision Tree trained with original data.
Boosted Decision Tree trained with replicated data.
We use the standard experimental workflow to create, train, and test the models:
1. Initialize the learning algorithms, using Two-Class Support Vector Machine and Two-Class Boosted
Decision Tree.
2. Use Train Model to apply the algorithm to the data and create the actual model.
3. Use Score Model to produce scores by using the test examples.
The following diagram shows a portion of this experiment, in which the original and replicated training sets are
used to train two different SVM models. Train Model is connected to the training set, and Score Model is
connected to the test set.
In the evaluation stage of the experiment, we compute the accuracy of each of the four models. For this
experiment, we use Evaluate Model to compare examples that have the same misclassification cost.
The Evaluate Model module can compute the performance metrics for as many as two scored models. So we
use one instance of Evaluate Model to evaluate the two SVM models and another instance of Evaluate Model
to evaluate the two Boosted Decision Tree models.
Notice that the replicated test dataset is used as the input for Score Model. In other words, the final accuracy
scores include the cost for getting the labels wrong.
Combine multiple results

The Evaluate Model module produces a table with a single row that contains various metrics. To create a single
set of accuracy results, we first use Add Rows to combine the results into a single table. We then use the
following Python script in the Execute Python Script module to add the model name and training approach for
each row in the table of results:
import pandas as pd
new_cols = pd.DataFrame(
columns=["Algorithm","Training"],
data=[
["SVM", "weighted"],
["SVM", "unweighted"],
["Boosted Decision Tree","weighted"],
["Boosted Decision Tree","unweighted"]
])
result = pd.concat([new_cols, dataframe1], axis=1)

return result,
Results
To view the results of the experiment, you can right-click the Visualize output of the last Select Columns in
Dataset module.
The first column lists the machine learning algorithm used to generate the model. The second column indicates
the type of the training set. The third column contains the cost-sensitive accuracy value.
From these results, you can see that the best accuracy is provided by the model that was created with Two-Class
Support Vector Machine and trained on the replicated training dataset.
Clean up resources
IMPORTANT
articles.
Delete everything
If you don't plan to use anything that you created, delete the entire resource group so you don't incur any
charges:


3. Select Delete.
Next steps
Sample 5 - Classification: Predict churn, appetency,
and up-selling
Learn how to build a complex machine learning experiment without writing a single line of code using the visual
interface.
This experiment trains three, two-class boosted decision tree classifiers to predict common tasks for customer
relationship management (CRM ) systems: churn, appetency, and up-selling. The data values and labels are split
across multiple data sources and scrambled to anonymize customer information, however, we can still use the
visual interface to combine data sets and train a model using the scrambled values.
Because we're trying to answer the question "Which one?" this is called a classification problem. However, you
can apply the same steps in this experiment to tackle any type of machine learning problem whether it be
regression, classification, clustering, and so on.
Prerequisites
4. Select the Open button for the Sample 5 experiment.
Data
The data we use for this experiment is from KDD Cup 2009. The dataset has 50,000 rows and 230 feature
columns. The task is to predict churn, appetency, and up-selling for customers who use these features. For more
information about the data and the task, see the KDD website.
Experiment summary
This visual interface sample experiment shows binary classifier prediction of churn, appetency, and up-selling, a
common task for customer relationship management (CRM ).
First, we do some simple data processing.
The raw dataset contains lots of missing values. We use the Clean Missing Data module to replace the
missing values with 0.
The features and the corresponding churn, appetency, and up-selling labels are in different datasets. We
use the Add Columns module to append the label columns to the feature columns. The first column, Col1,
is the label column. The rest of the columns, Var1, Var2, and so on, are the feature columns.
We use the Split Data module to split the dataset into train and test sets.
We then use the Boosted Decision Tree binary classifier with the default parameters to build the prediction
models. We build one model per task, that is, one model each to predict up-selling, appetency, and churn.
Results
Visualize the output of the Evaluate Model module to see the performance of the model on the test set. For the
up-selling task, the ROC curve shows that the model does better than a random model. The area under the curve
(AUC ) is 0.857. At threshold 0.5, the precision is 0.7, the recall is 0.463, and the F1 score is 0.545.
You can move the Threshold slider and see the metrics change for the binary classification task.
Clean up resources
IMPORTANT
articles.
Delete everything


3. Select Delete.
Next steps
Sample 6 - Classification: Predict flight delays using R
This experiment uses historical flight and weather data to predict if a scheduled passenger flight will be delayed
by more than 15 minutes.
This problem can be approached as a classification problem, predicting two classes -- delayed, or on time. To build
a classifier, this model using a large number of examples from historic flight data.
Here's the final experiment graph:
Prerequisites

Get the data

This experiment uses the Flight Delays Data dataset. It's part of the TranStats data collection from the U.S.
Department of Transportation. The dataset contains flight delay information from April to October 2013. Before
uploading the data to the visual interface, it has been pre-processed as follows:
Filtered to include the 70 busiest airports in the continental United States.
For canceled flights, relabeled as delayed by more than 15 mins.
Filtered out diverted flights.
Selected 14 columns.
To supplement the flight data, the Weather Dataset is used. The weather data contains hourly land-based
weather observations from NOAA, and represents observations from airport weather stations, covering the same
time period of April-October 2013. Before uploading to Azure ML visual interface, it has been pre-processed as
follows:
Weather station IDs were mapped to corresponding airport IDs.
Weather stations not associated with the 70 busiest airports were removed.
The Date column was split into separate columns: Year, Month, and Day.
Selected 26 columns.

A dataset usually requires some pre-processing before it can be analyzed.
Flight data
The columns Carrier, OriginAirportID, and DestAirportID are saved as integers. However, they're categorical
attributes, use the Edit Metadata module to convert them to categorical.
Then use the Select Columns in Dataset module to exclude from the dataset columns that are possible target
leakers: DepDelay, DepDel15, ArrDelay, Canceled, Year.
To join the flight records with the hourly weather records, use the scheduled departure time as one of the join
keys. To do the join, the CSRDepTime column must be rounded down to the nearest hour, which is done by in the
Execute R Script module.
Weather data
Columns that have a large proportion of missing values are excluded using the Project Columns module. These
columns include all string-valued columns: ValueForWindCharacter, WetBulbFarenheit, WetBulbCelsius,
PressureTendency, PressureChange, SeaLevelPressure, and StationPressure.
The Clean Missing Data module is then applied to the remaining columns to remove rows with missing data.
Weather observation times are rounded up to the nearest full hour. Scheduled flight times and the weather
observation times are rounded in opposite directions to ensure the model uses only weather before the flight
time.
Since weather data is reported in local time, time zone differences are accounted for by subtracting the time zone
columns from the scheduled departure time and the weather observation time. These operations are done using
the Execute R Script module.
Joining Datasets
Flight records are joined with weather data at origin of the flight (OriginAirportID ) using the Join Data module.
Flight records are joined with weather data using the destination of the flight (DestAirportID ).
Preparing Training and Test Samples

The Split Data module splits the data into April through September records for training, and October records for
test.
Year, month, and timezone columns are removed from the training dataset using the Select Columns module.
Define features
In machine learning, features are individual measurable properties of something you’re interested in. Finding a
strong set of features requires experimentation and domain knowledge. Some features are better for predicting
the target than others. Also, some features may have a strong correlation with other features, and won't add new
information to the model. These features can be removed.
To build a model, you can use all the features available, or select a subset of the features.
Choose and apply a learning algorithm

Create a model using the Two-Class Logistic Regression module and train it on the training dataset.
The result of the Train Model module is a trained classification model that can be used to score new samples to
make predictions. Use the test set to generate scores from the trained models. Then use the Evaluate Model
module to analyze and compare the quality of the models.
After you run the experiment, you can view the output from the Score Model module by clicking the output port
and selecting Visualize. The output includes the scored labels and the probabilities for the labels.
Finally, to test the quality of the results, add the Evaluate Model module to the experiment canvas, and connect
the left input port to the output of the Score Model module. Run the experiment and view the output of the
Evaluate Model module, by clicking the output port and selecting Visualize.
Evaluate
The logistic regression model has AUC of 0.631 on the test set.
Next steps
Deep learning vs. machine learning
This article helps you understand the relationship between deep learning vs. machine learning. You'll learn how
the two concepts compare and how they fit into the broader category of artificial intelligence. Finally, the article
describes how deep learning can be applied to real-world scenarios, such as fraud detection, voice and facial
recognition, sentiment analytics, and time series forecasting.
How do deep learning, machine learning, and AI relate to one another?
In order to understand the difference between deep learning vs. machine learning vs. AI, learn about each of these
fields of study:
Deep learning is a subset of machine learning based on artificial neural networks that permit a machine to
train itself. In this case, the learning process is called deep, because the structure of artificial neural networks
consists of multiple input, output, and hidden layers. Each layer contains units that transform the input data
into information that the next layer can use to perform a certain predictive task. Thanks to this structure, a
machine is able to learn through its own data processing.
Machine learning is a subset of artificial intelligence that includes techniques (such as deep learning) that
enable machines to improve at tasks with experience. In this case, the learning process is based on the
following steps:
1. Feed an algorithm with data by providing it with more information (for example, by performing feature
extraction).
2. Use this data to train a model.
3. Test and deploy the model.
4. Consume the deployed model to perform a certain automated predictive task.
Artificial intelligence (AI ) is a technique that enables computers to mimic human intelligence. It includes
machine learning. It is important to understand the relationship between AI vs. machine learning vs. deep
learning: Machine learning is a way of achieving artificial intelligence, which means by the use of machine
learning and deep learning techniques you can build computer systems and applications that can perform
tasks commonly associated with human intelligence, such as visual perception, speech recognition,
decision-making, and translation between languages.
Comparison of deep learning vs. machine learning
Now that you have the overview of machine learning vs. deep learning, compare the two techniques. In the case of
machine learning, the algorithm needs to be told how to make an accurate prediction by providing it with more
information; whereas, in the case of deep learning, the algorithm is able to learn that through its own data
processing thanks to the artificial neural network structure.
The table below compares the two techniques in more detail:
ALL MACHINE LEARNING ONLY DEEP LEARNING
Number of data points Can use small data amounts provided Requires a large amount of training
by users data to make concise conclusions
Hardware dependencies Can work on low-end machines. It Depends on high-end machines. It

doesn't need a large amount of inherently does a large number of
computational power matrix multiplication operations. These
operations can be efficiently optimized
using a GPU
Featurization process Requires features to be accurately It learns high-level features from data
identified and created by users and creates new features by itself
Learning approach Divides tasks into small pieces, and then Solves the problem on an end-to-end
combine received results into one basis
conclusion
Execution time Comparatively takes less time to train, Takes an unusually long time to train,
ranging from a few seconds to a few because there are many parameters in a
hours deep learning algorithm
Output The output is usually a numerical value, The output can be anything from a
like a score or a classification score, a text, an element, or a sound
Deep learning use cases: What problems does it solve?

Because of the artificial neural network structure, deep learning excels at identifying patterns in unstructured data,
such as images, sound, video, and text. For this reason, it's rapidly transforming many industries including
healthcare, energy, fintech, transportation, and others, to rethink traditional business processes. Some of the most
common applications for deep learning are described in the following paragraphs.
Named-entity recognition
One use of deep-learning networks is named-entity recognition, which is a way to extract from unstructured,
unlabeled data certain types of information like people, places, companies, or things. That information can then be
stored in a structured schema to build a list of addresses or serve as a benchmark for an identity validation engine.
Object detection
Deep learning has been applied in many object detection use cases. Object detection is actually a two-part process:
image classification, and then image localization. Image classification determines what the objects in the image are,
like a car or a person, while image localization provides the specific location of these objects. Object detection is
already being used in gaming, retail, tourism, and self-driving cars industries.
Image caption generation
Similar to the image recognition task, image captioning is the task where, for a given image, the system must
generate a caption that describes the contents of the image. Once you can detect objects in photographs and
generate labels for those objects, you can see that the next step is to turn those labels into a coherent sentence
description. Generally, the systems involve the use of very large convolutional neural networks for the object
detection in the photographs and then a recurrent neural network (RNN ) to turn the labels into a coherent
sentence.
Machine translation
Machine translation takes words, phrases, or sentences from one language and automatically translates them into
another language. Automatic machine translation has been around for a long time, but deep learning is achieving
top results in two specific areas: automatic translation of text (and speech-to-text) and automatic translation of
images. With the proper data transformation, a deep network is capable of understanding text, audio, and visual
signals. Machine translation can be used to identify snippets of sound in larger audio files and transcribe the
spoken word or image as text.
Text analytics
One important task that deep learning can perform is e-discovery. Companies are using deep learning-based text
analytics for insider trading detection and government regulatory compliance. Hedge funds use text analytics to
drill down into massive document repositories for obtaining insights into future investment performance and
market sentiment. The use case for deep learning-based text analytics revolves around its ability to parse massive
amounts of text data to perform analytics or yield aggregations.
What are artificial neural networks?

Artificial neural networks are formed by layers of connected nodes. Deep learning models use neural networks
with a very large number of layers. The most popular artificial neural network typologies are discussed below.
Feedforward neural network
The feedforward neural network is the most basic type of artificial neural network, in which the information travels
in only one direction from input layer to output layer. Feedforward neural networks transform an input by putting
it through a series of hidden layers. Every layer is made up of a set of neurons, where each layer is fully connected
to all neurons in the layer before. Finally, there is a last fully connected layer — the output layer — that represents
the predictions generated.
Recurrent neural network
Recurrent neural networks are a more widely used type of artificial neural network, which works on the principle
of saving the output of a layer and feeding it back to the input layer to help in predicting the outcome of the layer.
These neural networks possess greater learning abilities and are widely employed for more complex tasks, such as
learning handwriting or language recognition.
Convolutional neural networks
A convolutional neural network is a particularly effective type of artificial neural network, which presents a unique
architecture. First, the layers are organized in three dimensions: width, height, and depth. Further, the neurons in
one layer don't connect to all the neurons in the next layer, but only to a small region of it. Lastly, the final output
will be reduced to a single vector of probability scores, organized along the depth dimension. These neural
networks have been used in areas such as image recognition and classification.
Next steps
The following articles show you how to use deep learning technology on the Azure Machine Learning service:
Classify handwritten digits with a TensorFlow model
Classify handwritten digits with a TensorFlow estimator and Keras
Classify images with a Pytorch model
Classify handwritten digits with a Chainer model
What is an Azure Machine Learning service
workspace?
The workspace is the top-level resource for Azure Machine Learning service, providing a centralized place to
work with all the artifacts you create when you use Azure Machine Learning service. The workspace keeps a
history of all training runs, including logs, metrics, output, and a snapshot of your scripts. You use this
information to determine which training run produces the best model.
Once you have a model you like, you register it with the workspace. You then use the registered model and
scoring scripts to deploy to Azure Container Instances, Azure Kubernetes Service, or to a field-programmable
gate array (FPGA) as a REST-based HTTP endpoint. You can also deploy the model to an Azure IoT Edge device
as a module.
Taxonomy
A taxonomy of the workspace is illustrated in the following diagram:
The diagram shows the following components of a workspace:

A workspace can contain Notebook VMs, cloud resources configured with the Python environment necessary
to run Azure Machine Learning.
User roles enable you to share your workspace with other users, teams or projects.
Compute targets are used to run your experiments.
When you create the workspace, associated resources are also created for you.
Experiments are training runs you use to build your models. You can create and run experiments with
The Azure Machine Learning SDK for Python.
The automated machine learning experiments (preview ) section in the Azure portal.
The visual interface (preview ).
Pipelines are reusable workflows for training and retraining your model.
Datasets aid in management of the data you use for model training and pipeline creation.
Once you have a model you want to deploy, you create a registered model.
Use the registered model and a scoring script to create a deployment.
Tools for workspace interaction
You can interact with your workspace in the following ways:
On the web:
The Azure portal
The visual interface (preview )
In Python using Azure Machine Learning SDK
On the command line using the Azure Machine Learning CLI extension
Machine learning with a workspace

Machine learning tasks read and/or write artifacts to your workspace.
Run an experiment to train a model - writes experiment run results to the workspace.
Use automated ML to train a model - writes training results to the workspace.
Register a model in the workspace.
Deploy a model - uses the registered model to create a deployment.
Create and run reusable workflows.
View machine learning artifacts such as experiments, pipelines, models, deployments.
Track and monitor models.
Workspace management
You can also perform the following workspace management tasks:
WORKSPACE MANAGEMENT
TASK PORTAL SDK CLI
Create a workspace ✓ ✓ ✓
Create and manage ✓ ✓ ✓

compute resources
Manage workspace access ✓ ✓
Create a notebook VM ✓
Create a workspace
There are multiple ways to create a workspace.
Use the Azure portal for a point-and-click interface to walk you through each step.
Use the Azure Machine Learning SDK for Python to create a workspace on the fly from Python scripts or
Jupiter notebooks
Use an Azure Resource Manager template or the Azure Machine Learning CLI when you need to automate or
customize the creation with corporate security standards.
If you work in Visual Studio Code, use the VS Code extension.
Associated resources
When you create a new workspace, it automatically creates several Azure resources that are used by the
workspace:
Azure Container Registry: Registers docker containers that you use during training and when you deploy a
model. To minimize costs, ACR is lazy-loaded until deployment images are created.
Azure Storage account: Is used as the default datastore for the workspace. Jupyter notebooks that are used
with your notebook VMs are stored here as well.
Azure Application Insights: Stores monitoring information about your models.
Azure Key Vault: Stores secrets that are used by compute targets and other sensitive information that's
needed by the workspace.
NOTE
In addition to creating new versions, you can also use existing Azure services.
Next steps
To get started with Azure Machine Learning service, see:
Azure Machine Learning service overview
Create a workspace
Manage a workspace
Tutorial: Train a model
What are compute targets in Azure Machine
Learning service?
A compute target is a designated compute resource/environment where you run your training script or host
your service deployment. This location may be your local machine or a cloud-based compute resource. Using
compute targets make it easy for you to later change your compute environment without having to change your
code.
In a typical model development lifecycle, you might:
1. Start by developing and experimenting on a small amount of data. At this stage, we recommend your local
environment (local computer or cloud-based VM ) as your compute target.
2. Scale up to larger data, or do distributed training using one of these training compute targets.
3. Once your model is ready, deploy it to a web hosting environment or IoT device with one of these deployment
compute targets.
The compute resources you use for your compute targets are attached to a workspace. Compute resources other
than the local machine are shared by users of the workspace.
Training compute targets

Azure Machine Learning service has varying support across different compute resources. You can also attach
your own compute resource, although support for various scenarios may vary.
Compute targets can be reused from one training job to the next. For example, once you attach a remote
VM to your workspace, you can reuse it for multiple jobs.
TRAINING TARGETS GPU SUPPORT AUTOMATED ML ML PIPELINES VISUAL INTERFACE
Local computer maybe yes
Azure Machine yes yes & yes yes

Learning Compute hyperparameter tunin
g
Remote VM yes yes & yes

hyperparameter
tuning
Azure Databricks yes yes
Azure Data Lake yes

Analytics
Azure HDInsight yes
Azure Batch yes
Learn more about setting up and using a compute target for model training.
Deployment targets
The following compute resources can be used to host your model deployment.
COMPUTE TARGET USAGE GPU SUPPORT FPGA SUPPORT DESCRIPTION
Local web service Testing/debug Good for limited

testing and
troubleshooting.
Hardware acceleration
depends on using
libraries in the local
system.
Notebook Testing/debug Good for limited

VM web service testing and
troubleshooting.
Azure Kubernetes Real-time inference yes yes Good for high-scale

Service (AKS) production
deployments.
Provides fast
response time and
autoscaling of the
deployed service.
Cluster autoscaling is
not supported
through the Azure
Machine Learning
SDK. To change the
nodes in the AKS
cluster, use the UI for
your AKS cluster in
the Azure portal. AKS
is the only option
available for the visual
interface.
Azure Container Testing or dev Good for low scale,

Instances (ACI) CPU-based workloads
requiring <48-GB
RAM
Azure Machine (Preview) Run batch scoring on

Learning Compute Batch inference serverless compute.
Supports normal and
low-priority VMs.
Azure IoT Edge (Preview) IoT module Deploy & serve ML

models on IoT
devices.
Azure Data Box Edge via IoT Edge yes Deploy & serve ML
models on IoT
devices.
NOTE
While compute targets like local, Notebook VM, and Azure Machine Learning Compute support GPU for training and
experimentation, using GPU for inference is only supported on Azure Kubernetes Service.
Learn where and how to deploy your model to a compute target.
Azure Machine Learning compute (managed)

A managed compute resource is created and managed by Azure Machine Learning service. This compute is
optimized for machine learning workloads. Azure Machine Learning Compute is the only managed compute as of
May 30, 2019. Additional managed compute resources may be added in the future.
You can use Azure Machine Learning Compute for training and for batch inferencing (Preview ). With this
compute resource, you have:
Single- or multi-node cluster
Autoscales each time you submit a run
Automatic cluster management and job scheduling
Support for both CPU and GPU resources
You can create Azure Machine Learning Compute instances in Azure portal, with the SDK, or with the CLI. When
created it is automatically part of your workspace unlike other kinds of compute targets.
Unmanaged compute
An unmanaged compute target is not managed by Azure Machine Learning service. You create this type of
compute target outside Azure Machine Learning, then attach it to your workspace. Unmanaged compute
resources can require additional steps for you to maintain or to improve performance for machine learning
workloads.
Next steps
Learn how to:
Set up a compute target to train your model
Deploy your model to a compute target
What is the visual interface for Azure Machine
Learning service?
The visual interface (preview ) for Azure Machine Learning service enables you to prep data, train, test, deploy,
manage, and track machine learning models without writing code.
There is no programming required, you visually connect datasets and modules to construct your model.
The visual interface uses your Azure Machine Learning service workspace to:
Write artifacts of experiment runs into the workspace.
Access datasets.
Use the compute resources in the workspace to run the experiment.
Register models.
Deploy models as web services on compute resources in the workspace.
Workflow
The visual interface gives you an interactive, visual canvas to quickly build, test, and iterate on a model.
You drag-and-drop modules onto the canvas.
Connect the modules together to form an experiment.
Run the experiment using the compute resource of the Machine Learning Service workspace.
Iterate on your model design by editing the experiment and running it again.
When you're ready, convert your training experiment to a predictive experiment.
Deploy the predictive experiment as a web service so that your model can be accessed by others.
Experiment
Create an experiment from scratch, or use an existing sample experiment as a template. Each time you run an
experiment, artifacts are stored in your workspace.
An experiment consists of datasets and analytical modules, which you connect together to construct a model.
Specifically, a valid experiment has these characteristics:
Datasets may be connected only to modules.
Modules may be connected to either datasets or other modules.
All input ports for modules must have some connection to the data flow.
All required parameters for each module must be set.
To learn how to get started with the visual interface, see Tutorial: Predict automobile price with the visual
interface.
Dataset
A dataset is data that has been uploaded to the visual interface to use in the modeling process. A number of
sample datasets are included for you to experiment with, and you can upload more datasets as you need them.
Module
A module is an algorithm that you can perform on your data. The visual interface has a number of modules
ranging from data ingress functions to training, scoring, and validation processes.
A module may have a set of parameters that you can use to configure the module's internal algorithms. When you
select a module on the canvas, the module's parameters are displayed in the Properties pane to the right of the
canvas. You can modify the parameters in that pane to tune your model.
For some help navigating through the library of machine learning algorithms available, see Algorithm & module
reference overview
Compute resources
Use compute resources from your workspace to run your experiment or host your deployed models as web
services. The supported compute targets are:
COMPUTE TARGET TRAINING DEPLOYMENT
Azure Machine Learning compute ✓
Azure Kubernetes Service ✓
Compute targets are attached to your Machine Learning workspace. You manage your compute targets in your
workspace in the Azure portal.
Deployment
Once your predictive analytics model is ready, you deploy it as a web service right from the visual interface.
The web services provide an interface between an application and your scoring model. An external application can
communicate with the scoring model in real time. A call to a web service returns prediction results to an external
application. To make a call to a web service, you pass an API key that was created when you deployed the web
service. The web service is based on REST, a popular architecture choice for web programming projects.
To learn how to deploy your model, see Tutorial: Deploy a machine learning model with the visual interface.
Next steps
Learn the basics of predictive analytics and machine learning with Tutorial: Predict automobile price with the
visual interface
Use one of the samples and modify to suite your needs:
Sample 1 - Regression: Predict price
Sample 2 - Regression: Predict price and compare algorithms
Sample 5 - Classification: Predict churn, appetency, and up-selling
What is automated machine learning?
Automated machine learning, also referred to as autoML, is the process of automating the time consuming,
iterative tasks of machine learning model development. It allows data scientists, analysts, and developers to build
ML models with high scale, efficiency, and productivity all while sustaining model quality. Automated ML is based
on a breakthrough from our Microsoft Research division.
Traditional machine learning model development is resource-intensive, requiring significant domain knowledge
and time to produce and compare dozens of models. Apply automated ML when you want Azure Machine
Learning to train and tune a model for you using the target metric you specify. The service then iterates through
ML algorithms paired with feature selections, where each iteration produces a model with a training score. The
higher the score, the better the model is considered to "fit" your data.
With automated machine learning, you'll accelerate the time it takes to get production-ready ML models with
great ease and efficiency.
When to use automated ML

Automated ML democratizes the machine learning model development process, and empowers its users, no
matter their data science expertise, to identify an end-to-end machine learning pipeline for any problem.
Data scientists, analysts and developers across industries can use automated ML to:
Implement machine learning solutions without extensive programming knowledge
Save time and resources
Leverage data science best practices
Provide agile problem-solving
How automated ML works

Using Azure Machine Learning service, you can design and run your automated ML training experiments with
these steps:
1. Identify the ML problem to be solved: classification, forecasting, or regression
2. Specify the source and format of the labeled training data: Numpy arrays or Pandas dataframe
3. Configure the compute target for model training, such as your local computer, Azure Machine
Learning Computes, remote VMs, or Azure Databricks. Learn about automated training on a remote
resource.
4. Configure the automated machine learning parameters that determine how many iterations over
different models, hyperparameter settings, advanced preprocessing/featurization, and what metrics to look
at when determining the best model. You can configure the settings for automatic training experiment in
Azure portal or with the SDK.
5. Submit the training run.
During training, the Azure Machine Learning service creates a number of in parallel pipelines that try different
algorithms and parameters. It will stop once it hits the exit criteria defined in the experiment.
You can also inspect the logged run information, which contains metrics gathered during the run. The training run
produces a Python serialized object ( .pkl file) that contains the model and data preprocessing.
While model building is automated, you can also learn how important or relevant features are to the generated
models.
Preprocessing
In every automated machine learning experiment, your data is preprocessed using the default methods and
optionally through advanced preprocessing.
NOTE
Automatic preprocessing (standard)

In every automated machine learning experiment, your data is automatically scaled or normalized to help
algorithms perform well. During model training, one of the following scaling or normalization techniques will be
applied to each model.
SCALING & NORMALIZATION DESCRIPTION
StandardScaleWrapper Standardize features by removing the mean and scaling to

unit variance
MinMaxScalar Transforms features by scaling each feature by that column’s

minimum and maximum
MaxAbsScaler Scale each feature by its maximum absolute value
RobustScalar This Scaler features by their quantile range

SCALING & NORMALIZATION DESCRIPTION
PCA Linear dimensionality reduction using Singular Value

Decomposition of the data to project it to a lower
dimensional space
TruncatedSVDWrapper This transformer performs linear dimensionality reduction by

means of truncated singular value decomposition (SVD).
Contrary to PCA, this estimator does not center the data
before computing the singular value decomposition. This
means it can work with scipy.sparse matrices efficiently
SparseNormalizer Each sample (that is, each row of the data matrix) with at least
one non-zero component is re-scaled independently of other
samples so that its norm (l1 or l2) equals one
Advanced preprocessing: optional featurization

Additional advanced preprocessing and featurization are also available, such as missing values imputation,
encoding, and transforms. Learn more about what featurization is included. Enable this setting with:
Azure portal: Selecting the Preprocess checkbox in the Advanced settings with these steps.
Python SDK: Specifying "preprocess": True for the AutoMLConfig class.
Time-series forecasting
Building forecasts is an integral part of any business, whether it’s revenue, inventory, sales, or customer demand.
You can use automated ML to combine techniques and approaches and get a recommended, high-quality time-
series forecast.
An automated time-series experiment is treated as a multivariate regression problem. Past time-series values are
“pivoted” to become additional dimensions for the regressor together with other predictors. This approach, unlike
classical time series methods, has an advantage of naturally incorporating multiple contextual variables and their
relationship to one another during training. Automated ML learns a single, but often internally branched model
for all items in the dataset and prediction horizons. More data is thus available to estimate model parameters and
generalization to unseen series becomes possible.
Learn more and see an example of automated machine learning for time series forecasting.
Ensemble models
Automated machine learning supports ensemble models, which are enabled by default. Ensemble learning
improves machine learning results and predictive performance by combining multiple models as opposed to
using single models. The ensemble iterations appear as the final iterations of your run. Automated machine
learning uses both voting and stacking ensemble methods for combining models:
Voting: predicts based on the weighted average of predicted class probabilities (for classification tasks) or
predicted regression targets (for regression tasks).
Stacking: stacking combines heterogenous models and trains a meta-model based on the output from the
individual models. The current default meta-models are LogisticRegression for classification tasks and
ElasticNet for regression/forecasting tasks.
The Caruana ensemble selection algorithm with sorted ensemble initialization is used to decide which models to
use within the ensemble. At a high level, this algorithm initializes the ensemble with up to 5 models with the best
individual scores, and verifies that these models are within 5% threshold of the best score to avoid a poor initial
ensemble. Then for each ensemble iteration, a new model is added to the existing ensemble and the resulting
score is calculated. If a new model improved the existing ensemble score, the ensemble is updated to include the
new model.
See the how -to for changing default ensemble settings in automated machine learning.
Use with ONNX in C# apps

With Azure Machine Learning, you can use automated ML to build a Python model and have it converted to the
ONNX format. The ONNX runtime supports C#, so you can use the model built automatically in your C# apps
without any need for recoding or any of the network latencies that REST endpoints introduce. Try an example of
this flow in this Jupyter notebook.
Automated ML across Microsoft

Automated ML is also available in other Microsoft solutions such as:
INTEGRATIONS DESCRIPTION
ML.NET Automatic model selection and training in .NET apps using

Visual Studio and Visual Studio Code with ML.NET automated
ML (Preview).
HDInsight Scale out your automated ML training jobs on Spark in

HDInsight clusters in parallel.
Power BI Invoke machine learning models directly in Power BI

(Preview).
SQL Server Create new machine learning models over your data in SQL
Server 2019 big data clusters.
Next steps
See examples and learn how to build models using automated machine learning:
Follow the Tutorial: Automatically train a regression model with Azure Automated Machine Learning
Configure the settings for automatic training experiment:
In Azure portal interface, use these steps.
With the Python SDK, use these steps.
Learn how to auto train using time series data, use these steps.
Try out Jupyter Notebook samples for automated machine learning
MLOps: Manage, deploy, and monitor models with
Azure Machine Learning Service
In this article, learn about how to use Azure Machine Learning service to manage the lifecycle of your models.
Azure Machine Learning uses a Machine Learning Operations (MLOps) approach, which improves the quality
and consistency of your machine learning solutions.
Azure Machine Learning Service provides the following MLOps capabilities:
Deploy ML projects from anywhere
Monitor ML applications for operational and ML related issues - compare model inputs between
training and inference, explore model-specific metrics and provide monitoring and alerts on your ML
infrastructure.
Capture the data required for establishing an end to end audit trail of the ML lifecycle, including who
is publishing models, why changes are being made, and when models were deployed or used in production.
Automate the end to end ML lifecycle with Azure Machine Learning and Azure DevOps to frequently
update models, test new models, and continuously roll out new ML models alongside your other applications
and services.
To hear more on the concepts behind MLOps and how they apply to the Azure Machine Learning service, watch
the following video.
Deploy ML projects from anywhere

Turn your training process into a reproducible pipeline
Use ML pipelines from Azure Machine Learning to stitch together all of the steps involved in your model training
process, from data preparation to feature extraction to hyperparameter tuning to model evaluation.
For more information, see ML pipelines.
Register and track ML models
Model registration allows you to store and version your models in the Azure cloud, in your workspace. The
model registry makes it easy to organize and keep track of your trained models.
TIP
A registered model is a logical container for one or more files that make up your model. For example, if you have a model
that is stored in multiple files, you can register them as a single model in your Azure Machine Learning workspace. After
registration, you can then download or deploy the registered model and receive all the files that were registered.
Registered models are identified by name and version. Each time you register a model with the same name as an
existing one, the registry increments the version. You can also provide additional metadata tags during
registration that can be used when searching for models. The Azure Machine Learning service supports any
model that can be loaded using Python 3.5.2 or higher.
TIP
You can also register models trained outside the Azure Machine Learning service.
You can't delete a registered model that is being used in an active deployment. For more information, see the
register model section of Deploy models.
Package and debug models
Before deploying a model into production, it is packaged into a Docker image. In most cases, image creation
happens automatically in the background during deployment. For advanced scenarios, you can manually specify
the image.
If you run into problems with the deployment, you can deploy on your local development environment for
troubleshooting and debugging.
For more information, see Deploy models and Troubleshooting deployments.
Validate and profile models
Azure Machine Learning service can use profiling to determine the ideal CPU and memory settings to use when
deploying your model. Model validation happens as part of this process, using data that you supply for the
profiling process.
Convert and optimize models
Converting your model to Open Neural Network Exchange (ONNX) may improve performance. On average,
converting to ONNX can yield a 2x performance increase.
For more information on ONNX with Azure Machine Learning service, see the Create and accelerate ML models
article.
Use models
Trained machine learning models can be deployed as web services in the cloud or locally on your development
environment. You can also deploy models to Azure IoT Edge devices. Deployments can use CPU, GPU, or field-
programmable gate arrays (FPGA) for inferencing. You can also use models from Power BI.
When using a model as a web service or IoT Edge device, you provide the following items:
The model(s) that are used to score data submitted to the service/device.
An entry script. This script accepts requests, uses the model(s) to score the data, and return a response.
A conda environment file that describes the dependencies required by the model(s) and entry script.
Any additional assets such as text, data, etc. that are required by the model(s) and entry script.
These assets are packaged into a Docker image, and deployed as a web service or IoT Edge module.
Optionally, you can use the following parameters to further tune the deployment:
Enable GPU: Used to enable GPU support in the Docker image. The image must be used on Microsoft Azure
Services such as Azure Container Instances, Azure Kubernetes Service, Azure Machine Learning Compute, or
Azure Virtual Machines.
Extra docker file steps: A file that contains additional Docker steps to run when creating the Docker image.
Base image: A custom image to use as the base image. If you do not use a custom image, the base image is
provided by the Azure Machine Learning service.
You also provide the configuration of the target deployment platform. For example, the VM family type, available
memory, and number of cores when deploying to Azure Kubernetes Service.
When the image is created, components required by Azure Machine Learning Service are also added. For
example, assets needed to run the web service and interact with IoT Edge.
NOTE
You cannot modify or change the web server or IoT Edge components used in the Docker image. Azure Machine Learning
service uses a web server configuration and IoT Edge components that are tested and supported by Microsoft.
Web service
You can use your models in web services with the following compute targets:
Azure Container Instance
Azure Kubernetes Service
Local development environment
To deploy the model as a web service, you must provide the following items:
The model or ensemble of models.
Dependencies required to use the model. For example, a script that accepts requests and invokes the model,
conda dependencies, etc.
Deployment configuration that describes how and where to deploy the model.
For more information, see Deploy models.
IoT Edge devices
You can use models with IoT devices through Azure IoT Edge modules. IoT Edge modules are deployed to a
hardware device, which enables inference, or model scoring, on the device.
For more information, see Deploy models.
Analytics
Microsoft Power BI supports using machine learning models for data analytics. For more information, see Azure
Machine Learning integration in Power BI (Preview ).
Monitor ML applications for operational and ML related issues

Monitoring enables you to understand what data is being sent to your model, and the predictions that it returns.
This information helps you understand how your model is being used. The collected input data may also be
useful in training future versions of the model.
For more information, see How to enable model data collection.
Capture an end to end audit trail of the ML lifecycle

Azure ML gives you the capability to track the end to end audit trail of all of your ML assets. Specifically:
Azure ML integrates with Git to track information which repository / branch / commit your code came from.
Azure ML Datasets help you track and version data.
Azure ML Run history stores a snapshot of the code, data and compute used to train a model.
The Azure ML Model Registry captures all of the metadata associated with your model (which experiment
trained it, where it is being deployed, if its deployments are healthy).
Automate the end to end ML lifecycle

You can use GitHub and Azure Pipelines to create a continuous integration process that trains a model. In a
typical scenario, when a Data Scientist checks a change into the Git repo for a project, the Azure Pipeline will start
a training run. The results of the run can then be inspected to see the performance characteristics of the trained
model. You can also create a pipeline that deploys the model as a web service.
The Azure Machine Learning extension makes it easier to work with Azure Pipelines. It provides the following
enhancements to Azure Pipelines:
Enables workspace selection when defining a service connection.
Enables release pipelines to be triggered by trained models created in a training pipeline.
For more information on using Azure Pipelines with Azure Machine Learning, see the Continuous integration
and deployment of ML models with Azure Pipelines article and the Azure Machine Learning Service MLOps
repository.
Next steps
Learn more about how and where you can deploy models with the Azure Machine Learning service. For an
example of deployment, see Tutorial: Deploy an image classification model in Azure Container Instances.
Learn how to create continuous integration and deployment of ML models with Azure Pipelines.
Learn how to create client applications and services that Consume a model deployed as a web service.
What are ML pipelines in Azure Machine Learning
service?
Learn how about the machine learning pipelines you can build and manage with Azure Machine Learning
service.
Using machine learning (ML ) pipelines, data scientists, data engineers, and IT professionals can collaborate on
the steps involved in:
Data preparation, such as normalizations and transformations
Model training
Model evaluation
Deployment
Learn how to create your first pipeline.
Which Azure pipeline technology should I use?

The Azure cloud provides several other pipelines, each with a different purpose. The following table lists the
different pipelines and what they are used for:
PIPELINE WHAT IT DOES CANONICAL PIPE
Azure Machine Learning pipelines Defines reusable machine learning Data -> model
workflows that can be used as a
template for your machine learning
scenarios.
Azure Data Factory pipelines Groups data movement, Data -> data
transformation, and control activities
needed to perform a task.
Azure Pipelines Continuous integration and delivery of Code -> app/service

your application to any platform/any
cloud
Why build pipelines with Azure Machine Learning?

Machine learning pipelines optimize your workflow with speed, portability, and reuse so you can focus on your
expertise, machine learning, rather than on infrastructure and automation.
Pipelines are constructed from multiple steps, which are distinct computational units in the pipeline. Each step
can run independently and use isolated compute resources. Independent steps allow multiple data scientists to
work on the same pipeline at the same time without over-taxing compute resources, and also makes it easy to
use different compute types/sizes for each step.
After the pipeline is designed, there is often more fine-tuning around the training loop of the pipeline. When you
rerun a pipeline, the run jumps to the distinct steps that need to be rerun, such as an updated training script, and
skips what hasn't changed. The same paradigm applies to unchanged scripts used for the execution of the step.
This reuse functionality helps to avoid running costly and time-intensive steps like data ingestion and
transformation if the underlying data hasn't changed.
With Azure Machine Learning, you can use various toolkits and frameworks, such as PyTorch or TensorFlow, for
each step in your pipeline. Azure coordinates between the various compute targets you use, so that your
intermediate data can be shared with the downstream compute targets easily.
You can track the metrics for your pipeline experiments directly in the Azure portal. After a pipeline has been
published, you can configure a REST endpoint, which allows you to rerun the pipeline from any platform or
stack.
Key advantages
The key advantages of using pipelines for your machine learning workflows are:
KEY ADVANTAGE DESCRIPTION
Unattended runs Schedule steps to run in parallel or in sequence in a reliable

and unattended manner. Data preparation and modeling can
last days or weeks, and pipelines allow you to focus on other
tasks while the process is running.
Heterogenous compute Use multiple pipelines that are reliably coordinated across
heterogeneous and scalable compute resources and storage
locations. Make efficient use of available compute resources
by running individual pipeline steps on different compute
targets, such as HDInsight, GPU Data Science VMs, and
Databricks.
Reusability Create pipeline templates for specific scenarios, such as

retraining and batch-scoring. Trigger published pipelines
from external systems via simple REST calls.
Tracking and versioning Instead of manually tracking data and result paths as you
iterate, use the pipelines SDK to explicitly name and version
your data sources, inputs, and outputs. You can also manage
scripts and data separately for increased productivity.
Collaboration Pipelines allow data scientists to collaborate across all areas

of the machine learning design process, while being able to
concurrently work on pipeline steps.
The Python SDK for pipelines

Use the Python SDK to create your ML pipelines in your preferred integrated development environment (IDE )
or Jupyter notebooks. The Azure Machine Learning SDK offers imperative constructs for sequencing and
parallelizing the steps in your pipelines when no data dependency is present.
Using declarative data dependencies, you can optimize your tasks. The SDK includes a framework of pre-built
modules for common tasks, such as data transfer and model publishing. You can extend the framework to model
your own conventions by implementing custom steps reusable across pipelines. You can also manage compute
targets and storage resources directly from the SDK.
Save your pipelines as templates, and deploy them to a REST endpoint for batch-scoring or retraining jobs.
There are two Python packages for pipelines with Azure Machine Learning: azureml-pipelines-core and
azureml-pipeline-steps. To get started quickly, use one of the pre-built modules such as:
Running Python script in a step with PythonScriptStep
Transferring data between storage options with DataTransferStep
Creating an AutoML pipeline step with AutoMLStep
Next steps
Learn how to create your first pipeline.
Learn how to run batch predictions on large data.
See the SDK reference docs for pipelines.
Try out example Jupyter notebooks showcasing Azure Machine Learning pipelines. Learn how to run
notebooks to explore this service.
Model interpretability with Azure Machine Learning
service
In this article, you learn how to explain why your model made the predictions it did with the various interpretability
packages of the Azure Machine Learning Python SDK.
Using the classes and methods in the SDK, you can get:
Feature importance values for both raw and engineered features
Interpretability on real-world datasets at scale, during training and inference.
Interactive visualizations to aid you in the discovery of patterns in data and explanations at training time
During the training phase of the development cycle, model designers and evaluators can use interpretability output
of a model to verify hypotheses and build trust with stakeholders. They also use the insights into the model for
debugging, validating model behavior matches their objectives, and to check for bias.
In machine learning, features are the data fields used to predict a target data point. For example, to predict credit
risk, data fields for age, account size, and account age might be used. In this case, age, account size, and account
age are features. Feature importance tells you how each data field affected the model's predictions. For example,
age may be heavily used in the prediction while account size and age don't affect the prediction accuracy
significantly. This process allows data scientists to explain resulting predictions, so that stakeholders have visibility
into what data points are most important in the model.
Using these tools, you can explain machine learning models globally on all data, or locally on a specific data
point using the state-of-art technologies in an easy-to-use and scalable fashion.
The interpretability classes are made available through multiple SDK packages. Learn how to install SDK packages
for Azure Machine Learning.
azureml.explain.model , the main package, containing functionalities supported by Microsoft.
azureml.contrib.explain.model , preview, and experimental functionalities that you can try.
azureml.train.automl.automlexplainer package for interpreting automated machine learning models.
IMPORTANT
Content in the contrib namespace is not fully supported. As the experimental functionalities become mature, they will
gradually be moved to the main namespace.
How to interpret your model

You can apply the interpretability classes and methods to understand the model’s global behavior or specific
predictions. The former is called global explanation and the latter is called local explanation.
The methods can be also categorized based on whether the method is model agnostic or model specific. Some
methods target certain type of models. For example, SHAP’s tree explainer only applies to tree-based models.
Some methods treat the model as a black box, such as mimic explainer or SHAP’s kernel explainer. The explain
package leverages these different approaches based on data sets, model types, and use cases.
The output is a set of information on how a given model makes its prediction, such as:
Global/local relative feature importance
Global/local feature and prediction relationship
Explainers
There are two sets of explainers: Direct Explainers and Meta Explainers in the SDK.
Direct explainers come from integrated libraries. The SDK wraps all the explainers so that they expose a common
API and output format. If you are more comfortable directly using these explainers, you can directly invoke them
instead of using the common API and output format. The following are a list of the direct explainers available in
the SDK:
SHAP Tree Explainer: SHAP’s tree explainer, which focuses on polynomial time fast SHAP value
estimation algorithm specific to trees and ensembles of trees.
SHAP Deep Explainer: Based on the explanation from SHAP, Deep Explainer "is a high-speed
approximation algorithm for SHAP values in deep learning models that builds on a connection with
DeepLIFT described in the SHAP NIPS paper. TensorFlow models and Keras models using the TensorFlow
backend are supported (there is also preliminary support for PyTorch)".
SHAP Kernel Explainer: SHAP's Kernel explainer uses a specially weighted local linear regression to
estimate SHAP values for any model.
Mimic Explainer: Mimic explainer is based on the idea of global surrogate models. A global surrogate
model is an intrinsically interpretable model that is trained to approximate the predictions of a black box
model as accurately as possible. Data scientist can interpret the surrogate model to draw conclusions about
the black box model. You can use one of the following interpretable models as your surrogate model:
LightGBM (LinearExplainableModel), Linear Regression (LinearExplainableModel), Stochastic Gradient
Descent explainable model (SGDExplainableModel), and Decision Tree (DecisionTreeExplainableModel).
Permutation Feature Importance Explainer: Permutation Feature Importance is a technique used to
explain classification and regression models that is inspired by Breiman's Random Forests paper (see
section 10). At a high level, the way it works is by randomly shuffling data one feature at a time for the
entire dataset and calculating how much the performance metric of interest decreases. The larger the
change, the more important that feature is.
LIME Explainer ( contrib ): Based on LIME, LIME Explainer uses the state-of-the-art Local interpretable
model-agnostic explanations (LIME ) algorithm to create local surrogate models. Unlike the global surrogate
models, LIME focuses on training local surrogate models to explain individual predictions.
HAN Text Explainer ( contrib ): HAN Text Explainer uses a Hierarchical Attention Network for getting
model explanations from text data for a given black box text model. We train the HAN surrogate model on a
given teacher model's predicted outputs. After training globally across the text corpus, we have added a
fine-tune step for a specific document in order to improve the accuracy of the explanations. HAN uses a
bidirectional RNN with two attention layers, for sentence and word attention. Once the DNN is trained on
the teacher model and fine-tuned on a specific document, we can extract the word importances from the
attention layers. We have found HAN to be more accurate than LIME or SHAP for text data but more costly
in terms of training time as well. However, we have made improvements to the training time by giving the
user the option to initialize the network with GloVe word embeddings, although it is still slow. The training
time can be improved significantly by running HAN on a remote Azure GPU VM. The implementation of
HAN is described in 'Hierarchical Attention Networks for Document Classification (Yang et al., 2016)'.
Meta explainers automatically select a suitable direct explainer and generate the best explanation info based on
the given model and data sets. The meta explainers leverage all the libraries (SHAP, LIME, Mimic, etc.) that we
have integrated or developed. The following are the meta explainers available in the SDK:
Tabular Explainer: Used with tabular datasets.
Text Explainer: Used with text datasets.
Image Explainer: Used with image datasets.
In addition to Meta-selecting of the direct explainers, meta explainers develop additional features on top of the
underlying libraries and improve the speed and scalability over the direct explainers.
Currently TabularExplainer employs the following logic to invoke the Direct SHAP Explainers:
1. If it's a tree-based model, apply SHAP TreeExplainer , else
2. If it's a DNN model, apply SHAP DeepExplainer , else
3. Treat it as a black-box model and apply SHAP KernelExplainer
The intelligence built into TabularExplainer will become more sophisticated as more libraries are integrated into
the SDK and we learn about pros and cons of each explainer.
TabularExplainer has also made significant feature and performance enhancements over the Direct Explainers:
Summarization of the initialization dataset. In cases where speed of explanation is most important, we
summarize the initialization dataset and generate a small set of representative samples, which speeds up both
global and local explanation.
Sampling the evaluation data set. If the user passes in a large set of evaluation samples but doesn't actually
need all of them to be evaluated, the sampling parameter can be set to true to speed up the global explanation.
The following diagram shows the current structure of direct and meta explainers.
Models supported
Any models that are trained on datasets in Python numpy.array , pandas.DataFrame , iml.datatypes.DenseData , or
scipy.sparse.csr_matrix format are supported by the interpretability explain package of the SDK.
The explanation functions accept both models and pipelines as input. If a model is provided, the model must
implement the prediction function predict or predict_proba that conforms to the Scikit convention. If a pipeline
(name of the pipeline script) is provided, the explanation function assumes that the running pipeline script returns
a prediction. We support models trained via PyTorch, TensorFlow, and Keras deep learning frameworks.
Local and remote compute target
The explain package is designed to work with both local and remote compute targets. If run locally, The SDK
functions will not contact any Azure services. You can run explanation remotely on Azure Machine Learning
Compute and log the explanation info into Azure Machine Learning Run History Services. Once this information is
logged, reports and visualizations from the explanation are readily available on Azure Machine Learning
Workspace portal for user analysis.
Interpretability in training
Train and explain locally
1. Train your model in a local Jupyter notebook.
# load breast cancer dataset, a well-known small dataset that comes with scikit-learn
from sklearn.datasets import load_breast_cancer
from sklearn import svm
breast_cancer_data = load_breast_cancer()
classes = breast_cancer_data.target_names.tolist()
# split data into train and test

x_train, x_test, y_train, y_test = train_test_split(breast_cancer_data.data,
breast_cancer_data.target,
test_size=0.2,
random_state=0)
clf = svm.SVC(gamma=0.001, C=100., probability=True)
model = clf.fit(x_train, y_train)
2. Call the explainer: To initialize an explainer object, you need to pass your model and some training data to
the explainer's constructor. You can also optionally pass in feature names and output class names (if doing
classification) which will be used to make your explanations and visualizations more informative. Here is
how to instantiate an explainer object using TabularExplainer, MimicExplainer, and PFIExplainer locally.
TabularExplainer is calling one of the three SHAP explainers underneath ( TreeExplainer , DeepExplainer ,
or KernelExplainer ), and is automatically selecting the most appropriate one for your use case. You can
however, call each of its three underlying explainers directly.
from azureml.explain.model.tabular_explainer import TabularExplainer

# "features" and "classes" fields are optional
explainer = TabularExplainer(model,
x_train,
features=breast_cancer_data.feature_names,
classes=classes)
or
from azureml.explain.model.mimic.mimic_explainer import MimicExplainer
# you can use one of the following four interpretable models as a global surrogate to the black box
model
from azureml.explain.model.mimic.models.lightgbm_model import LGBMExplainableModel
from azureml.explain.model.mimic.models.linear_model import LinearExplainableModel
from azureml.explain.model.mimic.models.linear_model import SGDExplainableModel
from azureml.explain.model.mimic.models.tree_model import DecisionTreeExplainableModel

# augment_data is optional and if true, oversamples the initialization examples to improve surrogate
model accuracy to fit original model. Useful for high-dimensional data where the number of rows is less
than the number of columns.
# max_num_of_augmentations is optional and defines max number of times we can increase the input data
size.
# LGBMExplainableModel can be replaced with LinearExplainableModel, SGDExplainableModel, or
DecisionTreeExplainableModel
explainer = MimicExplainer(model,
x_train,
LGBMExplainableModel,
augment_data=True,
max_num_of_augmentations=10,
classes=classes)
or
from azureml.explain.model.permutation.permutation_importance import PFIExplainer

explainer = PFIExplainer(model,
classes=classes)
3. Get the global feature importance values.
# you can use the training data or the test data here
global_explanation = explainer.explain_global(x_train)
# if you used the PFIExplainer in the previous step, use the next line of code instead
# global_explanation = explainer.explain_global(x_train, true_labels=y_test)
# sorted feature importance values and feature names

sorted_global_importance_values = global_explanation.get_ranked_global_values()
sorted_global_importance_names = global_explanation.get_ranked_global_names()
dict(zip(sorted_global_importance_names, sorted_global_importance_values))
# alternatively, you can print out a dictionary that holds the top K feature names and values
global_explanation.get_feature_importance_dict()
4. Get the local feature importance values: use the following function calls to explain an individual instance or
a group of instances. Please note that PFIExplainer does not support local explanations.
# explain the first data point in the test set

local_explanation = explainer.explain_local(x_test[0])

sorted_local_importance_names = local_explanation.get_ranked_local_names()
sorted_local_importance_values = local_explanation.get_ranked_local_values()
or
# explain the first five data points in the test set
local_explanation = explainer.explain_local(x_test[0:4])

sorted_local_importance_names = local_explanation.get_ranked_local_names()
sorted_local_importance_values = local_explanation.get_ranked_local_values()
Train and explain remotely

While you can train on the various compute targets supported by Azure Machine Learning service, the example in
this section shows how to do this using an Azure Machine Learning Compute target.
1. Create a training script in a local Jupyter notebook (for example, run_explainer.py).
from azureml.contrib.explain.model.explanation.explanation_client import ExplanationClient

from azureml.core.run import Run
client = ExplanationClient.from_run(run)
# write code to get and split your data into train and test sets here
# write code to train your model here
# explain predictions on your local machine

x_train,
features=feature_names,
classes=classes)
# explain overall model predictions (global explanation)

global_explanation = explainer.explain_global(x_test)
# uploading global model explanation data for storage or visualization in webUX

# the explanation can then be downloaded on any compute
# multiple explanations can be uploaded
client.upload_model_explanation(global_explanation, comment='global explanation: all features')
# or you can only upload the explanation object with the top k feature info
#client.upload_model_explanation(global_explanation, top_k=2, comment='global explanation: Only top 2
features')
2. Follow the instructions on Set up compute targets for model training to learn about how to set up an Azure
Machine Learning Compute as your compute target and submit your training run.
3. Download the explanation in your local Jupyter notebook.
from azureml.contrib.explain.model.explanation.explanation_client import ExplanationClient
client = ExplanationClient.from_run(run)
# get model explanation data

explanation = client.download_model_explanation()
# or only get the top k (e.g., 4) most important features with their importance values
explanation = client.download_model_explanation(top_k=4)
global_importance_values = explanation.get_ranked_global_values()
global_importance_names = explanation.get_ranked_global_names()
print('global importance values: {}'.format(global_importance_values))
print('global importance names: {}'.format(global_importance_names))
Visualizations
Use the visualization dashboard to understand and interpret your model:
Global visualizations
The following plots provide a global view of the trained model along with its predictions and explanations.
PLOT DESCRIPTION
Data Exploration An overview of the dataset along with prediction values.
Global Importance Shows the top K (configurable K) important features globally.

This chart is useful for understanding the global behavior of
the underlying model.
Explanation Exploration Demonstrates how a feature is responsible for making a

change in model’s prediction values (or probability of
prediction values).
Summary Uses a signed local feature importance values across all data
points to show the distribution of the impact each feature has
on the prediction value.
Local visualizations
You can click on any individual data point at any time of the preceding plots to load the local feature importance
plot for the given data point.
PLOT DESCRIPTION
Local Importance Shows the top K (configurable K) important features globally.

This chart is useful for understanding the local behavior of the
underlying model on a specific data point.
Perturbation Exploration Allows you to change feature values of the selected data point
and observe how those changes will affect prediction value.
PLOT DESCRIPTION
Individual Conditional Expectation (ICE) Allows you to change a feature value from a minimum value
to a maximum value to see how the data point's prediction
changes when a feature changes.
Note you will need to have widget extensions of the visualization dashboard enabled prior to Jupyter kernel
starting.
Jupyter notebooks
jupyter nbextension install --py --sys-prefix azureml.contrib.explain.model.visualize

jupyter nbextension enable --py --sys-prefix azureml.contrib.explain.model.visualize
Jupyter Labs
jupyter labextension install @jupyter-widgets/jupyterlab-manager

jupyter labextension install microsoft-mli-widget
To load the visualization dashboard, use the following code:
from azureml.contrib.explain.model.visualize import ExplanationDashboard
ExplanationDashboard(global_explanation, model, x_test)
Raw feature transformations

Optionally, you can pass your feature transformation pipeline to the explainer to receive explanations in terms of
the raw features before the transformation (rather than engineered features). If you skip this, the explainer
provides explanations in terms of engineered features.
The format of supported transformations is same as the one described in sklearn-pandas. In general, any
transformations are supported as long as they operate on a single column and are therefore clearly one to many.
We can explain raw features by either using a sklearn.compose.ColumnTransformer or a list of fitted transformer
tuples. The cell below uses sklearn.compose.ColumnTransformer .
from sklearn.compose import ColumnTransformer
numeric_transformer = Pipeline(steps=[
('imputer', SimpleImputer(strategy='median')),
('scaler', StandardScaler())])
categorical_transformer = Pipeline(steps=[
('imputer', SimpleImputer(strategy='constant', fill_value='missing')),
('onehot', OneHotEncoder(handle_unknown='ignore'))])
preprocessor = ColumnTransformer(
transformers=[
('num', numeric_transformer, numeric_features),
('cat', categorical_transformer, categorical_features)])
# append classifier to preprocessing pipeline.

# now we have a full prediction pipeline.
clf = Pipeline(steps=[('preprocessor', preprocessor),
('classifier', LogisticRegression(solver='lbfgs'))])
# append classifier to preprocessing pipeline.

clf = Pipeline(steps=[('preprocessor', preprocessor),
# clf.steps[-1][1] returns the trained classification model

# pass transformation as an input to create the explanation object
tabular_explainer = TabularExplainer(clf.steps[-1][1],
initialization_examples=x_train,
features=dataset_feature_names,
classes=dataset_classes,
transformations=preprocessor)
In case you want to run the example with the list of fitted transformer tuples, use the following code:
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn_pandas import DataFrameMapper
# assume that we have created two arrays, numerical and categorical, which holds the numerical and categorical
feature names
numeric_transformations = [([f], Pipeline(steps=[('imputer', SimpleImputer(

strategy='median')), ('scaler', StandardScaler())])) for f in numerical]
categorical_transformations = [([f], OneHotEncoder(

handle_unknown='ignore', sparse=False)) for f in categorical]
transformations = numeric_transformations + categorical_transformations
# append model to preprocessing pipeline.

clf = Pipeline(steps=[('preprocessor', DataFrameMapper(transformations)),
# clf.steps[-1][1] returns the trained classification model

# pass transformation as an input to create the explanation object
tabular_explainer = TabularExplainer(clf.steps[-1][1],
transformations=transformations)
Interpretability at inferencing time

The explainer can be deployed along with the original model and can be used at scoring time to provide the local
explanation information. We also offer lighter-weight scoring explainers to make interpretability at inferencing
time more performant. The process of deploying a lighter-weight scoring explainer is similar to deploying a model
and includes the following steps:
1. Create an explanation object (e.g., using TabularExplainer):
from azureml.contrib.explain.model.tabular_explainer import TabularExplainer
transformations=transformations)
2. Create a scoring explainer using the explanation object:
from azureml.contrib.explain.model.scoring.scoring_explainer import KernelScoringExplainer, save
# create a lightweight explainer at scoring time

scoring_explainer = KernelScoringExplainer(explainer)
# pickle scoring explainer

# pickle scoring explainer locally
OUTPUT_DIR = 'my_directory'
save(scoring_explainer, directory=OUTPUT_DIR, exist_ok=True)
3. Configure and register an image that uses the scoring explainer model.
# register explainer model using the path from ScoringExplainer.save - could be done on remote compute
# scoring_explainer.pkl is the filename on disk, while my_scoring_explainer.pkl will be the filename in
cloud storage
run.upload_file('my_scoring_explainer.pkl', os.path.join(OUTPUT_DIR, 'scoring_explainer.pkl'))
scoring_explainer_model = run.register_model(model_name='my_scoring_explainer',
model_path='my_scoring_explainer.pkl')
print(scoring_explainer_model.name, scoring_explainer_model.id, scoring_explainer_model.version, sep =
'\t')
4. [Optional] Retrieve the scoring explainer from cloud and test the explanations
from azureml.contrib.explain.model.scoring.scoring_explainer import load
# retrieve the scoring explainer model from cloud"

scoring_explainer_model = Model(ws, 'my_scoring_explainer')
scoring_explainer_model_path = scoring_explainer_model.download(target_dir=os.getcwd(), exist_ok=True)
# load scoring explainer from disk

scoring_explainer = load(scoring_explainer_model_path)
# test scoring explainer locally

preds = scoring_explainer.explain(x_test)
print(preds)
5. Deploy the image to a compute target:

a. Create a scoring file (before this step, follow the steps in Deploy models with the Azure Machine
Learning service to register your original prediction model)
%%writefile score.py
import json
import numpy as np
import pandas as pd
import os
import pickle
def init():
global original_model
global scoring_model
# retrieve the path to the model file using the model name
# assume original model is named original_prediction_model
original_model_path = Model.get_model_path('original_prediction_model')
scoring_explainer_path = Model.get_model_path('my_scoring_explainer')
original_model = joblib.load(original_model_path)
scoring_explainer = joblib.load(scoring_explainer_path)
def run(raw_data):
# get predictions and explanations for each data point
data = pd.read_json(raw_data)
# make prediction
predictions = original_model.predict(data)
# retrieve model explanations
local_importance_values = scoring_explainer.explain(data)
# you can return any data type as long as it is JSON-serializable
return {'predictions': predictions.tolist(), 'local_importance_values':
local_importance_values}
b. Define the deployment configuration (This configuration depends on the requirements of your
model. The following example defines a configuration that uses one CPU core and 1 GB of memory)
aciconfig = AciWebservice.deploy_configuration(cpu_cores=1,
memory_gb=1,
tags={"data": "NAME_OF_THE_DATASET",
"method" : "local_explanation"},
description='Get local explanations for
NAME_OF_THE_PROBLEM')
c. Create a file with environment dependencies

# WARNING: to install this, g++ needs to be available on the Docker image and is not by default
(look at the next cell)
azureml_pip_packages = ['azureml-defaults', 'azureml-contrib-explain-model', 'azureml-core',

'azureml-telemetry', 'azureml-explain-model']
# specify CondaDependencies obj

myenv = CondaDependencies.create(conda_packages=['scikit-learn', 'pandas'],
pip_packages=['sklearn-pandas'] + azureml_pip_packages,
pin_sdk_version=False)
with open("myenv.yml","w") as f:
with open("myenv.yml","r") as f:
print(f.read())
d. Create a custom dockerfile with g++ installed
%%writefile dockerfile
RUN apt-get update && apt-get install -y g++
e. Deploy the created image (time estimate: 5 minutes)

from azureml.core.image import ContainerImage
# use the custom scoring, docker, and conda files we created above
image_config = ContainerImage.image_configuration(execution_script="score.py",
docker_file="dockerfile",
runtime="python",
# use configs and models generated above

service = Webservice.deploy_from_model(workspace=ws,
name='model-scoring-service',
deployment_config=aciconfig,
models=[scoring_explainer_model, original_model],
image_config=image_config)
6. Test the deployment

import requests
# create data to test service with

examples = x_list[:4]
input_data = examples.to_json()
headers = {'Content-Type':'application/json'}
# send request to service

resp = requests.post(service.scoring_uri, input_data, headers=headers)
print("POST to url", service.scoring_uri)

# can covert back to Python objects from json string if desired
7. Clean up: To delete a deployed web service, use service.delete() .
Next steps
To see a collection of Jupyter notebooks that demonstrate the instructions above, see the Azure Machine Learning
Interpretability sample notebooks.
ONNX and Azure Machine Learning: Create and
accelerate ML models
Learn how using the Open Neural Network Exchange (ONNX) can help optimize the inference of your machine
learning model. Inference, or model scoring, is the phase where the deployed model is used for prediction, most
commonly on production data.
Optimizing machine learning models for inference (or model scoring) is difficult since you need to tune the model
and the inference library to make the most of the hardware capabilities. The problem becomes extremely hard if
you want to get optimal performance on different kinds of platforms (cloud/edge, CPU/GPU, etc.), since each one
has different capabilities and characteristics. The complexity increases if you have models from a variety of
frameworks that need to run on a variety of platforms. It's very time consuming to optimize all the different
combinations of frameworks and hardware. A solution to train once in your preferred framework and run
anywhere on the cloud or edge is needed. This is where ONNX comes in.
Microsoft and a community of partners created ONNX as an open standard for representing machine learning
models. Models from many frameworks including TensorFlow, PyTorch, SciKit-Learn, Keras, Chainer, MXNet, and
MATLAB can be exported or converted to the standard ONNX format. Once the models are in the ONNX format,
they can be run on a variety of platforms and devices.
ONNX Runtime is a high-performance inference engine for deploying ONNX models to production. It's optimized
for both cloud and edge and works on Linux, Windows, and Mac. Written in C++, it also has C, Python, and C#
APIs. ONNX Runtime provides support for all of the ONNX-ML specification and also integrates with accelerators
on different hardware such as TensorRT on NVidia GPUs.
The ONNX Runtime is used in high scale Microsoft services such as Bing, Office, and Cognitive Services.
Performance gains are dependent on a number of factors but these Microsoft services have seen an average 2x
performance gain on CPU. ONNX Runtime is also used as part of Windows ML on hundreds of millions of
devices. You can use the runtime with Azure Machine Learning services. By using ONNX Runtime, you can benefit
from the extensive production-grade optimizations, testing, and ongoing improvements.
Get ONNX models

You can obtain ONNX models in several ways:
Train a new ONNX model in Azure Machine Learning service (see examples at the bottom of this article)
Convert existing model from another format to ONNX (see the tutorials)
Get a pre-trained ONNX model from the ONNX Model Zoo (see examples at the bottom of this article)
Generate a customized ONNX model from Azure Custom Vision service
Many models including image classification, object detection, and text processing can be represented as ONNX
models. However some models may not be able to be converted successfully. If you run into this situation, please
file an issue in the GitHub of the respective converter that you used. You can continue using your existing format
model until the issue is addressed.
Deploy ONNX models in Azure

With Azure Machine Learning service, you can deploy, manage, and monitor your ONNX models. Using the
standard deployment workflow and ONNX Runtime, you can create a REST endpoint hosted in the cloud. See
example Jupyter notebooks at the end of this article to try it out for yourself.
Install and use ONNX Runtime with Python
Python packages for ONNX Runtime are available on PyPi.org (CPU, GPU ). Please read system requirements
before installation.
To install ONNX Runtime for Python, use one of the following commands:
pip install onnxruntime # CPU build

pip install onnxruntime-gpu # GPU build
To call ONNX Runtime in your Python script, use:
import onnxruntime
session = onnxruntime.InferenceSession("path to model")
The documentation accompanying the model usually tells you the inputs and outputs for using the model. You can
also use a visualization tool such as Netron to view the model. ONNX Runtime also lets you query the model
metadata, inputs, and outputs:
session.get_modelmeta()
first_input_name = session.get_inputs()[0].name
first_output_name = session.get_outputs()[0].name
To inference your model, use run and pass in the list of outputs you want returned (leave empty if you want all of
them) and a map of the input values. The result is a list of the outputs.
results = session.run(["output1", "output2"], {

"input1": indata1, "input2": indata2})
results = session.run([], {"input1": indata1, "input2": indata2})
For the complete Python API reference, see the ONNX Runtime reference docs.
Examples
See how -to-use-azureml/deployment/onnx for example notebooks that create and deploy ONNX models.
Learn how to run notebooks by following the article, Use Jupyter notebooks to explore this service.
More info
Learn more about ONNX or contribute to the project:
ONNX project website
ONNX code on GitHub
Learn more about ONNX Runtime or contribute to the project:
ONNX Runtime GitHub Repo
Enterprise security for the Azure Machine Learning
service
In this article, you'll learn about security features available for the Azure Machine Learning service.
When you use a cloud service, a best practice is to restrict access to only the users who need it. Start by
understanding the authentication and authorization model used by the service. You might also want to restrict
network access or securely join resources in your on-premises network with the cloud. Data encryption is also vital,
both at rest and while data moves between services. Finally, you need to be able to monitor the service and
produce an audit log of all activity.
Authentication
Multi-factor authentication is supported if Azure Active Directory (Azure AD ) is configured to use it. Here's the
authentication process:
1. The client signs in to Azure AD and gets an Azure Resource Manager token. Users and service principals are
fully supported.
2. The client presents the token to Azure Resource Manager and to all Azure Machine Learning services.
3. The Machine Learning service provides a Machine Learning service token to the user compute target (for
example, Machine Learning Compute). This token is used by the user compute target to call back into the
Machine Learning service after the run is complete. Scope is limited to the workspace.
Authentication for web service deployment

Azure Machine Learning supports two forms of authentication for web services: key and token. Each web service
can enable only one form of authentication at a time.
AUTHENTICATION METHOD AZURE CONTAINER INSTANCES AKS
Key Disabled by default Enabled by default
Token Not available Disabled by default
Authentication with keys

When you enable key authentication for a deployment, you automatically create authentication keys.
Authentication is enabled by default when you deploy to Azure Kubernetes Service (AKS ).
Authentication is disabled by default when you deploy to Azure Container Instances.
To enable key authentication, use the auth_enabled parameter when you create or update a deployment.
If key authentication is enabled, you can use the get_keys method to retrieve a primary and secondary
authentication key:
primary, secondary = service.get_keys()

print(primary)
IMPORTANT
If you need to regenerate a key, use service.regen_key .
Authentication with tokens

When you enable token authentication for a web service, users must present an Azure Machine Learning JSON
Web Token to the web service to access it.
Token authentication is disabled by default when you deploy to Azure Kubernetes Service.
Token authentication isn't supported when you deploy to Azure Container Instances.
To control token authentication, use the token_auth_enabled parameter when you create or update a deployment.
If token authentication is enabled, you can use the get_token method to retrieve a JSON Web Token (JWT) and
that token's expiration time:
token, refresh_by = service.get_token()

print(token)
IMPORTANT
You'll need to request a new token after the token's refresh_by time.
We strongly recommend that you create your Azure Machine Learning workspace in the same region as your Azure
Kubernetes Service cluster.
To authenticate with a token, the web service will make a call to the region in which your Azure Machine Learning workspace
is created. If your workspace's region is unavailable, you won't be able to fetch a token for your web service, even if your
cluster is in a different region from your workspace. The result is that Azure AD Authentication is unavailable until your
workspace's region is available again.
Also, the greater the distance between your cluster's region and your workspace's region, the longer it will take to fetch a
token.
Authorization
You can create multiple workspaces, and each workspace can be shared by multiple people. When you share a
workspace, you can control access to it by assigning these roles to users:
Owner
Contributor
Reader
The following table lists some of the major Azure Machine Learning service operations and the roles that can
perform them:
AZURE MACHINE LEARNING
SERVICE OPERATION OWNER CONTRIBUTOR READER
Create workspace ✓ ✓
Share workspace ✓
Create compute target ✓ ✓
Attach compute target ✓ ✓
Attach data stores ✓ ✓
Run experiment ✓ ✓
View runs/metrics ✓ ✓ ✓
Register model ✓ ✓
Create image ✓ ✓
Deploy web service ✓ ✓
View models/images ✓ ✓ ✓
Call web service ✓ ✓ ✓
If the built-in roles don't meet your needs, you can create custom roles. Custom roles are supported only for
operations on the workspace and Machine Learning Compute. Custom roles can have read, write, or delete
permissions on the workspace and on the compute resource in that workspace. You can make the role available at
a specific workspace level, a specific resource-group level, or a specific subscription level. For more information,
see Manage users and roles in an Azure Machine Learning workspace.
Securing compute targets and data
Owners and contributors can use all compute targets and data stores that are attached to the workspace.
Each workspace also has an associated system-assigned managed identity that has the same name as the
workspace. The managed identity has the following permissions on attached resources used in the workspace.
For more information on managed identities, see Managed identities for Azure resources.
RESOURCE PERMISSIONS
Workspace Contributor
Storage account Storage Blob Data Contributor
Key vault Access to all keys, secrets, certificates
Azure Container Registry Contributor
Resource group that contains the workspace Contributor

RESOURCE PERMISSIONS
Resource group that contains the key vault (if different from Contributor
the one that contains the workspace)
We don't recommend that admins revoke the access of the managed identity to the resources mentioned in the
preceding table. You can restore access by using the resync keys operation.
The Azure Machine Learning service creates an additional application (the name starts with aml- ) with
contributor-level access in your subscription for every workspace region. For example, if you have one workspace
in East US and another workspace in North Europe in the same subscription, you'll see two of these applications.
These applications enable the Azure Machine Learning service to help you manage compute resources.
Network security
The Azure Machine Learning service relies on other Azure services for compute resources. Compute resources
(compute targets) are used to train and deploy models. You can create these compute targets in a virtual network.
For example, you can use Azure Data Science Virtual Machine to train a model and then deploy the model to AKS.
For more information, see How to run experiments and inference in a virtual network.
Data encryption
Encryption at rest
Azure Blob storage
The Azure Machine Learning service stores snapshots, output, and logs in the Azure Blob storage account that's
tied to the Azure Machine Learning service workspace and your subscription. All the data stored in Azure Blob
storage is encrypted at rest with Microsoft-managed keys.
For information on how to use your own keys for data stored in Azure Blob storage, see Azure Storage encryption
with customer-managed keys in Azure Key Vault.
Training data is typically also stored in Azure Blob storage so that it's accessible to training compute targets. This
storage isn't managed by Azure Machine Learning but mounted to compute targets as a remote file system.
For information on regenerating the access keys for the Azure storage accounts used with your workspace, see
Regenerate storage access keys.
Azure Cosmos DB
The Azure Machine Learning service stores metrics and metadata in the Azure Cosmos DB instance associated
with a Microsoft subscription managed by Azure Machine Learning service. All the data stored in Azure Cosmos
DB is encrypted at rest with Microsoft-managed keys.
Azure Container Registry
All container images in your registry (Azure Container Registry) are encrypted at rest. Azure automatically
encrypts an image before storing it and decrypts it on the fly when the Azure Machine Learning service pulls the
image.
Machine Learning Compute
The OS disk for each compute node stored in Azure Storage is encrypted with Microsoft-managed keys in Azure
Machine Learning service storage accounts. This compute target is ephemeral, and clusters are typically scaled
down when no runs are queued. The underlying virtual machine is de-provisioned, and the OS disk is deleted.
Azure Disk Encryption isn't supported for the OS disk.
Each virtual machine also has a local temporary disk for OS operations. If you want, you can use the disk to stage
training data. The disk isn't encrypted. For more information on how encryption at rest works in Azure, see Azure
data encryption at rest.
Encryption in transit
You can use SSL to secure internal communication between Azure Machine Learning microservices and to secure
external calls to the scoring endpoint. All Azure Storage access also occurs over a secure channel.
For more information, see Use SSL to secure a web service through Azure Machine Learning.
Using Azure Key Vault
The Azure Machine Learning service uses the Azure Key Vault instance associated with the workspace to store
credentials of various kinds:
The associated storage account connection string
Passwords to Azure Container Repository instances
Connection strings to data stores
SSH passwords and keys to compute targets like Azure HDInsight and VMs are stored in a separate key vault
that's associated with the Microsoft subscription. The Azure Machine Learning service doesn't store any passwords
or keys provided by users. Instead, it generates, authorizes, and stores its own SSH keys to connect to VMs and
HDInsight to run the experiments.
Each workspace has an associated system-assigned managed identity that has the same name as the workspace.
This managed identity has access to all keys, secrets, and certificates in the key vault.
Monitoring
Metrics
You can use Azure Monitor metrics to view and monitor metrics for your Azure Machine Learning service
workspace. In the Azure portal, select your workspace and then select Metrics:
The metrics include information on runs, deployments, and registrations.

For more information, see Metrics in Azure Monitor.
Activity log
You can view the activity log of a workspace to see various operations that are performed on the workspace. The
log includes basic information like the operation name, event initiator, and timestamp.
This screenshot shows the activity log of a workspace:
Scoring request details are stored in Application Insights. Application Insights is created in your subscription when
you create a workspace. Logged information includes fields like HTTPMethod, UserAgent, ComputeType,
RequestUrl, StatusCode, RequestId, and Duration.
IMPORTANT
Some actions in the Azure Machine Learning workspace don't log information to the activity log. For example, the start of a
training run and the registration of a model aren't logged.
Some of these actions appear in the Activities area of your workspace, but these notifications don't indicate who initiated
the activity.
Data flow diagrams

Create workspace
The following diagram shows the create workspace workflow.
The user signs in to Azure AD from one of the supported Azure Machine Learning service clients (Azure CLI,
Python SDK, Azure portal) and requests the appropriate Azure Resource Manager token.
The user calls Azure Resource Manager to create the workspace.
Azure Resource Manager contacts the Azure Machine Learning service resource provider to provision the
workspace.
Additional resources are created in the user's subscription during workspace creation:
Key Vault (to store secrets)
An Azure storage account (including blob and file share)
Azure Container Registry (to store Docker images for inference/scoring and experimentation)
Application Insights (to store telemetry)
The user can also provision other compute targets that are attached to a workspace (like Azure Kubernetes Service
or VMs) as needed.
Save source code (training scripts)

The following diagram shows the code snapshot workflow.
Associated with an Azure Machine Learning service workspace are directories (experiments) that contain the
source code (training scripts). These scripts are stored on your local machine and in the cloud (in the Azure Blob
storage for your subscription). The code snapshots are used for execution or inspection for historical auditing.
Training
The following diagram shows the training workflow.
The Azure Machine Learning service is called with the snapshot ID for the code snapshot saved in the
previous section.
The Azure Machine Learning service creates a run ID (optional) and a Machine Learning service token,
which is later used by compute targets like Machine Learning Compute/VMs to communicate with the
Machine Learning service.
You can choose either a managed compute target (like Machine Learning Compute) or an unmanaged
compute target (like VMs) to run your training jobs. Here are the data flows for both scenarios:
VMs/HDInsight, accessed by SSH credentials in a key vault in the Microsoft subscription. The Azure
Machine Learning service runs management code on the compute target that:
1. Prepares the environment. (Docker is an option for VMs and local computers. See the following steps for
Machine Learning Compute to understand how running experiments on Docker containers works.)
2. Downloads the code.
3. Sets up environment variables and configurations.
4. Runs user scripts (the code snapshot mentioned in the previous section).
Machine Learning Compute, accessed through a workspace-managed identity. Because Machine
Learning Compute is a managed compute target (that is, it's managed by Microsoft) it runs under your
Microsoft subscription.
1. Remote Docker construction is kicked off, if needed.
2. Management code is written to the user's Azure Files share.
3. The container is started with an initial command. That is, management code as described in the previous
step.
Querying runs and metrics
In the flow diagram below, this step occurs when the training compute target writes the run metrics back to the
Azure Machine Learning service from storage in the Cosmos DB database. Clients can call the Azure Machine
Learning service. Machine Learning will in turn pull metrics from the Cosmos DB database and return them back
to the client.
Creating web services

The following diagram shows the inference workflow. Inference, or model scoring, is the phase in which the
deployed model is used for prediction, most commonly on production data.
Here are the details:
The user registers a model by using a client like the Azure Machine Learning SDK.
The user creates an image by using a model, a score file, and other model dependencies.
The Docker image is created and stored in Azure Container Registry.
The web service is deployed to the compute target (Container Instances/AKS ) using the image created in the
previous step.
Scoring request details are stored in Application Insights, which is in the user’s subscription.
Telemetry is also pushed to the Microsoft/Azure subscription.
Next steps
Secure Azure Machine Learning web services with SSL
Consume a Machine Learning model deployed as a web service
How to run batch predictions
Monitor your Azure Machine Learning models with Application Insights
Collect data for models in production
Azure Machine Learning service SDK
Use the Azure Machine Learning service with Azure Virtual Network
Best practices for building recommendation systems
Build a real-time recommendation API on Azure
Manage access to an Azure Machine Learning
workspace
In this article, you learn how to manage access to an Azure Machine Learning workspace. Role-based access
control (RBAC ) is used to manage access to Azure resources. Users in your Azure Active Directory are assigned
specific roles, which grant access to resources. Azure provides both built-in roles and the ability to create custom
roles.
Default roles
An Azure Machine Learning workspace is an Azure resource. Like other Azure resources, when a new Azure
Machine Learning workspace is created, it comes with three default roles. You can add users to the workspace and
assign them to one of these built-in roles.
ROLE ACCESS LEVEL
Reader Read-only actions in the workspace. Readers can list and view
assets in a workspace, but can't create or update these assets.
Contributor View, create, edit, or delete (where applicable) assets in a

workspace. For example, contributors can create an
experiment, create or attach a compute cluster, submit a run,
and deploy a web service.
Owner Full access to the workspace, including the ability to view,

create, edit, or delete (where applicable) assets in a workspace.
Additionally, you can change role assignments.
IMPORTANT
Role access can be scoped to multiple levels in Azure. For example, someone with owner access to a workspace may not have
owner access to the resource group that contains the workspace. For more information, see How RBAC works.
For more information on specific built-in roles, see Built-in roles for Azure.
Manage workspace access

If you're an owner of a workspace, you can add and remove roles for the workspace. You can also assign roles to
users. Use the following links to discover how to manage access:
Azure portal UI
PowerShell
Azure CLI
REST API
Azure Resource Manager templates
If you have installed the Azure Machine Learning CLI, you can also use a CLI command to assign roles to users.
az ml workspace share -w <workspace_name> -g <resource_group_name> --role <role_name> --user
<user_corp_email_address>
The user field is the email address of an existing user in the instance of Azure Active Directory where the
workspace parent subscription lives. Here is an example of how to use this command:
az ml workspace share -w my_workspace -g my_resource_group --role Contributor --user jdoe@contoson.com
Create custom role

If the built-in roles are insufficient, you can create custom roles. Custom roles might have read, write, delete, and
compute resource permissions in that workspace. You can make the role available at a specific workspace level, a
specific resource group level, or a specific subscription level.
NOTE
You must be an owner of the resource at that level to create custom roles within that resource.
To create a custom role, first construct a role definition JSON file that specifies the permission and scope for the
role. The following example defines a custom role named "Data Scientist" scoped at a specific workspace level:
data_scientist_role.json :
{
"Name": "Data Scientist",
"IsCustom": true,
"Description": "Can run experiment but can't create or delete compute.",
"Actions": ["*"],
"NotActions": [
"Microsoft.MachineLearningServices/workspaces/*/delete",
"Microsoft.MachineLearningServices/workspaces/computes/*/write",
"Microsoft.MachineLearningServices/workspaces/computes/*/delete",
"Microsoft.Authorization/*/write"
],
"AssignableScopes": [
"/subscriptions/<subscription_id>/resourceGroups/<resource_group_name>/providers/Microsoft.MachineLearningServ
ices/workspaces/<workspace_name>"
]
}
You can change the AssignableScopes field to set the scope of this custom role at the subscription level, the
resource group level, or a specific workspace level.
This custom role can do everything in the workspace except for the following actions:
It can't create or update a compute resource.
It can't delete a compute resource.
It can't add, delete, or alter role assignments.
It can't delete the workspace.
To deploy this custom role, use the following Azure CLI command:
az role definition create --role-definition data_scientist_role.json

After deployment, this role becomes available in the specified workspace. Now you can add and assign this role in
the Azure portal. Or, you can assign this role to a user by using the az ml workspace share CLI command:
az ml workspace share -w my_workspace -g my_resource_group --role "Data Scientist" --user jdoe@contoson.com
For more information, see Custom roles for Azure resources.
Next steps
Enterprise security overview
Securely run experiments and inference/score inside a virtual network
Tutorial: Train models
Secure Azure ML experimentation and inference jobs
within an Azure Virtual Network
In this article, you'll learn how to secure experimentation/training jobs and inference/scoring jobs in Azure
Machine Learning within an Azure Virtual Network (vnet).
A virtual network acts as a security boundary, isolating your Azure resources from the public internet. You can
also join an Azure virtual network to your on-premises network. By joining networks, you can securely train your
models and access your deployed models for inference.
The Azure Machine Learning service relies on other Azure services for compute resources. Compute resources, or
compute targets, are used to train and deploy models. The targets can be created within a virtual network. For
example, you can use Microsoft Data Science Virtual Machine to train a model and then deploy the model to
Azure Kubernetes Service (AKS ). For more information about virtual networks, see Azure Virtual Network
overview.
This article also provides detailed information about advanced security settings, information that isn't necessary
for basic or experimental use cases. Certain sections of this article provide configuration information for a variety
of scenarios. You don't need to complete the instructions in order or in their entirety.
Prerequisites
An Azure Machine Learning service workspace.
General working knowledge of both the Azure Virtual Network service and IP networking.
A pre-existing virtual network and subnet to use with your compute resources.
Use a storage account for your workspace

To use an Azure storage account for the workspace in a virtual network, do the following:
1. Create an experimentation compute instance (for example, a Machine Learning Compute instance) behind
a virtual network, or attach an experimentation compute instance to the workspace (for example, an
HDInsight cluster or a virtual machine).
For more information, see the "Use a Machine Learning Compute instance" and "Use a virtual machine or
HDInsight cluster" sections in this article.
2. In the Azure portal, go to the storage that's attached to your workspace.
3. On the Azure Storage page, select Firewalls and virtual networks.

4. On the Firewalls and virtual networks page, do the following:
Select Selected networks.
Under Virtual networks, select the Add existing virtual network link. This action adds the virtual
network where your experimentation compute instance resides (see step 1).
Select the Allow trusted Microsoft services to access this storage account check box.
IMPORTANT
When working with the Azure Machine Learning SDK, your development environment must be able to connect to
the Azure Storage Account. When the storage account is inside a virtual network, the firewall must allow access from
the development environment's IP address.
To enable access to the storage account, visit the Firewalls and virtual networks for the storage account from a
web browser on the development client. Then use the Add your client IP address check box to add the client's IP
address to the ADDRESS RANGE. You can also use the ADDRESS RANGE field to manually enter the IP address of
the development environment. Once the IP address for the client has been added, it can access the storage account
using the SDK.
5. As you're running the experiment, in your experimentation code, change the run config to use Azure Blob
storage:
run_config.source_directory_data_store = "workspaceblobstore"
IMPORTANT
You can place the default storage account for the Azure Machine Learning service in a virtual network for experimentation
only. The default storage account is automatically provisioned when you create a workspace.
You can place non-default storage accounts in a virtual network for experimentation only. The storage_account
parameter in the Workspace.create() function allows you to specify a custom storage account by Azure resource ID.
Both the default and non-default storage accounts that are used for inference must have unrestricted access to the storage
account.
If you aren't sure whether you've modified the settings, see the "Change the default network access rule" section in
Configure Azure Storage firewalls and virtual networks. Follow the instructions to allow access from all networks during
inference, or model scoring.
Use a key vault instance with your workspace

The key vault instance that's associated with the workspace is used by the Azure Machine Learning service to
store the following credentials:
The associated storage account connection string
Passwords to Azure Container Repository instances
Connection strings to data stores
To use Azure Machine Learning experimentation capabilities with Azure Key Vault behind a virtual network, do
the following:
1. Go to the key vault that's associated with the workspace.
2. On the Key Vault page, in the left pane, select Firewalls and virtual networks.
3. On the Firewalls and virtual networks page, do the following:

Under Allow access from, select Selected networks.
Under Virtual networks, select Add existing virtual networks to add the virtual network where your
experimentation compute instance resides.
Under Allow trusted Microsoft services to bypass this firewall, select Yes.
Use a Machine Learning Compute instance
To use an Azure Machine Learning Compute instance in a virtual network, the following network requirements
must be met:
The virtual network must be in the same subscription and region as the Azure Machine Learning service
workspace.
The subnet that's specified for the compute cluster must have enough unassigned IP addresses to
accommodate the number of VMs that are targeted for the cluster. If the subnet doesn't have enough
unassigned IP addresses, the cluster will be partially allocated.
Check to see whether your security policies or locks on the virtual network's subscription or resource group
restrict permissions to manage the virtual network. If you plan to secure the virtual network by restricting
traffic, leave some ports open for the compute service. For more information, see the Required ports section.
If you're going to put multiple compute clusters in one virtual network, you might need to request a quota
increase for one or more of your resources.
The Machine Learning Compute instance automatically allocates additional networking resources in the resource
group that contains the virtual network. For each compute cluster, the service allocates the following resources:
One network security group
One public IP address
One load balancer
These resources are limited by the subscription's resource quotas.
Required ports
Machine Learning Compute currently uses the Azure Batch service to provision VMs in the specified virtual
network. The subnet must allow inbound communication from the Batch service. You use this communication to
schedule runs on the Machine Learning Compute nodes and to communicate with Azure Storage and other
resources. The Batch service adds network security groups (NSGs) at the level of network interfaces (NICs) that
are attached to VMs. These NSGs automatically configure inbound and outbound rules to allow the following
traffic:
Inbound TCP traffic on ports 29876 and 29877 from a Service Tag of BatchNodeManagement.
(Optional) Inbound TCP traffic on port 22 to permit remote access. Use this port only if you want to
connect by using SSH on the public IP.
Outbound traffic on any port to the virtual network.
Outbound traffic on any port to the internet.
Exercise caution if you modify or add inbound or outbound rules in Batch-configured NSGs. If an NSG blocks
communication to the compute nodes, the compute service sets the state of the compute nodes to unusable.
You don't need to specify NSGs at the subnet level, because the Azure Batch service configures its own NSGs.
However, if the specified subnet has associated NSGs or a firewall, configure the inbound and outbound security
rules as mentioned earlier.
The NSG rule configuration in the Azure portal is shown in the following images:
Limit outbound connectivity from the virtual network
If you don't want to use the default outbound rules and you do want to limit the outbound access of your virtual
network, do the following:
Deny outbound internet connection by using the NSG rules.
Limit outbound traffic to the following:
Azure Storage, by using Service Tag of Storage.Region_Name (for example, Storage.EastUS )
Azure Container Registry, by using Service Tag of AzureContainerRegistry.Region_Name (for
example, AzureContainerRegistry.EastUS )
Azure Machine Learning service, by using Service Tag of AzureMachineLearning
The NSG rule configuration in the Azure portal is shown in the following image:
User-defined routes for forced tunneling

If you're using forced tunneling with Machine Learning Compute, add user-defined routes (UDRs) to the subnet
that contains the compute resource.
Establish a UDR for each IP address that's used by the Azure Batch service in the region where your
resources exist. These UDRs enable the Batch service to communicate with compute nodes for task
scheduling. To get a list of IP addresses of the Batch service, use one of the following methods:
Download the Azure IP Ranges and Service Tags and search the file for
BatchNodeManagement.<region> , where <region> is your Azure region.
Use the Azure CLI to download the information. The following example downloads the IP address
information and filters out the information for the East US 2 region:
az network list-service-tags -l "East US 2" --query "values[?starts_with(id, 'Batch')] | [?

properties.region=='eastus2']"
Outbound traffic to Azure Storage must not be blocked by your on-premises network appliance.
Specifically, the URLs are in the form <account>.table.core.windows.net , <account>.queue.core.windows.net
, and <account>.blob.core.windows.net .
When you add the UDRs, define the route for each related Batch IP address prefix and set Next hop type to
Internet. The following image shows an example of this UDR in the Azure portal:
For more information, see Create an Azure Batch pool in a virtual network.
Create a Machine Learning Compute cluster in a virtual network
To create a Machine Learning Compute cluster, do the following:
1. In the Azure portal, select your Azure Machine Learning service workspace.
2. In the Application section, select Compute, and then select Add compute.
3. To configure this compute resource to use a virtual network, do the following:
a. For Network configuration, select Advanced.
b. In the Resource group drop-down list, select the resource group that contains the virtual network.
c. In the Virtual network drop-down list, select the virtual network that contains the subnet.
d. In the Subnet drop-down list, select the subnet to use.
You can also create a Machine Learning Compute cluster by using the Azure Machine Learning SDK. The
following code creates a new Machine Learning Compute cluster in the default subnet of a virtual network
named mynetwork :
from azureml.core.compute import ComputeTarget, AmlCompute
from azureml.core.compute_target import ComputeTargetException
# The Azure virtual network name, subnet, and resource group

vnet_name = 'mynetwork'
subnet_name = 'default'
vnet_resourcegroup_name = 'mygroup'
# Choose a name for your CPU cluster

cpu_cluster_name = "cpucluster"
# Verify that cluster does not exist already

try:
cpu_cluster = ComputeTarget(workspace=ws, name=cpu_cluster_name)
print("Found existing cpucluster")
except ComputeTargetException:
print("Creating new cpucluster")
# Specify the configuration for the new cluster

compute_config = AmlCompute.provisioning_configuration(vm_size="STANDARD_D2_V2",
min_nodes=0,
max_nodes=4,
vnet_resourcegroup_name=vnet_resourcegroup_name,
vnet_name=vnet_name,
subnet_name=subnet_name)
# Create the cluster with the specified name and configuration

cpu_cluster = ComputeTarget.create(ws, cpu_cluster_name, compute_config)
# Wait for the cluster to be completed, show the output log

cpu_cluster.wait_for_completion(show_output=True)
When the creation process finishes, you train your model by using the cluster in an experiment. For more
information, see Select and use a compute target for training.
Use a virtual machine or HDInsight cluster

IMPORTANT
The Azure Machine Learning service supports only virtual machines that are running Ubuntu.
To use a virtual machine or Azure HDInsight cluster in a virtual network with your workspace, do the following:
1. Create a VM or HDInsight cluster by using the Azure portal or the Azure CLI, and put the cluster in an
Azure virtual network. For more information, see the following articles:
Create and manage Azure virtual networks for Linux VMs
Extend HDInsight using an Azure virtual network
2. To allow the Azure Machine Learning service to communicate with the SSH port on the VM or cluster,
configure a source entry for the network security group. The SSH port is usually port 22. To allow traffic
from this source, do the following:
In the Source drop-down list, select Service Tag.
In the Source service tag drop-down list, select AzureMachineLearning.
In the Source port ranges drop-down list, select *.
In the Destination drop-down list, select Any.
In the Destination port ranges drop-down list, select 22.
Under Protocol, select Any.
Under Action, select Allow.
Keep the default outbound rules for the network security group. For more information, see the default
security rules in Security groups.
If you don't want to use the default outbound rules and you do want to limit the outbound access of your
virtual network, see the Limit outbound connectivity from the virtual network section.
3. Attach the VM or HDInsight cluster to your Azure Machine Learning service workspace. For more
information, see Set up compute targets for model training.
Use Azure Kubernetes Service (AKS)

To add AKS in a virtual network to your workspace, do the following:
IMPORTANT
Before you begin the following procedure, follow the prerequisites in the Configure advanced networking in Azure
Kubernetes Service (AKS) how-to and plan the IP addressing for your cluster.
The AKS instance and the Azure virtual network must be in the same region.
1. In the Azure portal, make sure that the NSG that controls the virtual network has an inbound rule that's
enabled for the Azure Machine Learning service by using AzureMachineLearning as the SOURCE.
2. Select your Azure Machine Learning service workspace.

3. In the Application section, select Compute, and then select Add compute.
4. To configure this compute resource to use a virtual network, do the following:
For Network configuration, select Advanced.
In the Resource group drop-down list, select the resource group that contains the virtual network.
In the Virtual network drop-down list, select the virtual network that contains the subnet.
In the Subnet drop-down list, select the subnet.
In the Kubernetes Service address range box, enter the Kubernetes service address range. This
address range uses a Classless Inter-Domain Routing (CIDR ) notation IP range to define the IP
addresses that are available for the cluster. It must not overlap with any subnet IP ranges (for
example, 10.0.0.0/16).
In the Kubernetes DNS service IP address box, enter the Kubernetes DNS service IP address.
This IP address is assigned to the Kubernetes DNS service. It must be within the Kubernetes service
address range (for example, 10.0.0.10).
In the Docker bridge address box, enter the Docker bridge address. This IP address is assigned to
Docker Bridge. It must not be in any subnet IP ranges, or the Kubernetes service address range (for
example, 172.17.0.1/16).
5. Make sure that the NSG group that controls the virtual network has an inbound security rule enabled for
the scoring endpoint so that it can be called from outside the virtual network.
IMPORTANT
Keep the default outbound rules for the NSG. For more information, see the default security rules in Security groups.
You can also use the Azure Machine Learning SDK to add Azure Kubernetes Service in a virtual network. If you
already have an AKS cluster in a virtual network, attach it to the workspace as described in How to deploy to AKS.
The following code creates a new AKS instance in the default subnet of a virtual network named mynetwork :
from azureml.core.compute import ComputeTarget, AksCompute
# Create the compute configuration and set virtual network information

config = AksCompute.provisioning_configuration(location="eastus2")
config.vnet_resourcegroup_name = "mygroup"
config.vnet_name = "mynetwork"
config.subnet_name = "default"
config.service_cidr = "10.0.0.0/16"
config.dns_service_ip = "10.0.0.10"
config.docker_bridge_cidr = "172.17.0.1/16"
# Create the compute target

aks_target = ComputeTarget.create(workspace=ws,
name="myaks",
provisioning_configuration=config)
When the creation process is completed, you can run inference, or model scoring, on an AKS cluster behind a
virtual network. For more information, see How to deploy to AKS.
Next steps
Set up training environments
Where to deploy models
Securely deploy models with SSL
Use SSL to secure a web service through Azure
Machine Learning
This article shows you how to secure a web service that's deployed through the Azure Machine Learning service.
You use HTTPS to restrict access to web services and secure the data that clients submit. HTTPS helps secure
communications between a client and a web service by encrypting communications between the two. Encryption
uses Transport Layer Security (TLS ). TLS is sometimes still referred to as Secure Sockets Layer (SSL ), which was
the predecessor of TLS.
TIP
The Azure Machine Learning SDK uses the term "SSL" for properties that are related to secure communications. This doesn't
mean that your web service doesn't use TLS. SSL is just a more commonly recognized term.
TLS and SSL both rely on digital certificates, which help with encryption and identity verification. For more
information on how digital certificates work, see the Wikipedia topic Public key infrastructure.
WARNING
If you don't use HTTPS for your web service, data that's sent to and from the service might be visible to others on the
internet.
HTTPS also enables the client to verify the authenticity of the server that it's connecting to. This feature protects clients
against man-in-the-middle attacks.
This is the general process to secure a web service:

1. Get a domain name.
2. Get a digital certificate.
3. Deploy or update the web service with SSL enabled.
4. Update your DNS to point to the web service.
IMPORTANT
If you're deploying to Azure Kubernetes Service (AKS), you can purchase your own certificate or use a certificate that's
provided by Microsoft. If you use a certificate from Microsoft, you don't need to get a domain name or SSL certificate. For
more information, see the Enable SSL and deploy section of this article.
There are slight differences when you secure web services across deployment targets.
Get a domain name

If you don't already own a domain name, purchase one from a domain name registrar. The process and price
differ among registrars. The registrar provides tools to manage the domain name. You use these tools to map a
fully qualified domain name (FQDN ) (such as www.contoso.com) to the IP address that hosts your web service.
Get an SSL certificate
There are many ways to get an SSL certificate (digital certificate). The most common is to purchase one from a
certificate authority (CA). Regardless of where you get the certificate, you need the following files:
A certificate. The certificate must contain the full certificate chain, and it must be "PEM -encoded."
A key. The key must also be PEM -encoded.
When you request a certificate, you must provide the FQDN of the address that you plan to use for the web
service (for example, www.contoso.com). The address that's stamped into the certificate and the address that the
clients use are compared to verify the identity of the web service. If those addresses don't match, the client gets an
error message.
TIP
If the certificate authority can't provide the certificate and key as PEM-encoded files, you can use a utility such as OpenSSL
to change the format.
WARNING
Use self-signed certificates only for development. Don't use them in production environments. Self-signed certificates can
cause problems in your client applications. For more information, see the documentation for the network libraries that your
client application uses.
Enable SSL and deploy

To deploy (or redeploy) the service with SSL enabled, set the ssl_enabled parameter to "True" wherever it's
applicable. Set the ssl_certificate parameter to the value of the certificate file. Set the ssl_key to the value of the key
file.
Deploy on AKS and field-programmable gate array (FPGA )
NOTE
The information in this section also applies when you deploy a secure web service for the visual interface. If you aren't
familiar with using the Python SDK, see What is the Azure Machine Learning SDK for Python?.
When you deploy to AKS, you can create a new AKS cluster or attach an existing one.
If you create a new cluster, you use AksCompute.provisionining_configuration().
If you attach an existing cluster, you use AksCompute.attach_configuration(). Both return a configuration
object that has an enable_ssl method.
The enable_ssl method can use a certificate that's provided by Microsoft or a certificate that you purchase.
When you use a certificate from Microsoft, you must use the leaf_domain_label parameter. This parameter
generates the DNS name for the service. For example, a value of "myservice" creates a domain name of
"myservice<six-random-characters>.<azureregion>.cloudapp.azure.com", where <azureregion> is the
region that contains the service. Optionally, you can use the overwrite_existing_domain parameter to
overwrite the existing leaf_domain_label.
To deploy (or redeploy) the service with SSL enabled, set the ssl_enabled parameter to "True" wherever it's
applicable. Set the ssl_certificate parameter to the value of the certificate file. Set the ssl_key to the value of
the key file.
IMPORTANT
When you use a certificate from Microsoft, you don't need to purchase your own certificate or domain name.
The following example demonstrates how to create a configuration that enables an SSL certificate from
Microsoft:
from azureml.core.compute import AksCompute

# Config used to create a new AKS cluster and enable SSL
provisioning_config = AksCompute.provisioning_configuration()
provisioning_config.enable_ssl(leaf_domain_label = "myservice")
# Config used to attach an existing AKS cluster to your workspace and enable SSL
attach_config = AksCompute.attach_configuration(resource_group = resource_group,
cluster_name = cluster_name)
attach_config.enable_ssl(leaf_domain_label = "myservice")
When you use a certificate that you purchased, you use the ssl_cert_pem_file, ssl_key_pem_file, and
ssl_cname parameters. The following example demonstrates how to use .pem files to create a configuration
that uses an SSL certificate that you purchased:

# Config used to create a new AKS cluster and enable SSL
provisioning_config = AksCompute.provisioning_configuration()
provisioning_config.enable_ssl(ssl_cert_pem_file="cert.pem",
ssl_key_pem_file="key.pem", ssl_cname="www.contoso.com")
# Config used to attach an existing AKS cluster to your workspace and enable SSL
attach_config.enable_ssl(ssl_cert_pem_file="cert.pem",
ssl_key_pem_file="key.pem", ssl_cname="www.contoso.com")
For more information about enable_ssl, see AksProvisioningConfiguration.enable_ssl() and

AksAttachConfiguration.enable_ssl().
Deploy on Azure Container Instances
When you deploy to Azure Container Instances, you provide values for SSL -related parameters, as the following
code snippet shows:
aci_config = AciWebservice.deploy_configuration(
ssl_enabled=True, ssl_cert_pem_file="cert.pem", ssl_key_pem_file="key.pem", ssl_cname="www.contoso.com")
For more information, see AciWebservice.deploy_configuration().
Update your DNS

Next, you must update your DNS to point to the web service.
For Container Instances:
Use the tools from your domain name registrar to update the DNS record for your domain name. The
record must point to the IP address of the service.
There can be a delay of minutes or hours before clients can resolve the domain name, depending on the
registrar and the "time to live" (TTL ) that's configured for the domain name.
For AKS:
WARNING
If you used leaf_domain_label to create the service by using a certificate from Microsoft, don't manually update the
DNS value for the cluster. The value should be set automatically.
Update the DNS of the Public IP Address of the AKS cluster on the Configuration tab under Settings in
the left pane. (See the following image.) The Public IP Address is a resource type that's created under the
resource group that contains the AKS agent nodes and other networking resources.
Update the SSL certificate

SSL certificates expire and must be renewed. Typically this happens every year. Use the information in the
following sections to update and renew your certificate for models deployed to Azure Kubernetes Service:
Update a Microsoft generated certificate
If the certificate was originally generated by Microsoft (when using the leaf_domain_label to create the service),
use one of the following examples to update the certificate:
Use the SDK

from azureml.core.compute.aks import AksUpdateConfiguration
from azureml.core.compute.aks import SslConfiguration
# Get the existing cluster

aks_target = AksCompute(ws, clustername)
# Update the existing certificate by referencing the leaf domain label

ssl_configuration = SslConfiguration(leaf_domain_label="myaks", overwrite_existing_domain=True)
update_config = AksUpdateConfiguration(ssl_configuration)
aks_target.update(update_config)
Use the CLI
az ml computetarget update aks -g "myresourcegroup" -w "myresourceworkspace" -n "myaks" --ssl-leaf-domain-

label "myaks" --ssl-overwrite-domain True
For more information, see the following reference docs:

SslConfiguration
AksUpdateConfiguration
Update custom certificate
If the certificate was originally generated by a certificate authority, use the following steps:
1. Use the documentation provided by the certificate authority to renew the certificate. This process creates
new certificate files.
2. Use either the SDK or CLI to update the service with the new certificate:
Use the SDK

# Read the certificate file

def get_content(file_name):
with open(file_name, 'r') as f:
return f.read()

# Update cluster with custom certificate

ssl_configuration = SslConfiguration(cname="myaks", cert=get_content('cert.pem'),
key=get_content('key.pem'))
Use the CLI
az ml computetarget update aks -g "myresourcegroup" -w "myresourceworkspace" -n "myaks" --ssl-cname

"myaks"--ssl-cert-file "cert.pem" --ssl-key-file "key.pem"
For more information, see the following reference docs:

SslConfiguration
AksUpdateConfiguration
Disable SSL
To disable SSL for a model deployed to Azure Kubernetes Service, create an SslConfiguration with
status="Disabled" , then perform an update:


# Disable SSL
ssl_configuration = SslConfiguration(status="Disabled")
Next steps
Learn how to:
Consume a machine learning model deployed as a web service
Securely run experiments and inference inside an Azure virtual network
Regenerate storage account access keys
Learn how to change the access keys for Azure Storage accounts used by the Azure Machine Learning service.
Azure Machine Learning can use storage accounts to store data or trained models.
For security purposes, you may need to change the access keys for an Azure Storage account. When you
regenerate the access key, Azure Machine Learning must be updated to use the new key. Azure Machine Learning
may be using the storage account for both model storage and as a datastore.
Prerequisites
An Azure Machine Learning service workspace. For more information, see the Create a workspace article.
The Azure Machine Learning SDK.
The Azure Machine Learning CLI extension.
What needs to be updated

Storage accounts can be used by the Azure Machine Learning service workspace (storing logs, models, snapshots,
etc.) and as a datastore. The process to update the workspace is a single Azure CLI command, and can be ran after
updating the storage key. The process of updating datastores is more involved, and requires discovering what
datastores are currently using the storage account and then re-registering them.
IMPORTANT
Update the workspace using the Azure CLI, and the datastores using Python, at the same time. Updating only one or the
other is not sufficient, and may cause errors until both are updated.
To discover the storage accounts that are used by your datastores, use the following code:
import azureml.core
from azureml.core import Workspace, Datastore
default_ds = ws.get_default_datastore()
print("Default datstore: " + default_ds.name + ", storage account name: " +
default_ds.account_name + ", container name: " + default_ds.container_name)
datastores = ws.datastores
for name, ds in datastores.items():
if ds.datastore_type == "AzureBlob":
print("Blob store - datastore name: " + name + ", storage account name: " +
ds.account_name + ", container name: " + ds.container_name)
if ds.datastore_type == "AzureFile":
print("File share - datastore name: " + name + ", storage account name: " +
ds.account_name + ", container name: " + ds.container_name)
This code looks for any registered datastores that use Azure Storage and lists the following information:
Datastore name: The name of the datastore that the storage account is registered under.
Storage account name: The name of the Azure Storage account.
Container: The container in the storage account that is used by this registration.
It also indicates whether the datastore is for an Azure Blob or an Azure File share, as there are different methods to
re-register each type of datastore.
If an entry exists for the storage account that you plan on regenerating access keys for, save the datastore name,
storage account name, and container name.
Update the access key

To update Azure Machine Learning service to use the new key, use the following steps:
IMPORTANT
Perform all steps, updating both the workspace using the CLI, and datastores using Python. Updating only one or the other
may cause errors until both are updated.
1. Regenerate the key. For information on regenerating an access key, see the Manage a storage account
article. Save the new key.
2. To update the workspace to use the new key, use the following steps:
a. To sign in to the Azure subscription that contains your workspace by using the following Azure CLI
command:
az login
b. To update the workspace to use the new key, use the following command. Replace myworkspace with
your Azure Machine Learning workspace name, and replace myresourcegroup with the name of the
Azure resource group that contains the workspace.
az ml workspace sync-keys -w myworkspace -g myresourcegroup
TIP
If you get an error that the ml extension is not installed, use the following command to install it:
az extension add -n azure-cli-ml
This command automatically syncs the new keys for the Azure storage account used by the
workspace.
3. To re-register datastore(s) that use the storage account, use the values from the What needs to be updated
section and the key from step 1 with the following code:
# Re-register the blob container
ds_blob = Datastore.register_azure_blob_container(workspace=ws,
datastore_name='your datastore name',
container_name='your container name',
account_name='your storage account name',
account_key='new storage account key',
overwrite=True)
# Re-register file shares
ds_file = Datastore.register_azure_file_share(workspace=ws,
file_share_name='your container name',
account_key='new storage account key',
overwrite=True)
Since overwrite=True is specified, this code overwrites the existing registration and updates it to use the
new key.
Next steps
For more information on registering datastores, see the Datastore class reference.
Create and manage Azure Machine Learning
service workspaces
In this article, you'll create, view, and delete Azure Machine Learning service workspaces in the Azure
portal for Azure Machine Learning service. The portal is the easiest way to get started with workspaces but
as your needs change or requirements for automation increase you can also create and delete workspaces
using the CLI, with Python code or via the VS Code extension.
Create a workspace
To create a workspace, you need an Azure subscription. If you don’t have an Azure subscription, create a

location.
FIELD DESCRIPTION
FIELD DESCRIPTION

others.

enter a name to create a new resource group. A
resource group holds related resources for an Azure
solution. In this example, we use docs-aml.
WARNING

Download a configuration file
1. If you will be creating a Notebook VM, skip this step.
2. If you plan to use code on your local environment that references this workspace, select Download
config.json from the Overview section of the workspace.
Place the file into the directory structure with your Python scripts or Jupyter Notebooks. It can be in
the same directory, a subdirectory named .azureml, or in a parent directory. When you create a
Notebook VM, this file is added to the correct directory on the VM for you.
View a workspace
1. In top left corner of the portal, select All services.
2. In the All services filter field, type machine learning service.
3. Select Machine Learning service workspaces.
4. Look through the list of workspaces found. You can filter based on subscription, resource groups,
and locations.
5. Select a workspace to display its properties.
Delete a workspace
Use the Delete button at the top of the workspace you wish to delete.
Clean up resources
IMPORTANT
The resources you created can be used as prerequisites to other Azure Machine Learning service tutorials and how-
to articles.

Next steps
Follow the full-length tutorial to learn how to use a workspace to build, train, and deploy models with Azure
Tutorial: Train models
Use an Azure Resource Manager template to create a
workspace for Azure Machine Learning service
In this article, you learn several ways to create an Azure Machine Learning service workspace using Azure
Resource Manager templates. A Resource Manager template makes it easy to create resources as a single,
coordinated operation. A template is a JSON document that defines the resources that are needed for a
deployment. It may also specify deployment parameters. Parameters are used to provide input values when using
the template.
For more information, see Deploy an application with Azure Resource Manager template.
Prerequisites
An Azure subscription. If you do not have one, try the free or paid version of Azure Machine Learning
service.
To use a template from a CLI, you need either Azure PowerShell or the Azure CLI.
Resource Manager template

The following Resource Manager template can be used to create an Azure Machine Learning service workspace
and associated Azure resources:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceName": {
"type": "string",
"metadata": {
"description": "Specifies the name of the Azure Machine Learning service workspace."
}
},
"location": {
"type": "string",
"defaultValue": "southcentralus",
"allowedValues": [
"eastus",
"eastus2",
"southcentralus",
"southeastasia",
"westcentralus",
"westeurope",
"westus2"
],
"metadata": {
"description": "Specifies the location for all resources."
}
}
},
"variables": {
"storageAccountName": "[concat('sa',uniqueString(resourceGroup().id))]",
"storageAccountType": "Standard_LRS",
"keyVaultName": "[concat('kv',uniqueString(resourceGroup().id))]",
"tenantId": "[subscription().tenantId]",
"applicationInsightsName": "[concat('ai',uniqueString(resourceGroup().id))]",
"applicationInsightsName": "[concat('ai',uniqueString(resourceGroup().id))]",
"containerRegistryName": "[concat('cr',uniqueString(resourceGroup().id))]"
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2018-07-01",
"name": "[variables('storageAccountName')]",
"location": "[parameters('location')]",
"sku": {
"name": "[variables('storageAccountType')]"
},
"kind": "StorageV2",
"properties": {
"encryption": {
"services": {
"blob": {
"enabled": true
},
"file": {
"enabled": true
}
},
"keySource": "Microsoft.Storage"
},
"supportsHttpsTrafficOnly": true
}
},
{
"type": "Microsoft.KeyVault/vaults",
"apiVersion": "2018-02-14",
"name": "[variables('keyVaultName')]",
"properties": {
"tenantId": "[variables('tenantId')]",
"sku": {
"name": "standard",
"family": "A"
},
"accessPolicies": []
}
},
{
"type": "Microsoft.Insights/components",
"apiVersion": "2015-05-01",
"name": "[variables('applicationInsightsName')]",
"location": "
[if(or(equals(parameters('location'),'eastus2'),equals(parameters('location'),'westcentralus')),'southcentralu
s',parameters('location'))]",
"kind": "web",
"properties": {
"Application_Type": "web"
}
},
{
"type": "Microsoft.ContainerRegistry/registries",
"apiVersion": "2017-10-01",
"name": "[variables('containerRegistryName')]",
"sku": {
"name": "Standard"
},
"properties": {
"adminUserEnabled": true
}
},
{
"type": "Microsoft.MachineLearningServices/workspaces",
"apiVersion": "2018-11-19",
"name": "[parameters('workspaceName')]",
"name": "[parameters('workspaceName')]",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', variables('storageAccountName'))]",
"[resourceId('Microsoft.KeyVault/vaults', variables('keyVaultName'))]",
"[resourceId('Microsoft.Insights/components', variables('applicationInsightsName'))]",
"[resourceId('Microsoft.ContainerRegistry/registries', variables('containerRegistryName'))]"
],
"identity": {
"type": "systemAssigned"
},
"properties": {
"friendlyName": "[parameters('workspaceName')]",
"keyVault": "[resourceId('Microsoft.KeyVault/vaults',variables('keyVaultName'))]",
"applicationInsights": "
[resourceId('Microsoft.Insights/components',variables('applicationInsightsName'))]",
"containerRegistry": "
[resourceId('Microsoft.ContainerRegistry/registries',variables('containerRegistryName'))]",
"storageAccount": "[resourceId('Microsoft.Storage/storageAccounts/',variables('storageAccountName'))]"
}
}
]
}
This template creates the following Azure services:

Azure Resource Group
Azure Storage Account
Azure Key Vault
Azure Application Insights
Azure Container Registry
Azure Machine Learning workspace
The resource group is the container that holds the services. The various services are required by the Azure
Machine Learning workspace.
The example template has two parameters:
The location where the resource group and services will be created.
The template will use the location you select for most resources. The exception is the Application Insights
service, which is not available in all of the locations that the other services are. If you select a location where
it is not available, the service will be created in the South Central US location.
The workspace name, which is the friendly name of the Azure Machine Learning workspace.
The names of the other services are generated randomly.
For more information on templates, see the following articles:
Author Azure Resource Manager templates
Deploy an application with Azure Resource Manager templates
Microsoft.MachineLearningServices resource types
Use the Azure portal

1. Follow the steps in Deploy resources from custom template. When you arrive at the Edit template screen,
paste in the template from this document.
2. Select Save to use the template. Provide the following information and agree to the listed terms and
conditions:
Subscription: Select the Azure subscription to use for these resources.
Resource group: Select or create a resource group to contain the services.
Workspace name: The name to use for the Azure Machine Learning workspace that will be created. The
workspace name must be between 3 and 33 characters. It may only contain alphanumeric characters and
'-'.
Location: Select the location where the resources will be created.
For more information, see Deploy resources from custom template.
Use Azure PowerShell

This example assumes that you have saved the template to a file named azuredeploy.json in the current directory:
New-AzResourceGroup -Name examplegroup -Location "East US"

new-azresourcegroupdeployment -name exampledeployment `
-resourcegroupname examplegroup -location "East US" `
-templatefile .\azuredeploy.json -workspaceName "exampleworkspace"
For more information, see Deploy resources with Resource Manager templates and Azure PowerShell and Deploy
private Resource Manager template with SAS token and Azure PowerShell.
Use Azure CLI

This example assumes that you have saved the template to a file named azuredeploy.json in the current directory:
az group create --name examplegroup --location "East US"

az group deployment create \
--name exampledeployment \
--resource-group examplegroup \
--template-file azuredeploy.json \
--parameters workspaceName=exampleworkspace location=eastus
For more information, see Deploy resources with Resource Manager templates and Azure CLI and Deploy private
Resource Manager template with SAS token and Azure CLI.
Azure Key Vault access policy and Azure Resource Manager templates
When you use an Azure Resource Manager template to create the workspace and associated resources (including
Azure Key Vault), multiple times. For example, using the template multiple times with the same parameters as part
of a continuous integration and deployment pipeline.
Most resource creation operations through templates are idempotent, but Key Vault clears the access policies each
time the template is used. Clearing the access policies breaks access to the Key Vault for any existing workspace
that is using it. For example, Stop/Create functionalities of Azure Notebooks VM may fail.
To avoid this problem, we recommend one of the following approaches:
Do not deploy the template more than once for the same parameters. Or delete the existing resources
before using the template to recreate them.
Examine the Key Vault access policies and then use these policies to set the accessPolicies property of the
template.
Check if the Key Vault resource already exists. If it does, do not recreate it through the template. For
example, add a parameter that allows you to disable the creation of the Key Vault resource if it already exists.
Next steps
Deploy resources with Resource Manager templates and Resource Manager REST API.
Creating and deploying Azure resource groups through Visual Studio.
Configure a development environment for Azure
Machine Learning
In this article, you learn how to configure a development environment to work with Azure Machine Learning
service. Azure Machine Learning service is platform agnostic. The only hard requirement for your development
environment is Python 3. An isolated environment like Anaconda or Virtualenv is also recommended.
The following table shows each development environment covered in this article, along with pros and cons.
ENVIRONMENT PROS CONS
Cloud-based notebook VM Easiest way to get started. The entire Lack of control over your development
SDK is already installed in your environment and dependencies.
workspace VM, and notebook tutorials Additional cost incurred for Linux VM
are pre-cloned and ready to run. (VM can be stopped when not in use to
avoid charges). See pricing details.
Local environment Full control of your development Takes longer to get started. Necessary
environment and dependencies. Run SDK packages must be installed, and an
with any build tool, environment, or IDE environment must also be installed if
of your choice. you don't already have one.
Azure Databricks Ideal for running large-scale intensive Overkill for experimental machine
machine learning workflows on the learning, or smaller-scale experiments
scalable Apache Spark platform. and workflows. Additional cost incurred
for Azure Databricks. See pricing details.
The Data Science Virtual Machine Similar to the cloud-based notebook A slower getting started experience
(DSVM) VM (Python and the SDK are pre- compared to the cloud-based notebook
installed), but with additional popular VM.
data science and machine learning tools
pre-installed. Easy to scale and combine
with other custom tools and workflows.
Azure Notebooks Free and lightweight getting started Less-powerful VMs available compared
experience, with Python and the SDK to cloud-based notebook VM. Isolated
pre-installed. from workspace and other resources.
This article also provides additional usage tips for the following tools:
Jupyter Notebooks: If you're already using the Jupyter Notebook, the SDK has some extras that you should
install.
Visual Studio Code: If you use Visual Studio Code, it has some useful extensions that you can install.
Prerequisites
An Azure Machine Learning service workspace. To create the workspace, see Create an Azure Machine Learning
service workspace. A workspace is all you need to get started with your own cloud-based notebook server, a
DSVM, Azure Databricks, or Azure Notebooks.
To install the SDK environment for your local computer, Jupyter Notebook server or Visual Studio Code you also
need:
Either the Anaconda or Miniconda package manager.
On Linux or macOS, you need the bash shell.
TIP
If you're on Linux or macOS and use a shell other than bash (for example, zsh) you might receive errors when you
run some commands. To work around this problem, use the bash command to start a new bash shell and run the
commands there.
On Windows, you need the command prompt or Anaconda prompt (installed by Anaconda and
Miniconda).
Your own cloud-based notebook VM

The notebook virtual machine (Preview ) is a secure, cloud-based Azure workstation that provides data scientists
with a Jupyter notebook server, JupyterLab, and a fully prepared ML environment.
The notebook VM is:
Secure. Since VM and notebook access is secured with HTTPS and Azure Active Directory by default, IT
Pros can easily enforce single sign-on and other security features such as multi-factor authentication.
Preconfigured. This fully prepared Python ML environment draws its pedigree from the popular IaaS
Data Science VM and includes:
Azure ML Python SDK (latest)
Automatic configuration to work with your workspace
A Jupyter notebook server
JupyterLab notebook IDE
Preconfigured GPU drivers
A selection of deep learning frameworks
If you are into code, the VM includes tutorials and samples to help you explore and learn how to use Azure
Machine Learning service. The sample notebooks are stored in the Azure Blob Storage account of your
workspace making them shareable across VMs. When run, they also have access to the data stores and
compute resources of your workspace.
Simple setup: Create one anytime from within your Azure Machine Learning workspace. Provide just a
name and specify an Azure VM type. Try it now with this Tutorial: Setup environment and workspace.
Customizable. While a managed and secure VM offering, you retain full access to the hardware
capabilities and customize it to your heart’s desire. For example, quickly create the latest NVidia V100
powered VM to perform step-by-step debugging of novel Neural Network architecture.
To stop incurring notebook VM charges, stop the notebook VM.
Data Science Virtual Machine

The DSVM is a customized virtual machine (VM ) image. It's designed for data science work that's pre-configured
with:
Packages such as TensorFlow, PyTorch, Scikit-learn, XGBoost, and the Azure Machine Learning SDK
Popular data science tools such as Spark Standalone and Drill
Azure tools such as the Azure CLI, AzCopy, and Storage Explorer
Integrated development environments (IDEs) such as Visual Studio Code and PyCharm
Jupyter Notebook Server
The Azure Machine Learning SDK works on either the Ubuntu or Windows version of the DSVM. But if you plan
to use the DSVM as a compute target as well, only Ubuntu is supported.
To use the DSVM as a development environment, do the following:
1. Create a DSVM in either of the following environments:
The Azure portal:
Create an Ubuntu Data Science Virtual Machine
Create a Windows Data Science Virtual Machine
The Azure CLI:
IMPORTANT
When you use the Azure CLI, you must first sign in to your Azure subscription by using the
az login command.
When you use the commands in this step, you must provide a resource group name, a name for the
VM, a username, and a password.
To create an Ubuntu Data Science Virtual Machine, use the following command:
# create a Ubuntu DSVM in your resource group

# note you need to be at least a contributor to the resource group in order to execute
this command successfully
# If you need to create a new resource group use: "az group create --name YOUR-RESOURCE-
GROUP-NAME --location YOUR-REGION (For example: westus2)"
az vm create --resource-group YOUR-RESOURCE-GROUP-NAME --name YOUR-VM-NAME --image
microsoft-dsvm:linux-data-science-vm-ubuntu:linuxdsvmubuntu:latest --admin-username YOUR-
USERNAME --admin-password YOUR-PASSWORD --generate-ssh-keys --authentication-type password
To create a Windows Data Science Virtual Machine, use the following command:
# create a Windows Server 2016 DSVM in your resource group

# note you need to be at least a contributor to the resource group in order to execute
this command successfully
az vm create --resource-group YOUR-RESOURCE-GROUP-NAME --name YOUR-VM-NAME --image
microsoft-dsvm:dsvm-windows:server-2016:latest --admin-username YOUR-USERNAME --admin-
password YOUR-PASSWORD --authentication-type password
2. The Azure Machine Learning SDK is already installed on the DSVM. To use the Conda environment that
contains the SDK, use one of the following commands:
For Ubuntu DSVM:
conda activate py36
For Windows DSVM:
conda activate AzureML

3. To verify that you can access the SDK and check the version, use the following Python code:
import azureml.core
print(azureml.core.VERSION)
4. To configure the DSVM to use your Azure Machine Learning service workspace, see the Create a
workspace configuration file section.
For more information, see Data Science Virtual Machines.
Local computer
When you're using a local computer (which might also be a remote virtual machine), create an Anaconda
environment and install the SDK by doing the following:
1. Download and install Anaconda (Python 3.7 version) if you don't already have it.
2. Open an Anaconda prompt and create an environment with the following commands:
Run the following command to create the environment.
conda create -n myenv python=3.6.5
Then activate the environment.
conda activate myenv
This example creates an environment using python 3.6.5, but any specific subversions can be chosen. SDK
compatibility may not be guaranteed with certain major versions (3.5+ is recommended), and it's
recommended to try a different version/subversion in your Anaconda environment if you run into errors. It
will take several minutes to create the environment while components and packages are downloaded.
3. Run the following commands in your new environment to enable environment-specific ipython kernels.
This will ensure expected kernel and package import behavior when working with Jupyter Notebooks
within Anaconda environments:
conda install notebook ipykernel
Then run the following command to create the kernel:
ipython kernel install --user
4. Use the following commands to install packages:

This command installs the base Azure Machine Learning SDK with notebook and automl extras. The
automl extra is a large install, and can be removed from the brackets if you don't intend to run automated
machine learning experiments. The automl extra also includes the Azure Machine Learning Data Prep SDK
by default as a dependency.
pip install azureml-sdk[notebooks,automl]

NOTE
If you get a message that PyYAML can't be uninstalled, use the following command instead:
pip install --upgrade azureml-sdk[notebooks,automl] --ignore-installed PyYAML
Starting with macOS Catalina, zsh (Z shell) is the default login shell and interactive shell. In zsh, use the
following command which escapes brackets with "\" (backslash):
pip install --upgrade azureml-sdk\[notebooks,automl\]
It will take several minutes to install the SDK. For more information on installation options, see the install
guide.
5. Install other packages for your machine learning experimentation.
Use either of the following commands and replace <new package> with the package you want to install.
Installing packages via conda install requires that the package is part of the current channels (new
channels can be added in Anaconda Cloud).
conda install <new package>
Alternatively, you can install packages via pip .
pip install <new package>
Jupyter Notebooks
Jupyter Notebooks are part of the Jupyter Project. They provide an interactive coding experience where you
create documents that mix live code with narrative text and graphics. Jupyter Notebooks are also a great way to
share your results with others, because you can save the output of your code sections in the document. You can
install Jupyter Notebooks on a variety of platforms.
The procedure in the Local computer section installs necessary components for running Jupyter Notebooks in an
Anaconda environment. To enable these components in your Jupyter Notebook environment, do the following:
1. Open an Anaconda prompt and activate your environment.
conda activate myenv
2. Clone the GitHub repository for a set of sample notebooks.
3. Launch the Jupyter Notebook server with the following command:
jupyter notebook
4. To verify that Jupyter Notebook can use the SDK, create a New notebook, select Python 3 as your kernel,
and then run the following command in a notebook cell:
import azureml.core
azureml.core.VERSION
5. If you encounter issues importing modules and receive a ModuleNotFoundError , ensure your Jupyter kernel
is connected to the correct path for your environment by running the following code in a Notebook cell.
import sys
sys.path
6. To configure the Jupyter Notebook to use your Azure Machine Learning service workspace, go to the
Create a workspace configuration file section.
Visual Studio Code
Visual Studio Code is a cross platform code editor. It relies on a local Python 3 and Conda installation for Python
support, but it provides additional tools for working with AI. It also provides support for selecting the Conda
environment from within the code editor.
To use Visual Studio Code for development, do the following:
1. To learn how to use Visual Studio Code for Python development, see Get started with Python in VSCode.
2. To select the Conda environment, open VS Code, and then select Ctrl+Shift+P (Linux and Windows) or
Command+Shift+P (Mac). The Command Pallet opens.
3. Enter Python: Select Interpreter, and then select the Conda environment.
4. To validate that you can use the SDK, create and then run a new Python file (.py) that contains the following
code:
import azureml.core
azureml.core.VERSION
5. To install the Azure Machine Learning extension for Visual Studio Code, see Tools for AI.
For more information, see Use Azure Machine Learning for Visual Studio Code.
Azure Databricks
Azure Databricks is an Apache Spark-based environment in the Azure cloud. It provides a collaborative
Notebook-based environment with CPU or GPU -based compute cluster.
How Azure Databricks works with Azure Machine Learning service:
You can train a model using Spark MLlib and deploy the model to ACI/AKS from within Azure Databricks.
You can also use automated machine learning capabilities in a special Azure ML SDK with Azure Databricks.
You can use Azure Databricks as a compute target from an Azure Machine Learning pipeline.
Set up your Databricks cluster
Create a Databricks cluster. Some settings apply only if you install the SDK for automated machine learning on
Databricks. It will take few minutes to create the cluster.
Use these settings:
SETTING APPLIES TO VALUE
Cluster name always yourclustername
Databricks Runtime always Any non ML runtime (non ML 4.x, 5.x)

SETTING APPLIES TO VALUE
Python version always 3
Workers always 2 or higher
Worker node VM types Automated ML Memory optimized VM preferred

(determines max # of concurrent only
iterations)
Enable Autoscaling Automated ML Uncheck

only
Wait until the cluster is running before proceeding further.

Install the correct SDK into a Databricks library
Once the cluster is running, create a library to attach the appropriate Azure Machine Learning SDK package to
your cluster.
1. Choose only one option (no other SDK installation are supported)
SDK PACKAGE EX TRAS SOURCE PYPI NAME
For Databricks Upload Python Egg or PyPI azureml-sdk[databricks]
For Databricks -with- Upload Python Egg or PyPI azureml-sdk[automl_databricks]

automated ML capabilities
WARNING
No other SDK extras can be installed. Choose only one of the preceding options [databricks] or [automl_databricks].
Do not select Attach automatically to all clusters.

Select Attach next to your cluster name.
2. Monitor for errors until status changes to Attached, which may take several minutes. If this step fails,
check the following:
Try restarting your cluster by:
a. In the left pane, select Clusters.
b. In the table, select your cluster name.
c. On the Libraries tab, select Restart.
Also consider:
In AutoML config, when using Azure Databricks add the following parameters:
a. max_concurrent_iterations is based on number of worker nodes in your cluster.
b. spark_context=sc is based on the default spark context.
Or, if you have an old SDK version, deselect it from cluster’s installed libs and move to trash. Install the
new SDK version and restart the cluster. If there is an issue after the restart, detach and reattach your
cluster.
If install was successful, the imported library should look like one of these:
SDK for Databricks without automated machine learning
SDK for Databricks WITH automated machine learning
Start exploring
Try it out:
Download the notebook archive file for Azure Databricks/Azure Machine Learning SDK and import the
archive file into your Databricks cluster. While many sample notebooks are available, only these sample
notebooks work with Azure Databricks.
Learn how to create a pipeline with Databricks as the training compute.
Azure Notebooks
Azure Notebooks (preview ) is an interactive development environment in the Azure cloud. It's an easy way to get
started with Azure Machine Learning development.
The Azure Machine Learning SDK is already installed.
After you create an Azure Machine Learning service workspace in the Azure portal, you can click a button to
automatically configure your Azure Notebook environment to work with the workspace.
Use the Azure portal to get started with Azure Notebooks. Open your workspace and from the Overview section,
select Get Started in Azure Notebooks.
By default, Azure Notebooks uses a free service tier that is limited to 4 GB of memory and 1 GB of data. You can,
however, remove these limits by attaching a Data Science Virtual Machine instance to the Azure Notebooks
project. For more information, see Manage and configure Azure Notebooks projects - Compute tier.
Create a workspace configuration file

The workspace configuration file is a JSON file that tells the SDK how to communicate with your Azure Machine
Learning service workspace. The file is named config.json, and it has the following format:
{
"subscription_id": "<subscription-id>",
"resource_group": "<resource-group>",
"workspace_name": "<workspace-name>"
}
This JSON file must be in the directory structure that contains your Python scripts or Jupyter Notebooks. It can
be in the same directory, a subdirectory named .azureml, or in a parent directory.
To use this file from your code, use ws=Workspace.from_config() . This code loads the information from the file and
connects to your workspace.
You can create the configuration file in three ways:
Use ws.write_config: to write a config.json file. The file contains the configuration information for your
workspace. You can download or copy the config.json to other development environments.
Download the file: In the Azure portal, select Download config.json from the Overview section of
your workspace.
Create the file programmatically: In the following code snippet, you connect to a workspace by
providing the subscription ID, resource group, and workspace name. It then saves the workspace
configuration to the file:
subscription_id = '<subscription-id>'
resource_group = '<resource-group>'
workspace_name = '<workspace-name>'
try:
ws = Workspace(subscription_id = subscription_id, resource_group = resource_group, workspace_name =
workspace_name)
ws.write_config()
print('Library configuration succeeded')
except:
print('Workspace not found')
This code writes the configuration file to the .azureml/config.json file.
Next steps
Train a model on Azure Machine Learning with the MNIST dataset
View the Azure Machine Learning SDK for Python reference
Learn about the data prep package for Azure Machine Learning
Create and manage environments for training and
deployment
In this article, learn how to create and manage Azure Machine Learning environments so you can track and
reproduce your projects' software dependencies as they evolve.
Software dependency management is a common task for developers. You want to be able to ensure that builds are
reproducible without a lot of manual software configuration. With solutions for local development, such as pip and
Conda in mind, the Azure Machine Learning service Environments class provides a solution for both local and
distributed cloud development.
The examples in this article show how to:
Create an environment and specify package dependencies
Retrieve and update environments
Use environment for training
Use environment for web service deployment
What are environments

Environments specify the Python packages, environment variables, and software settings around your training and
scoring scripts, and run times (Python, Spark, or Docker). They are managed and versioned entities within your
Azure Machine Learning workspace that enable reproducible, auditable, and portable machine learning workflows
across different compute targets.
You can use an environment object on your local compute to develop your training script, reuse that same
environment on Azure Machine Learning Compute for model training at scale, and even deploy your model with
that same environment.
The following illustrates that the same environment object can be used in both your run configuration for training
and in your inferencing and deployment configuration for web service deployments.
Types of environments
Environments can broadly be divided into two categories: user-managed and system -managed.
For a user-managed environment, you're responsible for setting up your environment and installing every
package your training script needs on the compute target. Conda will not check your environment or install
anything for you.
System-managed environments are used when you want Conda to manage the Python environment and the
script dependencies for you. The service assumes this type of environment by default, due to its usefulness on
remote compute targets that are not manually configurable.
Prerequisites
The Azure Machine Learning SDK for Python installed.
Create an environment
There are multiple ways to create an environment for your experiments.
Instantiate an environment object
To manually create an environment import the Environment class from the SDK and instantiate an environment
object with the following code.
from azureml.core import Environment

Environment(name="myenv")
Conda and pip specification files

You can also create an environment from a Conda specification or a pip requirements file. Use the
from_conda_specification() or the from_pip_requirements() method, and include your environment name and the
file path of the desired file in the method argument.
# From a Conda specification file

myenv = Environment.from_conda_specification(name = "myenv",
file_path = "path-to-conda-specification-file")
#From a pip requirements file

myenv = Environment.from_pip_requirements(name = "myenv"
file_path = "path-to-pip-requirements-file")
Existing Conda environment

If you have an existing Conda environment on your local computer, the service offers a solution for creating an
environment object from it. This way you can reuse your local interactive environment on remote runs.
The following creates an environment object out of the existing Conda environment mycondaenv with the
from_existing_conda_environment() method.
myenv = Environment.from_existing_conda_environment(name = "myenv",

conda_environment_name = "mycondaenv")
Automatically create environments

Automatically create an environment by submitting a training run with the submit() method. When you submit a
training run, the building of new environment can take several minutes depending on the size of the required
dependencies.
If you don't specify an environment in your run configuration prior to submitting the run, a default environment is
created for you.
from azureml.core import ScriptRunConfig, Experiment, Environment
# Create experiment
myexp = Experiment(workspace=ws, name = "environment-example")
# Attaches training script and compute target to run config

runconfig = ScriptRunConfig(source_directory=".", script="example.py")
runconfig.run_config.target = "local"
# Submit the run

run = myexp.submit(config=runconfig)
# Shows each step of run

run.wait_for_completion(show_output=True)
Similarly, if you use an Estimator object for training you can submit the estimator instance directly as a run
without having to specify an environment, this is because the Estimator object already encapsulates the
environment and compute target.
Add packages to an environment

Add packages to an environment with Conda, pip, or private wheel files. Specify each package dependency using
the CondaDependency class, and add it to the environment's PythonSection.
Conda and pip packages
If a package is available in a Conda package repository, it is recommended to use the Conda over pip installation.
The reason is that Conda packages typically come with pre-built binaries that make installation more reliable.
The following example adds scikit-learn , specifically version 0.21.3, and pillow package to the environment,
myenv with the add_conda_package() and add_pip_package() methods, respectively.

from azureml.core.environment import CondaDependencies
myenv = Environment(name="myenv")
conda_dep = CondaDependencies()
# Installs scikit-learn version 0.21.3 conda package

conda_dep.add_conda_package("scikit-learn==0.21.3")
# Adds dependencies to PythonSection of myenv

myenv.python.conda_dependencies=conda_dep
Private wheel files

You can use private pip wheel files by first uploading it to your workspace storage by using static
add_private_pip_wheel() method, then capturing the storage URL, and passing the URL to the add_pip_package()
method
# During environment creation the service replaces the URL by secure SAS URL, so your wheel file is kept
private and secure
whl_url = Environment.add_private_pip_wheel(workspace=ws,file_path = "my-custom.whl")
conda_dep.add_pip_package(whl_url)
Manage environments
Manage environments so you can update, track, and reuse them across compute targets and with other users of
the workspace.
Register environments
The environment is automatically registered with your workspace when you submit a run or deploy a web service.
You can also manually register the environment using the register() method. This operation makes the
environment into an entity that is tracked and versioned in the cloud, and can be shared between workspace users.
The following registers the environment, myenv , to the workspace, ws .
myenv.register(workspace=ws)
When used for the first time, in training or deployment, the environment is registered with the workspace, built,
and deployed on the compute target. The environments are cached by the service. Reusing a cached environment
takes much less time than using a new service or one that has been updated.
Get existing environments
The Environment class offers methods that allow you to retrieve existing environments in your workspace by
name, as a list or by specific training run. For troubleshooting or auditing purposes, reproducibility
View list of environments
View the environments in your workspace with list(), and then select one to reuse.

list("workspace_name")
Get environment by name

You can also get a specific environment by name and version. The following code uses the get() method to retrieve
version 1 of the environment, myenv on the ws workspace.
restored_environment = Environment.get(workspace=ws,name="myenv",version="1")
Training run specific environment

To get the environment used for a specific run after training completes, use the get_environment()method in the
Run class.

Run.get_environment()
Update an existing environment

If you make changes to an existing environment, such as add a Python package, a new version of an environment
is created when you either submit run, deploy model or manually register the environment. The versioning allows
you to view changes to the environment over time.
To update a Python package version of an existing environment, specify the exact version number for that package.
Otherwise, the Azure Machine Learning will reuse the existing environment with package versions from when the
environment was created.
Debug the image build
This example uses the build() method to create an environment manually as a Docker image, and monitors the
output logs from the image build using wait_for_completion(). The built image then appears under the workspace
Azure Container Registry, which is helpful for debugging.
from azureml.core import Image
build = env.build()
build.wait_for_completion(show_output=True)
Docker and environments

The DockerSection of the Azure Machine Learning Environments class, allows you to customize and control in
detail the guest operating system in which your training run executes.
When you enable Docker, the service builds a Docker image and creates a Python environment with your
specifications within that Docker container. This provides additional isolation and reproducibility for your training
runs.
# Creates the environment inside a Docker container.

myenv.docker.enabled = True
Once built, the Docker image appears in the Azure Container Registry that's associated with the workspace, by
default. The repository name has the form azureml/azureml_<uuid>. The unique identifier (uuuid) part
corresponds to a hash computed from the environment configuration. This allows the service to determine
whether an image corresponding to the given environment already exists for reuse.
Additionally, the service automatically uses one of the Ubuntu Linux-based base images, and installs the specified
Python packages. The base image has CPU and GPU versions, and you can specify the GPU image by setting
gpu_support=True .
# Specify custom Docker base image and registry, if you don't want to use the defaults
myenv.docker.base_image="your_base-image"
myenv.docker.base_image_registry="your_registry_location"
# Specify GPU image

myenv.docker.gpu_support=True
NOTE
If you specify environment.python.user_managed_dependencies=False while using a custom Docker image, the service will
build a Conda environment within the image and execute the run in that environment, instead of using Python libraries you
may have installed on the base image. Set the parameter to True to use your own installed packages.
Using environments for training

To submit a training run, you need to combine your environment, compute target and training Python script into a
run configuration; a wrapper object used for submitting runs.
When you submit a training run, the building of a new environment can take several minutes depending on the
size of the required dependencies. The environments are cached by the service, therefore as long as the
environment definition remains unchanged, the full setup time is incurred only once.
The following is a local script run example where you would use ScriptRunConfig as your wrapper object.
from azureml.core import Environment, ScriptRunConfig, Experiment
exp = Experiment(name="myexp", workspace = ws)

# Instantiate environment
# Add training script to run config

runconfig = ScriptRunConfig(source_directory=".", script="train.py")
# Attach compute target to run config

runconfig.compute_target = "local"
# Attach environment to run config

runconfig.environment = myenv
# Submit run
run = exp.submit(runconfig)
NOTE
To disable run history or run snapshots, use the setting under ScriptRunConfig.run_config.history .
If you don't specify the environment in your run configuration, the service will create a default environment for
you when you submit your run.
Train with an estimator
If you are using an estimator for training, you can simply submit the estimator instance directly, as it already
encapsulates the environment and compute target.
The following uses an estimator for a single-node training run on a remote compute for a scikit-learn model, and
assumes a previously created compute target object, compute_target and datastore object, ds .
from azureml.train.estimator import Estimator
script_params = {
'--data-folder': ds.as_mount(),
}
sk_est = Estimator(source_directory='./my-sklearn-proj',
entry_script='train.py',
conda_packages=['scikit-learn'])
# Submit the run

run = experiment.submit(sk_est)
Using environments for web service deployment

You can use environments when deploying your model as a web service. This enables a reproducible, connected
workflow where you can train, test, and deploy your model using the exact same libraries in both your training and
inferencing computes.
To deploy a web service, combine the environment, inference compute, scoring script, and registered model in
your deployment object, deploy(). Learn more about deploying web services.
In this example, assume you have completed a training run want to deploy that model to an Azure Container
Instance (ACI). When building the web service, the model and scoring files are mounted on the image and the
Azure Machine Learning inferencing stack is added to the image.
from azureml.core.model import InferenceConfig, Model

from azureml.core.webservice import AciWebservice, Webservice
# Register the model to deploy

model = run.register_model(model_name = "mymodel", model_path = "outputs/model.pkl")
# Combine scoring script & environment in Inference configuration

inference_config = InferenceConfig(entry_script="score.py", environment=myenv)
# Set deployment configuration

deployment_config = AciWebservice.deploy_configuration(cpu_cores = 1, memory_gb = 1)
# Define the model, inference & deployment configuration and web service name and location to deploy
service = Model.deploy(
workspace = ws,
name = "my_web_service",
models = [model],
inference_config = inference_config,
deployment_config = deployment_config)
Example notebooks
This example notebook expands upon concepts and methods demonstrated in this article.
Next steps
Tutorial: Train a model uses a managed compute target to train a model.
Once you have a trained model, learn how and where to deploy models.
View the Environment class SDK reference.
Enable logging in Azure Machine Learning service
The Azure Machine Learning Python SDK allows you to enable logging using both the default Python logging
package, as well as using SDK-specific functionality both for local logging and logging to your workspace in the
portal. Logs provide developers with real-time information about the application state, and can help with
diagnosing errors or warnings. In this article, you learn different ways of enabling logging in the following areas:
Training models and compute targets
Image creation
Deployed models
Python logging settings
Create an Azure Machine Learning service workspace. Use the guide for more information the SDK.
Training models and compute target logging

There are multiple ways to enable logging during the model training process, and the examples shown will
illustrate common design patterns. You can easily log run-related data to your workspace in the cloud by using the
start_logging function on the Experiment class.
exp = Experiment(workspace=ws, name='test_experiment')

run = exp.start_logging()
run.log("test-val", 10)
See the reference documentation for the Run class for additional logging functions.
To enable local logging of application state during training progress, use the show_output parameter. Enabling
verbose logging allows you to see details from the training process as well as information about any remote
resources or compute targets. Use the following code to enable logging upon experiment submission.
experiment = Experiment(ws, experiment_name)

run = experiment.submit(config=run_config_object, show_output=True)
You can also use the same parameter in the wait_for_completion function on the resulting run.
The SDK also supports using the default python logging package in certain scenarios for training. The following
example enables a logging level of INFO in an AutoMLConfig object.
import logging
automated_ml_config = AutoMLConfig(task='regression',
verbosity=logging.INFO,
X=your_training_features,
y=your_training_labels,
iterations=30,
iteration_timeout_minutes=5,
primary_metric="spearman_correlation")
You can also use the show_output parameter when creating a persistent compute target. Specify the parameter in
the wait_for_completion function to enable logging during compute target creation.
compute_target = ComputeTarget.attach(
workspace=ws, name="example", attach_configuration=config)
compute.wait_for_completion(show_output=True)
Logging during image creation

Enabling logging during image creation will allow you to see any errors during the build process. Set the
show_output param on the wait_for_deployment() function.
service = Webservice.deploy_from_image(deployment_config=your_config,
image=image,
name="example-image",
workspace=ws
)
Logging for deployed models

To retrieve logs from a previously deployed web service, load the service and use the get_logs() function. The
logs may contain detailed information about any errors that occurred during deployment.
# load existing web service

service = Webservice(name="service-name", workspace=ws)
logs = service.get_logs()
You can also log custom stack traces for your web service by enabling Application Insights, which allows you to
monitor request/response times, failure rates, and exceptions. Call the update() function on an existing web
service to enable Application Insights.
service.update(enable_app_insights=True)
See the how -to for more information on how to work with Application Insights in the Azure portal.
Python native logging settings
Certain logs in the SDK may contain an error that instructs you to set the logging level to DEBUG. To set the
logging level, add the following code to your script.
import logging
logging.basicConfig(level=logging.DEBUG)
Where to save and write files for Azure Machine
Learning experiments
In this article, you learn where to save input files, and where to write output files from your experiments to prevent
storage limit errors and experiment latency.
When launching training runs on a compute target, they are isolated from outside environments. The purpose of
this design is to ensure reproducibility and portability of the experiment. If you run the same script twice, on the
same or another compute target, you receive the same results. With this design, you can treat compute targets as
stateless computation resources, each having no affinity to the jobs that are running after they are finished.
Where to save input files

Before you can initiate an experiment on a compute target or your local machine, you must ensure that the
necessary files are available to that compute target, such as dependency files and data files your code needs to run.
Azure Machine Learning runs training scripts by copying the entire script folder to the target compute context, and
then takes a snapshot. The storage limit for experiment snapshots is 300 MB and/or 2000 files.
For this reason, we recommend:
Storing your files in an Azure Machine Learning datastore. This prevents experiment latency issues,
and has the advantages of accessing data from a remote compute target, which means authentication and
mounting are managed by Azure Machine Learning service. Learn more about specifying a datastore as
your source directory, and uploading files to your datastore in the Access data from your datastores article.
If you only need a couple data files and dependency scripts and can't use a datastore, place the files
in the same folder directory as your training script. Specify this folder as your source_directory directly in
your training script, or in the code that calls your training script.
Storage limits of experiment snapshots
For experiments, Azure Machine Learning automatically makes an experiment snapshot of your code based on the
directory you suggest when you configure the run. This has a total limit of 300 MB and/or 2000 files. If you exceed
this limit, you'll see the following error:
While attempting to take snapshot of .

Your total snapshot size exceeds the limit of 300.0 MB
To resolve this error, store your experiment files on a datastore. If you can't use a datastore, the below table offers
possible alternate solutions.
EXPERIMENT DESCRIPTION STORAGE LIMIT SOLUTION
Less than 2000 files & can't use a datastore Override snapshot size limit with
azureml._restclient.snapshots_client.SNAPSHOT_MAX_SIZE_BYTES
= 'insert_desired_size'
This may take several minutes depending on the number and
size of files.
EXPERIMENT DESCRIPTION STORAGE LIMIT SOLUTION
Must use specific script directory Make a .amlignore file to exclude files from your experiment
snapshot that are not part of the source code. Add the
filenames to the .amlignore file and place it in the same
directory as your training script. The .amlignore file uses the
same syntax and patterns as a .gitignore file.
Pipeline Use a different subdirectory for each step
Jupyter notebooks Create a .amlignore file or move your notebook into a new,
empty, subdirectory and run your code again.
Where to write files

Due to the isolation of training experiments, the changes to files that happen during runs are not necessarily
persisted outside of your environment. If your script modifies the files local to compute, the changes are not
persisted for your next experiment run, and they're not propagated back to the client machine automatically.
Therefore, the changes made during the first experiment run don't and shouldn't affect those in the second.
When writing changes, we recommend writing files to an Azure Machine Learning datastore. See Access data from
your datastores.
If you don't require a datastore, write files to the ./outputs and/or ./logs folder.
IMPORTANT
Two folders, outputs and logs, receive special treatment by Azure Machine Learning. During training, when you write files to
./outputs and ./logs folders, the files will automatically upload to your run history, so that you have access to them once
your run is finished.
For output such as status messages or scoring results, write files to the ./outputs folder, so they are
persisted as artifacts in run history. Be mindful of the number and size of files written to this folder, as
latency may occur when the contents are uploaded to run history. If latency is a concern, writing files to a
datastore is recommended.
To save written file as logs in run history, write files to ./logs folder. The logs are uploaded in real time,
so this method is suitable for streaming live updates from a remote run.
Next steps
Learn more about accessing data from your datastores.
Learn more about How to Set Up Training Targets.
Get started with Azure Machine Learning for Visual
Studio Code
In this article, you'll learn how to use the extension for Azure Machine Learning for Visual Studio Code to train
and deploy machine learning and deep learning models.
The Azure Machine Learning service provides support for experiments that you run locally and on remote
compute targets. For every experiment, you can keep track of multiple runs as often you need to iteratively try
different techniques, hyperparameters, and more. You can use Azure Machine Learning to track custom metrics
and experiment runs, enabling data science reproducibility and auditability.
You can also deploy these models for your testing and production needs.
Prerequisites
If you don’t have an Azure subscription, create a free account before you begin. Try the free or paid version
of the Azure Machine Learning service.
Visual Studio Code must be installed. Visual Studio Code is a lightweight but powerful source code editor
that runs on your desktop. It comes with built-in support for Python and other programming languages. If
you haven't already installed Visual Studio Code, find out how.
Install Python 3.5 or later.
Install the extension for Azure Machine Learning for Visual Studio
Code
When you install the Azure Machine Learning extension, two more extensions are automatically installed (if you
have internet access). They're the Azure Account extension and the Microsoft Python extension.
To work with Azure Machine Learning, you need to turn Visual Studio Code into a Python-integrated
development environment (IDE ). You need the Microsoft Python extension to use Python in Visual Studio Code.
This extension is automatically installed with the Azure Machine Learning extension. The extension makes Visual
Studio Code an excellent IDE, and it works on any operating system with a variety of Python interpreters. The
Microsoft Python extension uses all of the power of Visual Studio Code to provide autocomplete, IntelliSense,
linting, debugging, and unit testing. The extension also allows you to easily switch between Python environments,
including virtual and conda environments. For more information about editing, running, and debugging Python
code, see the Python hello-world tutorial.
To install the Azure Machine Learning extension:
1. Open Visual Studio Code.
2. In a web browser, go to Azure Machine Learning for Visual Studio Code extension (preview ).
3. On that web page, select Install.
4. On the extension tab, select Install.
5. A welcome tab for the extension opens in Visual Studio Code, and the Azure symbol (outlined in red in the
following screenshot) is added to the activity bar.
6. In the dialog box, select Sign In and follow the prompts to authenticate with Azure.
The Azure Account extension, which was installed along with the Azure Machine Learning for Visual Studio
Code extension, helps you authenticate with your Azure account. For a list of commands, see the page for
the Azure Account extension.
TIP
Check out the IntelliCode extension for Visual Studio Code (preview). IntelliCode provides a set of AI-assisted capabilities for
IntelliSense in Python, such as inferring the most relevant autocompletions based on the current code context.
Install the Azure Machine Learning SDK

1. Make sure that Python 3.5 or later is installed and is recognized by Visual Studio Code. If you install it now,
restart Visual Studio Code and select a Python interpreter.
2. In the integrated terminal window, specify the Python interpreter to use. Or select Enter to use your default
Python interpreter.
3. In the lower-right corner of the window, a notification appears, indicating that the Azure Machine Learning
SDK is being automatically installed. The newly created Python environment is local and private, and it has
the Visual Studio Code prerequisites for working with the Azure Machine Learning service.
Get started with Azure Machine Learning
Before you start training and deploying machine learning models in Visual Studio Code, you need to create an
Azure Machine Learning service workspace in the cloud. This workspace will contain your models and resources.
To create a workspace and add your first experiment:
1. On the Visual Studio Code activity bar, select the Azure icon. The Azure Machine Learning sidebar appears.
2. Right-click your Azure subscription and select Create Workspace. A list appears. In the example animated
image, the subscription name is Free Trial, and the workspace is TeamWorkspace.
3. Select a resource group from the list or create a new one by using the wizard in the command palette.
4. In the field, type a unique and clear name for your new workspace. In the example image, the workspace is
named TeamWorkspace.
5. Select Enter to create the new workspace. It appears in the tree, below the subscription name.
6. Right-click the Experiment node, and choose Create Experiment from the context menu. Experiments
keep track of your runs using Azure Machine Learning.
7. In the field, enter a name for your experiment. In the example screenshots, the experiment is named
MNIST.
8. Select Enter to create the new experiment. The experiment appears in the tree, below the workspace name.
9. In a workspace, you can right-click an experiment to set it as the Active experiment. The Active
experiment is your current experiment. Your open folder in Visual Studio Code will be linked to this
experiment in the cloud. This folder should contain your local Python scripts.
Now your key metrics will be stored within the experiment history. Similarly, the models you train will be
automatically uploaded to Azure Machine Learning and stored alongside your experiment metrics and logs.
Create and manage compute targets
With Azure Machine Learning for Visual Studio Code, you can prepare your data, train models, and deploy them
both locally and on remote compute targets.
The extension supports several remote compute targets for Azure Machine Learning. For more information, see
the full list of supported compute targets for Azure Machine Learning.
Create compute targets for Azure Machine Learning in Visual Studio Code
To create a compute target:
2. In the tree view, expand your Azure subscription and Azure Machine Learning service workspace. In the
following example image, the subscription name is Free Trial, and the workspace is TeamWorkspace.
3. Under the workspace node, right-click the Compute node and choose Create Compute.
4. Choose the compute target type from the list.
5. On the command palette, select a virtual machine size.
6. On the command palette, in the field, enter a name for the compute target.
7. In the JSON config file that opens on a new tab, specify any advanced properties. You can specify
properties such as a maximum node count.
8. When you finish configuring your compute target, in the lower-right corner of the window, select Submit.
Here's an example of how to create an Azure Machine Learning compute (AMLCompute):
The run configuration file
The Visual Studio Code extension automatically creates a local compute target and run configurations for your
local and docker environments on your local computer. You can find the run configuration files under the
associated compute target node.
Train and tune models

Use Azure Machine Learning for Visual Studio Code (preview ) to rapidly iterate on your code, step through and
debug, and use your solution for source code control.
To run your experiment locally by using Azure Machine Learning:
2. In the tree view, expand your Azure subscription and Azure Machine Learning service workspace.
3. Under the workspace node, expand the Compute node and right-click the Run Config of the compute
you want to use.
4. Select Run Experiment.
5. From the File Explorer, select the script you want to run.
6. Select View Experiment Run to see the integrated Azure Machine Learning portal to monitor your runs
and see your trained models.
Here's an example of how to run an experiment locally:
Use remote computes for experiments in Visual Studio Code
To use a remote compute target for training, you need to create a run configuration file. This file tells Azure
Machine Learning not only where to run your experiment but also how to prepare the environment.
The conda dependencies file
By default, a new conda environment is created for you, and your installation dependencies are managed.
However, you must specify your dependencies and their versions in the aml_config/conda_dependencies.yml file.
The following snippet from the default aml_config/conda_dependencies.yml specifies tensorflow=1.12.0 . If you
don't specify the version of the dependency, the latest version will be used. You can add additional dependencies
in the config file.
# The dependencies defined in this file will be automatically provisioned for runs with
userManagedDependencies=False.
name: project_environment
dependencies:
# The python interpreter version.
# Currently Azure ML only supports 3.5.2 and later.
- python=3.6.2
- tensorflow=1.12.0
- pip:
# Required packages for AzureML execution, history, and data preparation.
- --index-url https://azuremlsdktestpypi.azureedge.net/sdk-release/Preview/E7501C02541B433786111FE8E140CAA1
- --extra-index-url https://pypi.python.org/simple
- azureml-defaults
To run your experiment with Azure Machine Learning on a remote compute target:
3. In the editor window, right-click your Python script, and select AML: Run as Experiment in Azure.
4. On the command palette, select the compute target.
5. On the command palette, in the field, enter the run configuration name.
6. Edit the conda_dependencies.yml file to specify the experiment's runtime dependencies. Then in the lower-
right corner of the window, select Submit.
7. Select View Experiment Run to see the integrated Azure Machine Learning portal to monitor your runs
and see your trained models.
Here's an example of how to run an experiment on a remote compute target:
Deploy and manage models

In Azure Machine Learning, you can deploy and manage your machine learning models in the cloud and at the
edge.
Register your model to Azure Machine Learning from Visual Studio Code
Now that you've trained your model, you can register it in your workspace. You can track and deploy registered
models.
To register your model:
3. Under the workspace node, right-click Models and choose Register Model.
4. On the command palette, in the field, enter a model name.
5. From the list, choose whether to upload a model file (for single models) or a model folder (for models
with multiple files, such as TensorFlow ).
6. Select your folder or file.
7. When you finish configuring your model properties, in the lower-right corner of the window, select
Submit.
Here's an example of how to register your model to Azure Machine Learning:
Deploy your service from Visual Studio Code

In Visual Studio Code, you can deploy your web service to:
Azure Container Instances (ACI) for testing.
Azure Kubernetes Service (AKS ) for production.
You don't need to create an ACI container to test in advance, because ACI containers are created on the fly.
However, you do need to configure AKS clusters in advance. For more information, see Deploy models with the
Azure Machine Learning service.
To deploy a web service:
2. In the tree view, expand your Azure subscription and your Azure Machine Learning service workspace.
3. Under the workspace node, expand the Models node.
4. Right-click the model you want to deploy, and choose Deploy Service from Registered Model from the
context menu.
5. On the command palette, choose the compute target you want to deploy to.
6. On the command palette, in the field, enter a name for this service.
7. On the command palette, select the Enter key on your keyboard to browse for and select the script file.
8. On the command palette, select the Enter key on your keyboard to browse for and select the conda
dependency file.
9. When you finish configuring your service properties, in the lower-right corner of the window, select
Submit to deploy. In the service properties file, you can specify a local docker file or a schema.json file.
The web service is now deployed.
Here's an example of how to deploy a web service:
Use keyboard shortcuts

You can use the keyboard to access Azure Machine Learning features in Visual Studio Code. The most important
keyboard shortcut to know is Ctrl+Shift+P, which displays the command palette. From the command palette, you
have access to all of the functionality of Visual Studio Code, including keyboard shortcuts for the most common
operations.
Next steps
For a walkthrough of how to train with Azure Machine Learning outside of Visual Studio Code, see Tutorial:
Train models with Azure Machine Learning.
For a walkthrough of how to edit, run, and debug code locally, see the Python hello-world tutorial.
Access data in Azure storage services
In this article, learn how to easily access your data in Azure storage services via Azure Machine Learning
datastores. Datastores are used to store connection information, like your subscription ID and token
authorization, to access your storage without having to hard code that information in your scripts.
This how -to shows examples of the following tasks:
Register datastores
Get datastores from workspace
Upload and download data using datastores
Access data during training
Prerequisites
To use datastores, you first need a workspace.
Start by either creating a new workspace or retrieving an existing one:
import azureml.core
Register datastores
All the register methods are on the Datastore class and have the form register_azure_*.
The following examples show you to register an Azure Blob Container or an Azure File Share as a datastore.
For an Azure Blob Container Datastore, use register_azure_blob-container()
datastore = Datastore.register_azure_blob_container(workspace=ws,
container_name='your azure blob container name',
account_key='your storage account key',
create_if_not_exists=True)
For an Azure File Share Datastore, use register_azure_file_share() . For example:
datastore = Datastore.register_azure_file_share(workspace=ws,
file_share_name='your file share name',
account_key='your storage account key',
create_if_not_exists=True)
Storage guidance
We recommend Azure Blob Container. Both standard and premium storage are available for blobs. Although
more expensive, we suggest premium storage due to faster throughput speeds that may improve the speed of
your training runs, particularly if you train against a large data set. See the Azure pricing calculator for storage
account cost information.
Get datastores from your workspace

To get a specific datastore registered in the current workspace, use the get() static method on Datastore class:
#get named datastore from current workspace

datastore = Datastore.get(ws, datastore_name='your datastore name')
To get the list of datastores registered with a given workspace, you can use the datastores property on a
workspace object:
#list all datastores registered in current workspace

datastores = ws.datastores
for name, datastore in datastores.items():
print(name, datastore.datastore_type)
When you create a workspace, an Azure Blob Container and an Azure File Share are registered to the workspace
named workspaceblobstore and workspacefilestore respectively. They store the connection information of the
Blob Container and the File Share that is provisioned in the storage account attached to the workspace. The
workspaceblobstore is set as the default datastore.
To get the workspace's default datastore:
datastore = ws.get_default_datastore()
To define a different default datastore for the current workspace, use set_default_datastore() method on the
workspace object:
#define default datastore for current workspace

ws.set_default_datastore('your datastore name')
Upload & download data

The upload() and download() methods described in the following examples are specific to and operate
identically for the AzureBlobDatastore and AzureFileDatastore classes.
Upload
Upload either a directory or individual files to the datastore using the Python SDK.
To upload a directory to a datastore datastore :
import azureml.data
from azureml.data.azure_storage_datastore import AzureFileDatastore, AzureBlobDatastore
datastore.upload(src_dir='your source directory',

target_path='your target path',
overwrite=True,
show_progress=True)
The target_path parameter specifies the location in the file share (or blob container) to upload. It defaults to
None , in which case the data gets uploaded to root. When overwrite=True any existing data at target_path is
overwritten.
Or upload a list of individual files to the datastore via the upload_files() method.
Download
Similarly, download data from a datastore to your local file system.
datastore.download(target_path='your target path',

prefix='your prefix',
show_progress=True)
The target_path parameter is the location of the local directory to download the data to. To specify a path to the
folder in the file share (or blob container) to download, provide that path to prefix . If prefix is None , all the
contents of your file share (or blob container) will get downloaded.
Access your data during training

To access data during training, you can either download or mount your data from your Azure storage services to
the compute target via datastores.
The following table lists the methods that tell the compute target how to use the datastores during runs.
WAY METHOD DESCRIPTION
Mount as_mount() Use to mount the datastore on the

compute target.
Download as_download() Use to download the contents of your

datastore to the location specified by
path_on_compute .
This download happens before the run.
Upload as_upload() Use to upload a file from the location

specified by path_on_compute to your
datastore.
This upload happens after your run.
To reference a specific folder or file in your datastore and make it available on the compute target, use the
datastore path() method.
#to mount the full contents in your storage to the compute target
datastore.as_mount()
#to download the contents of the `./bar` directory in your storage to the compute target
datastore.path('./bar').as_download()
NOTE
Any datastore or datastore.path object resolves to an environment variable name of the format
"$AZUREML_DATAREFERENCE_XXXX" , whose value represents the mount/download path on the target compute. The
datastore path on the target compute might not be the same as the execution path for the training script.
Examples
The following code examples are specific to the Estimator class for accessing data during training.
script_params is a dictionary containing parameters to the entry_script. You can use it to pass in a datastore and
describe how data should be made available on compute target. Learn more from our end-to-end tutorial.
script_params = {
'--data_dir': datastore.path('/bar').as_mount()
}
est = Estimator(source_directory='your code directory',

compute_target=compute_target
)
You can also pass in a list of datastores to the Estimator constructor inputs parameter to mount or copy data
to/from your datastore(s). This code example:
Downloads all the contents in datastore1 to the compute target before your training script train.py is run
Downloads the folder './foo' in datastore2 to the compute target before train.py is run
Uploads the file './bar.pkl' from the compute target to the datastore3 after your script has run
est = Estimator(source_directory='your code directory',

inputs=[datastore1.as_download(), datastore2.path('./foo').as_download(),
datastore3.as_upload(path_on_compute='./bar.pkl')])
Compute and datastore matrix

Datastores currently support storing connection information to the storage services listed in the following matrix.
This matrix displays the available data access functionalities for the different compute targets and datastore
scenarios. Learn more about the compute targets for Azure Machine Learning.
AZUREDATALAKEGEN2
DATASTORE
AZUREPOSTGRESQLDA
TASTORE
AZUREBLOBDATASTOR AZUREDATALAKEDATA AZURESQLDATABASED
COMPUTE E AZUREFILEDATASTORE STORE ATASTORE
Local as_download(), as_download(), N/A N/A

as_upload() as_upload()
Azure Machine as_mount(), as_mount(), N/A N/A

Learning Compute as_download(), as_download(),
as_upload(), as_upload(),
ML pipelines ML pipelines
Virtual machines as_download(), as_download() N/A N/A

HDInsight as_download() as_download() N/A N/A

Data transfer ML pipelines N/A ML pipelines ML pipelines
Databricks ML pipelines N/A ML pipelines N/A

AZUREDATALAKEGEN2
DATASTORE
AZUREPOSTGRESQLDA
TASTORE
AZUREBLOBDATASTOR AZUREDATALAKEDATA AZURESQLDATABASED
COMPUTE E AZUREFILEDATASTORE STORE ATASTORE
Azure Batch ML pipelines N/A N/A N/A
Azure DataLake N/A N/A ML pipelines N/A

Analytics
NOTE
There may be scenarios in which highly iterative, large data processes run faster using as_download() instead of
as_mount() ; this can be validated experimentally.
Accessing source code during training

Azure blob storage has higher throughput speeds than Azure file share and will scale to large numbers of jobs
started in parallel. For this reason, we recommend configuring your runs to use blob storage for transferring
source code files.
The following code example specifies in the run configuration which blob datastore to use for source code
transfers.
# workspaceblobstore is the default blob storage

run_config.source_directory_data_store = "workspaceblobstore"
Access data during scoring

The Azure Machine Learning service provides several ways to use your models for scoring. Some of these
methods do not provide access to datastores. Use the following table to understand which methods allow you to
access datastores during scoring:
METHOD DATASTORE ACCESS DESCRIPTION
Batch prediction ✔ Make predictions on large quantities of

data asynchronously.
Web service Deploy model(s) as a web service.
IoT Edge module Deploy model(s) to IoT Edge devices.
For situations where the SDK does not provide access to datastores, you may be able to create custom code
using the relevant Azure SDK to access the data. For example, using the Azure Storage SDK for Python to access
data stored in blobs.
Next steps
Train a model
Deploy a model
Create and access datasets (Preview) in Azure
Machine Learning
In this article, you'll learn how to create Azure Machine Learning datasets (preview ), and how to access data from
local or remote experiments.
With Azure Machine Learning datasets, you can:
Keep a single copy of data in your storage referenced by datasets.
Easily access data during model training without worrying about connection strings or data paths.
Share data & collaborate with other users.
Prerequisites
To create and work with datasets, you need:
An Azure subscription. If you don’t have an Azure subscription, create a free account before you begin. Try
the free or paid version of Azure Machine Learning service today.
An Azure Machine Learning service workspace
The Azure Machine Learning SDK for Python installed, which includes the azureml-datasets package.
NOTE
Some Dataset classes (preview) have dependencies on the azureml-dataprep package. For Linux users, these classes are
supported only on the following distributions: Red Hat Enterprise Linux, Ubuntu, Fedora, and CentOS.
Dataset Types
Datasets are categorized into various types based on how users consume them in training. Currently we support
TabularDatasets which represent data in a tabular format by parsing the provided file or list of files. This provides
you with the ability to materialize the data into a pandas DataFrame. A TabularDataset object can be created from
csv, tsv, parquet files, SQL query results etc. For a complete list, please visit our documentation.
To find out more about upcoming API changes, see What is Azure Machine Learning service?
Create datasets
By creating a dataset, you create a reference to the data source location, along with a copy of its metadata. The
data remains in its existing location, so no extra storage cost is incurred.
For the data to be accessible by Azure Machine Learning service, datasets must be created from paths in Azure
datastores or public web urls.
To create Datasets from an Azure datastore:
Verify you have contributor or owner access to the registered Azure datastore.
Create the dataset by referencing to a path in the datastore.
from azureml.core.datastore import Datastore
from azureml.core.dataset import Dataset
datastore_name = 'your datastore name'
# get existing workspace

workspace = Workspace.from_config()
# retrieve an existing datastore in the workspace by name

datastore = Datastore.get(workspace, datastore_name)
Create TabularDatasets
Use the from_delimited_files() method on TabularDatasetFactory class to read files in csv or tsv format, and
create an unregistered TabularDataset. If you are reading from multiple files, results will be aggregated into one
tabular representation.
# create a TabularDataset from multiple paths in datastore

datastore_paths = [
(datastore, 'weather/2018/11.csv'),
(datastore, 'weather/2018/12.csv'),
(datastore, 'weather/2019/*.csv')
]
weather_ds = Dataset.Tabular.from_delimited_files(path=datastore_paths)
# create a TabularDataset from a delimited file behind a public web url

web_path ='https://dprepdata.blob.core.windows.net/demo/Titanic.csv'
titanic_ds = Dataset.Tabular.from_delimited_files(path=web_path)
# preview the first 3 rows of titanic_ds

titanic_ds.take(3).to_pandas_dataframe()
PASS
ENGE SURV PCLAS PARC TICKE CABI EMBA
RID IVED S NAME SEX AGE SIBSP H T FARE N RKED
0 1 0 3 Brau male 22.0 1 0 A/5 7.250 S

nd, 2117 0
Mr. 1
Owe
n
Harri
s
1 2 1 1 Cumi femal 38.0 1 0 PC 71.28 C85 C

ngs, e 1759 33
Mrs. 9
John
Bradl
ey
(Flore
nce
Brigg
s Th...
2 3 1 3 Heikk femal 26.0 0 0 STON 7.925 S

inen, e /O2. 0
Miss. 3101
Laina 282
Register datasets
To complete the creation process, register your datasets with workspace:
Use the register() method to register datasets to your workspace so they can be shared with others and reused
across various experiments.
titanic_ds = titanic_ds.register(workspace = workspace,

name = 'titanic_ds',
description = 'titanic training data')
Version datasets
You can register a new dataset under the same name by creating a new version. Dataset version is a way to
bookmark the state of your data, so you can apply a specific version of the dataset for experimentation or future
reproduction. Typical scenarios to consider versioning:
When new data is available for retraining.
When you are applying different data preparation or feature engineering approaches.
# create a TabularDataset from new Titanic training data

web_paths = [
'https://dprepdata.blob.core.windows.net/demo/Titanic.csv',
'https://dprepdata.blob.core.windows.net/demo/Titanic2.csv'
]
titanic_ds = Dataset.Tabular.from_delimited_files(path=web_paths)
# create a new version of titanic_ds

titanic_ds = titanic_ds.register(workspace = workspace,
name = 'titanic_ds',
description = 'new titanic training data',
create_new_version = True)
Access your data during training

Registered datasets are accessible locally and remotely on compute clusters like the Azure Machine Learning
compute. To access your registered Dataset across experiments, use the following code to get your workspace and
registered dataset by name. The get_by_name() method on the Dataset class by default returns the latest version
of the dataset registered with the workspace.
%%writefile $script_folder/train.py
from azureml.core import Dataset, Run
workspace = run.experiment.workspace
dataset_name = 'titanic_ds'
# Get a dataset by name

titanic_ds = Dataset.get_by_name(workspace=workspace, name=dataset_name)
# Load a TabularDataset into pandas DataFrame

df = titanic_ds.to_pandas_dataframe()
Next steps
Use automated machine learning to train with TabularDatasets.
For more examples of training with datasets, see the sample notebooks.
Set up and use compute targets for model training
With Azure Machine Learning service, you can train your model on a variety of resources or environments,
collectively referred to as compute targets. A compute target can be a local machine or a cloud resource, such
as an Azure Machine Learning Compute, Azure HDInsight or a remote virtual machine. You can also create
compute targets for model deployment as described in "Where and how to deploy your models".
You can create and manage a compute target using the Azure Machine Learning SDK, Azure portal, Azure CLI
or Azure Machine Learning VS Code extension. If you have compute targets that were created through another
service (for example, an HDInsight cluster), you can use them by attaching them to your Azure Machine
Learning service workspace.
In this article, you learn how to use various compute targets for model training. The steps for all compute
targets follow the same workflow:
1. Create a compute target if you don’t already have one.
2. Attach the compute target to your workspace.
3. Configure the compute target so that it contains the Python environment and package dependencies
needed by your script.
NOTE
Compute targets for training

Azure Machine Learning service has varying support across different compute targets. A typical model
development lifecycle starts with dev/experimentation on a small amount of data. At this stage, we recommend
using a local environment. For example, your local computer or a cloud-based VM. As you scale up your
training on larger data sets, or do distributed training, we recommend using Azure Machine Learning Compute
to create a single- or multi-node cluster that autoscales each time you submit a run. You can also attach your
own compute resource, although support for various scenarios may vary as detailed below:
Compute targets can be reused from one training job to the next. For example, once you attach a remote
VM to your workspace, you can reuse it for multiple jobs.
Local computer maybe yes
Azure Machine yes yes & yes yes

Learning Compute hyperparameter tuni
ng
Remote VM yes yes & yes

hyperparameter
tuning
Azure Databricks yes yes

Azure Data Lake yes

Analytics
Azure HDInsight yes
Azure Batch yes
NOTE
Azure Machine Learning Compute can be created as a persistent resource or created dynamically when you request a
run. Run-based creation removes the compute target after the training run is complete, so you cannot reuse compute
targets created this way.
What's a run configuration?

When training, it is common to start on your local computer, and later run that training script on a different
compute target. With Azure Machine Learning service, you can run your script on various compute targets
without having to change your script.
All you need to do is define the environment for each compute target within a run configuration. Then, when
you want to run your training experiment on a different compute target, specify the run configuration for that
compute. For details of specifying an environment and binding it to run configuration, see Create and manage
environments for training and deployment
Learn more about submitting experiments at the end of this article.
What's an estimator?
To facilitate model training using popular frameworks, the Azure Machine Learning Python SDK provides an
alternative higher-level abstraction, the estimator class. This class allows you to easily construct run
configurations. You can create and use a generic Estimator to submit training scripts that use any learning
framework you choose (such as scikit-learn).
For PyTorch, TensorFlow, and Chainer tasks, Azure Machine Learning also provides respective PyTorch,
TensorFlow, and Chainer estimators to simplify using these frameworks.
For more information, see Train ML Models with estimators.
What's an ML Pipeline?
With ML pipelines, you can optimize your workflow with simplicity, speed, portability, and reuse. When building
pipelines with Azure Machine Learning, you can focus on your expertise, machine learning, rather than on
infrastructure and automation.
ML pipelines are constructed from multiple steps, which are distinct computational units in the pipeline. Each
step can run independently and use isolated compute resources. This allows multiple data scientists to work on
the same pipeline at the same time without over-taxing compute resources, and also makes it easy to use
different compute types/sizes for each step.
TIP
ML Pipelines can use run configuration or estimators when training models.
While ML pipelines can train models, they can also prepare data before training and deploy models after
training. One of the primary use cases for pipelines is batch scoring. For more information, see Pipelines:
Optimize machine learning workflows.
Set up in Python
Use the sections below to configure these compute targets:
Local computer
Azure Machine Learning Compute
Remote virtual machines
Azure HDInsight
Local computer
1. Create and attach: There's no need to create or attach a compute target to use your local computer as
the training environment.
2. Configure: When you use your local computer as a compute target, the training code is run in your
development environment. If that environment already has the Python packages you need, use the user-
managed environment.
from azureml.core.runconfig import RunConfiguration
# Edit a run configuration property on the fly.

run_local = RunConfiguration()
run_local.environment.python.user_managed_dependencies = True
Now that you’ve attached the compute and configured your run, the next step is to submit the training run.
Azure Machine Learning Compute is a managed-compute infrastructure that allows the user to easily create a
single or multi-node compute. The compute is created within your workspace region as a resource that can be
shared with other users in your workspace. The compute scales up automatically when a job is submitted, and
can be put in an Azure Virtual Network. The compute executes in a containerized environment and packages
your model dependencies in a Docker container.
You can use Azure Machine Learning Compute to distribute the training process across a cluster of CPU or
GPU compute nodes in the cloud. For more information on the VM sizes that include GPUs, see GPU -
optimized virtual machine sizes.
Azure Machine Learning Compute has default limits, such as the number of cores that can be allocated. For
more information, see Manage and request quotas for Azure resources.
You can create an Azure Machine Learning compute environment on demand when you schedule a run, or as a
persistent resource.
Run-based creation
You can create Azure Machine Learning Compute as a compute target at run time. The compute is
automatically created for your run. The compute is deleted automatically once the run completes.
NOTE
To specify the max number of nodes to use, you would normally set node_count to the number of nodes. There is
currently (04/04/2019) a bug that prevents this from working. As a workaround, use the
amlcompute._cluster_max_node_count property of the run configuration. For example,
run_config.amlcompute._cluster_max_node_count = 5 .
IMPORTANT
Run-based creation of Azure Machine Learning compute is currently in Preview. Don't use run-based creation if you use
automated hyperparameter tuning or automated machine learning. To use hyperparameter tuning or automated
machine learning, create a persistent compute target instead.
1. Create, attach, and configure: The run-based creation performs all the necessary steps to create, attach,
and configure the compute target with the run configuration.
# First, list the supported VM families for Azure Machine Learning Compute
print(AmlCompute.supported_vmsizes(workspace=ws))

# Create a new runconfig object
run_temp_compute = RunConfiguration()
# Signal that you want to use AmlCompute to execute the script

run_temp_compute.target = "amlcompute"
# AmlCompute is created in the same region as your workspace

# Set the VM size for AmlCompute from the list of supported_vmsizes
run_temp_compute.amlcompute.vm_size = 'STANDARD_D2_V2'
Persistent compute
A persistent Azure Machine Learning Compute can be reused across jobs. The compute can be shared with
other users in the workspace and is kept between jobs.
1. Create and attach: To create a persistent Azure Machine Learning Compute resource in Python, specify
the vm_size and max_nodes properties. Azure Machine Learning then uses smart defaults for the other
properties. The compute autoscales down to zero nodes when it isn't used. Dedicated VMs are created to
run your jobs as needed.
vm_size: The VM family of the nodes created by Azure Machine Learning Compute.
max_nodes: The max number of nodes to autoscale up to when you run a job on Azure Machine
Learning Compute.
# Choose a name for your CPU cluster

cpu_cluster_name = "cpucluster"
# Verify that cluster does not exist already

try:
cpu_cluster = ComputeTarget(workspace=ws, name=cpu_cluster_name)
print('Found existing cluster, use it.')
compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_D2_V2',
max_nodes=4)
cpu_cluster = ComputeTarget.create(ws, cpu_cluster_name, compute_config)
cpu_cluster.wait_for_completion(show_output=True)
You can also configure several advanced properties when you create Azure Machine Learning Compute.
The properties allow you to create a persistent cluster of fixed size, or within an existing Azure Virtual
Network in your subscription. See the AmlCompute class for details.
Or you can create and attach a persistent Azure Machine Learning Compute resource in the Azure
portal.
2. Configure: Create a run configuration for the persistent compute target.

from azureml.core.runconfig import DEFAULT_CPU_IMAGE
# Create a new runconfig object

run_amlcompute = RunConfiguration()
# Use the cpu_cluster you created above.

run_amlcompute.target = cpu_cluster
# Enable Docker
run_amlcompute.environment.docker.enabled = True
# Set Docker base image to the default CPU-based image

run_amlcompute.environment.docker.base_image = DEFAULT_CPU_IMAGE
# Use conda_dependencies.yml to create a conda environment in the Docker image for execution
run_amlcompute.environment.python.user_managed_dependencies = False
# Auto-prepare the Docker image when used for execution (if it is not already prepared)
run_amlcompute.auto_prepare_environment = True
# Specify CondaDependencies obj, add necessary packages

run_amlcompute.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=
['scikit-learn'])
Remote virtual machines
Azure Machine Learning also supports bringing your own compute resource and attaching it to your
workspace. One such resource type is an arbitrary remote VM, as long as it's accessible from Azure Machine
Learning service. The resource can be an Azure VM, a remote server in your organization, or on-premises.
Specifically, given the IP address and credentials (user name and password, or SSH key), you can use any
accessible VM for remote runs.
You can use a system-built conda environment, an already existing Python environment, or a Docker container.
To execute on a Docker container, you must have a Docker Engine running on the VM. This functionality is
especially useful when you want a more flexible, cloud-based dev/experimentation environment than your local
machine.
Use the Azure Data Science Virtual Machine (DSVM ) as the Azure VM of choice for this scenario. This VM is a
pre-configured data science and AI development environment in Azure. The VM offers a curated choice of tools
and frameworks for full-lifecycle machine learning development. For more information on how to use the
DSVM with Azure Machine Learning, see Configure a development environment.
1. Create: Create a DSVM before using it to train your model. To create this resource, see Provision the
Data Science Virtual Machine for Linux (Ubuntu).
WARNING
Azure Machine Learning only supports virtual machines that run Ubuntu. When you create a VM or choose an
existing VM, you must select a VM that uses Ubuntu.
2. Attach: To attach an existing virtual machine as a compute target, you must provide the fully qualified
domain name (FQDN ), user name, and password for the virtual machine. In the example, replace
<fqdn> with the public FQDN of the VM, or the public IP address. Replace <username> and
<password> with the SSH user name and password for the VM.
from azureml.core.compute import RemoteCompute, ComputeTarget
# Create the compute config

compute_target_name = "attach-dsvm"
attach_config = RemoteCompute.attach_configuration(address = "<fqdn>",
ssh_port=22,
username='<username>',
password="<password>")
# If you authenticate with SSH keys instead, use this code:

# ssh_port=22,
# username='<username>',
# password=None,
# private_key_file="<path-to-file>",
# private_key_passphrase="<passphrase>")
# Attach the compute

compute = ComputeTarget.attach(ws, compute_target_name, attach_config)
compute.wait_for_completion(show_output=True)
Or you can attach the DSVM to your workspace using the Azure portal.
3. Configure: Create a run configuration for the DSVM compute target. Docker and conda are used to
create and configure the training environment on the DSVM.
import azureml.core
run_dsvm = RunConfiguration(framework = "python")
# Set the compute target to the Linux DSVM

run_dsvm.target = compute_target_name
# Use Docker in the remote VM

run_dsvm.environment.docker.enabled = True
# Use the CPU base image

# To use GPU in DSVM, you must also use the GPU base Docker image
"azureml.core.runconfig.DEFAULT_GPU_IMAGE"
run_dsvm.environment.docker.base_image = azureml.core.runconfig.DEFAULT_CPU_IMAGE
print('Base Docker image is:', run_dsvm.environment.docker.base_image)
# Specify the CondaDependencies object

run_dsvm.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-
learn'])
Azure HDInsight
Azure HDInsight is a popular platform for big-data analytics. The platform provides Apache Spark, which can
be used to train your model.
1. Create: Create the HDInsight cluster before you use it to train your model. To create a Spark on
HDInsight cluster, see Create a Spark Cluster in HDInsight.
When you create the cluster, you must specify an SSH user name and password. Take note of these
values, as you need them to use HDInsight as a compute target.
After the cluster is created, connect to it with the hostname <clustername>-ssh.azurehdinsight.net,
where <clustername> is the name that you provided for the cluster.
2. Attach: To attach an HDInsight cluster as a compute target, you must provide the hostname, user name,
and password for the HDInsight cluster. The following example uses the SDK to attach a cluster to your
workspace. In the example, replace <clustername> with the name of your cluster. Replace <username>
and <password> with the SSH user name and password for the cluster.
from azureml.core.compute import ComputeTarget, HDInsightCompute

from azureml.exceptions import ComputeTargetException
try:
# if you want to connect using SSH key instead of username/password you can provide parameters
private_key_file and private_key_passphrase
attach_config = HDInsightCompute.attach_configuration(address='<clustername>-ssh.azureinsight.net',
ssh_port=22,
username='<ssh-username>',
password='<ssh-pwd>')
hdi_compute = ComputeTarget.attach(workspace=ws,
name='myhdi',
attach_configuration=attach_config)
except ComputeTargetException as e:
print("Caught = {}".format(e.message))
hdi_compute.wait_for_completion(show_output=True)
Or you can attach the HDInsight cluster to your workspace using the Azure portal.
3. Configure: Create a run configuration for the HDI compute target.

# use pyspark framework

run_hdi = RunConfiguration(framework="pyspark")
# Set compute target to the HDI cluster

run_hdi.target = hdi_compute.name
# specify CondaDependencies object to ask system installing numpy

cd = CondaDependencies()
cd.add_conda_package('numpy')
run_hdi.environment.python.conda_dependencies = cd
Azure Batch
Azure Batch is used to run large-scale parallel and high-performance computing (HPC ) applications efficiently
in the cloud. AzureBatchStep can be used in an Azure Machine Learning Pipeline to submit jobs to an Azure
Batch pool of machines.
To attach Azure Batch as a compute target, you must use the Azure Machine Learning SDK and provide the
following information:
Azure Batch compute name: A friendly name to be used for the compute within the workspace
Azure Batch account name: The name of the Azure Batch account
Resource Group: The resource group that contains the Azure Batch account.
The following code demonstrates how to attach Azure Batch as a compute target:
from azureml.core.compute import ComputeTarget, BatchCompute

# Name to associate with new compute in workspace

batch_compute_name = 'mybatchcompute'
# Batch account details needed to attach as compute to workspace

batch_account_name = "<batch_account_name>" # Name of the Batch account
# Name of the resource group which contains this account
batch_resource_group = "<batch_resource_group>"
try:
# check if the compute is already attached
batch_compute = BatchCompute(ws, batch_compute_name)
print('Attaching Batch compute...')
provisioning_config = BatchCompute.attach_configuration(
resource_group=batch_resource_group, account_name=batch_account_name)
batch_compute = ComputeTarget.attach(
ws, batch_compute_name, provisioning_config)
batch_compute.wait_for_completion()
print("Provisioning state:{}".format(batch_compute.provisioning_state))
print("Provisioning errors:{}".format(batch_compute.provisioning_errors))
print("Using Batch compute:{}".format(batch_compute.cluster_resource_id))

Set up in Azure portal
You can access the compute targets that are associated with your workspace in the Azure portal. You can use
the portal to:
View compute targets attached to your workspace
Create a compute target in your workspace
Attach a compute target that was created outside the workspace
After a target is created and attached to your workspace, you will use it in your run configuration with a
ComputeTarget object:

myvm = ComputeTarget(workspace=ws, name='my-vm-name')
View compute targets

To see the compute targets for your workspace, use the following steps:
1. Navigate to the Azure portal and open your workspace.
2. Under Applications, select Compute.
Create a compute target

Follow the previous steps to view the list of compute targets. Then use these steps to create a compute target:
1. Select the plus sign (+) to add a compute target.
2. Enter a name for the compute target.
3. Select Machine Learning Compute as the type of compute to use for Training.
NOTE
Azure Machine Learning Compute is the only managed-compute resource you can create in the Azure portal. All
other compute resources can be attached after they are created.
4. Fill out the form. Provide values for the required properties, especially VM Family, and the maximum
nodes to use to spin up the compute.
5. Select Create.
6. View the status of the create operation by selecting the compute target from the list:
7. You then see the details for the compute target:
Attach compute targets

To use compute targets created outside the Azure Machine Learning service workspace, you must attach them.
Attaching a compute target makes it available to your workspace.
Follow the steps described earlier to view the list of compute targets. Then use the following steps to attach a
compute target:
1. Select the plus sign (+) to add a compute target.
2. Enter a name for the compute target.
3. Select the type of compute to attach for Training:
IMPORTANT
Not all compute types can be attached from the Azure portal. The compute types that can currently be attached
for training include:
A remote VM
Azure Databricks (for use in machine learning pipelines)
Azure Data Lake Analytics (for use in machine learning pipelines)
Azure HDInsight
4. Fill out the form and provide values for the required properties.
NOTE
Microsoft recommends that you use SSH keys, which are more secure than passwords. Passwords are vulnerable
to brute force attacks. SSH keys rely on cryptographic signatures. For information on how to create SSH keys for
use with Azure Virtual Machines, see the following documents:
Create and use SSH keys on Linux or macOS
Create and use SSH keys on Windows
5. Select Attach.
6. View the status of the attach operation by selecting the compute target from the list.
Set up with CLI

You can access the compute targets that are associated with your workspace using the CLI extension for Azure
Machine Learning service. You can use the CLI to:
Create a managed compute target
Update a managed compute target
Attach an unmanaged compute target
For more information, see Resource management.
Set up with VS Code

You can access, create and manage the compute targets that are associated with your workspace using the VS
Code extension for Azure Machine Learning service.
Submit training run using Azure Machine Learning SDK

After you create a run configuration, you use it to run your experiment. The code pattern to submit a training
run is the same for all types of compute targets:
1. Create an experiment to run
2. Submit the run.
3. Wait for the run to complete.
IMPORTANT
When you submit the training run, a snapshot of the directory that contains your training scripts is created and sent to
the compute target. It is also stored as part of the experiment in your workspace. If you change files and submit the run
again, only the changed files will be uploaded.
To prevent files from being included in the snapshot, create a .gitignore or .amlignore file in the directory and add the
files to it. The .amlignore file uses the same syntax and patterns as the .gitignore file. If both files exist, the
.amlignore file takes precedence.
For more information, see Snapshots.
First, create an experiment in your workspace.

experiment_name = 'my_experiment'
exp = Experiment(workspace=ws, name=experiment_name)
Submit the experiment

Submit the experiment with a ScriptRunConfig object. This object includes the:
source_directory: The source directory that contains your training script
script: Identify the training script
run_config: The run configuration, which in turn defines where the training will occur.
For example, to use the local target configuration:
from azureml.core import ScriptRunConfig

import os
script_folder = os.getcwd()
src = ScriptRunConfig(source_directory = script_folder, script = 'train.py', run_config = run_local)
run = exp.submit(src)
run.wait_for_completion(show_output = True)
Switch the same experiment to run in a different compute target by using a different run configuration, such as
the amlcompute target:
src = ScriptRunConfig(source_directory = script_folder, script = 'train.py', run_config = run_amlcompute)

run.wait_for_completion(show_output = True)
Or you can:
Submit the experiment with an Estimator object as shown in Train ML models with estimators.
Submit a HyperDrive run for hyperparameter tuning.
Submit an experiment via the VS Code extension.
Create run configuration and submit run using Azure Machine

Learning CLI
You can use Azure CLI and Machine Learning CLI extension to create run configurations and submit runs on
different compute targets. The following examples assume that you have an existing Azure Machine Learning
Workspace and you have logged in to Azure using az login CLI command.
Create run configuration
The simplest way to create run configuration is to navigate the folder that contains your machine learning
Python scripts, and use CLI command
az ml folder attach
This command creates a subfolder .azureml that contains template run configuration files for different
compute targets. You can copy and edit these files to customize your configuration, for example to add Python
packages or change Docker settings.
First, create an experiment for your runs
az ml experiment create -n <experiment>
Script run
To submit a script run, execute a command
az ml run submit-script -e <experiment> -c <runconfig> my_train.py
HyperDrive run
You can use HyperDrive with Azure CLI to perform parameter tuning runs. First, create a HyperDrive
configuration file in the following format. See Tune hyperparameters for your model article for details on
hyperparameter tuning parameters.
# hdconfig.yml
sampling:
type: random # Supported options: Random, Grid, Bayesian
parameter_space: # specify a name|expression|values tuple for each parameter.
- name: --penalty # The name of a script parameter to generate values for.
expression: choice # supported options: choice, randint, uniform, quniform, loguniform, qloguniform,
normal, qnormal, lognormal, qlognormal
values: [0.5, 1, 1.5] # The list of values, the number of values is dependent on the expression
specified.
policy:
type: BanditPolicy # Supported options: BanditPolicy, MedianStoppingPolicy, TruncationSelectionPolicy,
NoTerminationPolicy
evaluation_interval: 1 # Policy properties are policy specific. See the above link for policy specific
parameter details.
slack_factor: 0.2
primary_metric_name: Accuracy # The metric used when evaluating the policy
primary_metric_goal: Maximize # Maximize|Minimize
max_total_runs: 8 # The maximum number of runs to generate
max_concurrent_runs: 2 # The number of runs that can run concurrently.
max_duration_minutes: 100 # The maximum length of time to run the experiment before cancelling.
Add this file alongside the run configuration files. Then submit a HyperDrive run using:
az ml run submit-hyperdrive -e <experiment> -c <runconfig> --hyperdrive-configuration-name <hdconfig>

my_train.py
Note the arguments section in runconfig and parameter space in HyperDrive config. They contain the
command-line arguments to be passed to training script. The value in runconfig stays the same for each
iteration, while the range in HyperDrive config is iterated over. Do not specify the same argument in both files.
For more details on these az ml CLI commands and full set of arguments, see the reference documentation.
Git tracking and integration

repository is stored in the run history. For example, the current commit ID for the repository is logged as part
of the history.
Notebook examples
See these notebooks for examples of training with various compute targets:
how -to-use-azureml/training
tutorials/img-classification-part1-training.ipynb
Next steps
Tutorial: Train a model uses a managed compute target to train a model.
Learn how to efficiently tune hyperparameters to build better models.
Once you have a trained model, learn how and where to deploy models.
View the RunConfiguration class SDK reference.
Use Azure Machine Learning service with Azure Virtual Networks
Train models with Azure Machine Learning using
estimator
With Azure Machine Learning, you can easily submit your training script to various compute targets, using
RunConfiguration object and ScriptRunConfig object. That pattern gives you a lot of flexibility and maximum
control.
To facilitate deep learning model training, the Azure Machine Learning Python SDK provides an alternative
higher-level abstraction, the estimator class, which allows users to easily construct run configurations. You can
create and use a generic Estimator to submit training script using any learning framework you choose (such as
scikit-learn) on any compute target you choose, whether it's your local machine, a single VM in Azure, or a GPU
cluster in Azure. For PyTorch, TensorFlow and Chainer tasks, Azure Machine Learning also provides respective
PyTorch, TensorFlow, and Chainer estimators to simplify using these frameworks.
Train with an estimator

Once you've created your workspace and set up your development environment, training a model in Azure
Machine Learning involves the following steps:
1. Create a remote compute target (note you can also use local computer as compute target)
2. Upload your training data to datastore (Optional)
3. Create your training script
4. Create an Estimator object
5. Submit the estimator to an experiment object under the workspace
This article focuses on steps 4-5. For steps 1-3, refer to the train a model tutorial for an example.
Single -node training
Use an Estimator for a single-node training run on remote compute in Azure for a scikit-learn model. You
should have already created your compute target object compute_target and your datastore object ds .
script_params = {
'--data-folder': ds.as_mount(),
}
sk_est = Estimator(source_directory='./my-sklearn-proj',
conda_packages=['scikit-learn'])
This code snippet specifies the following parameters to the Estimator constructor.
PARAMETER DESCRIPTION
PARAMETER DESCRIPTION
source_directory Local directory that contains all of your code needed for the
training job. This folder gets copied from your local machine
to the remote compute.
script_params Dictionary specifying the command-line arguments to pass

to your training script entry_script , in the form of
<command-line argument, value> pairs. To specify a
verbose flag in script_params , use
<command-line argument, ""> .
compute_target Remote compute target that your training script will run on,
in this case an Azure Machine Learning Compute
(AmlCompute) cluster. (Note even though AmlCompute
cluster is the commonly used target, it is also possible to
choose other compute target types such as Azure VMs or
even local computer.)
entry_script Filepath (relative to the source_directory ) of the training

script to be run on the remote compute. This file, and any
additional files it depends on, should be located in this folder.
conda_packages List of Python packages to be installed via conda needed by

your training script.
The constructor has another parameter called pip_packages that you use for any pip packages needed.
Now that you've created your Estimator object, submit the training job to be run on the remote compute with a
call to the submit function on your Experiment object experiment .
run = experiment.submit(sk_est)
print(run.get_portal_url())
IMPORTANT
Special Folders Two folders, outputs and logs, receive special treatment by Azure Machine Learning. During training,
when you write files to folders named outputs and logs that are relative to the root directory ( ./outputs and ./logs ,
respectively), the files will automatically upload to your run history so that you have access to them once your run is
finished.
To create artifacts during training (such as model files, checkpoints, data files, or plotted images) write these to the
./outputs folder.
Similarly, you can write any logs from your training run to the ./logs folder. To utilize Azure Machine Learning's
TensorBoard integration make sure you write your TensorBoard logs to this folder. While your run is in progress, you will be
able to launch TensorBoard and stream these logs. Later, you will also be able to restore the logs from any of your previous
runs.
For example, to download a file written to the outputs folder to your local machine after your remote training run:
run.download_file(name='outputs/my_output_file', output_file_path='my_destination_path')
Distributed training and custom Docker images

There are two additional training scenarios you can carry out with the Estimator :
Using a custom Docker image
Distributed training on a multi-node cluster
The following code shows how to carry out distributed training for a Keras model. In addition, instead of using
the default Azure Machine Learning images, it specifies a custom docker image from Docker Hub
continuumio/miniconda for training.
You should have already created your compute target object compute_target . You create the estimator as follows:
estimator = Estimator(source_directory='./my-keras-proj',
node_count=2,
process_count_per_node=1,
distributed_backend='mpi',
conda_packages=['tensorflow', 'keras'],
custom_docker_image='continuumio/miniconda')
The above code exposes the following new parameters to the Estimator constructor:
PARAMETER DESCRIPTION DEFAULT
custom_docker_image Name of the image you want to use. None

Only provide images available in public
docker repositories (in this case Docker
Hub). To use an image from a private
docker repository, use the constructor's
environment_definition parameter
instead. See example.
node_count Number of nodes to use for your 1

training job.
process_count_per_node Number of processes (or "workers") to 1

run on each node. In this case, you use
the 2 GPUs available on each node.
distributed_backend Backend for launching distributed None

training, which the Estimator offers via
MPI. To carry out parallel or distributed
training (e.g., node_count >1 or
process_count_per_node >1 or both),
set distributed_backend='mpi' . The
MPI implementation used by AML is
Open MPI.
Finally, submit the training job:
run = experiment.submit(estimator)

the history.
Examples
For a notebook that shows the basics of an estimator pattern, see:
how -to-use-azureml/training-with-deep-learning/how -to-use-estimator
For a notebook that trains a scikit-learn model by using estimator, see:
tutorials/img-classification-part1-training.ipynb
For notebooks on training models by using deep-learning-framework specific estimators, see:
how -to-use-azureml/training-with-deep-learning
Next steps
Track run metrics during training
Train PyTorch models
Train TensorFlow models
Deploy a trained model
Build scikit-learn models at scale with Azure Machine
Learning service
In this article, learn how to run your scikit-learn training scripts at enterprise scale using Azure Machine Learning's
SKlearn estimator class.
The example scripts in this article are used to classify iris flower images to build a machine learning model based
on scikit-learn's iris dataset.
Whether you're training a machine learning scikit-learn model from the ground-up or you're bringing an existing
model into the cloud, you can use Azure Machine Learning to scale out open-source training jobs using elastic
cloud compute resources. You can build, deploy, version and monitor production-grade models with Azure
Machine Learning.
Prerequisites
Run this code on either of these environments:
Azure Machine Learning Notebook VM - no downloads or installation necessary
Complete the Tutorial: Setup environment and workspace to create a dedicated notebook server pre-
loaded with the SDK and the sample repository.
In the samples training folder on the notebook server, find a completed and expanded notebook by
navigating to this directory: how-to-use-azureml > training > train-hyperparameter-tune-deploy-
with-sklearn folder.
Your own Jupyter Notebook server
Install the Azure Machine Learning SDK.
Create a workspace configuration file.
Download the dataset and sample script file
iris dataset
train_iris.py
You can also find a completed Jupyter Notebook version of this guide on the GitHub samples page. The
notebook includes an expanded section covering intelligent hyperparameter tuning and retrieving the
best model by primary metrics.
Set up the experiment

This section sets up the training experiment by loading the required python packages, initializing a workspace,
creating an experiment, and uploading the training data and training scripts.
Import packages
First, import the necessary Python libraries.
import os
import urllib
import shutil
import azureml


Initialize a workspace
The Azure Machine Learning service workspace is the top-level resource for the service. It provides you with a
centralized place to work with all the artifacts you create. In the Python SDK, you can access the workspace
artifacts by creating a workspace object.
Create a workspace object from the config.json file created in the prerequisites section.
Create a machine learning experiment

Create an experiment and a folder to hold your training scripts. In this example, create an experiment called
"sklearn-iris".
project_folder = './sklearn-iris'
os.makedirs(project_folder, exist_ok=True)
exp = Experiment(workspace=ws, name='sklearn-iris')
Upload dataset and scripts

The datastore is a place where data can be stored and accessed by mounting or copying the data to the compute
target. Each workspace provides a default datastore. Upload the data and training scripts to the datastore so that
they can be easily accessed during training.
1. Create the directory for your data.
os.makedirs('./data/iris', exist_ok=True)
2. Upload the iris dataset to the default datastore.
ds.upload(src_dir='./data/iris', target_path='iris', overwrite=True, show_progress=True)
3. Upload the scikit-learn training script, train_iris.py .
shutil.copy('./train_iris.py', project_folder)
Create or get a compute target

Create a compute target for your scikit-learn job to run on. Scikit-learn only supports single node, CPU computing.
The following code, creates an Azure Machine Learning managed compute (AmlCompute) for your remote
training compute resource. Creation of AmlCompute takes approximately 5 minutes. If the AmlCompute with that
name is already in your workspace, this code will skip the creation process.
cluster_name = "cpu-cluster"
try:
compute_target = ComputeTarget(workspace=ws, name=cluster_name)
print('Found existing compute target')
print('Creating a new compute target...')
max_nodes=4)
compute_target = ComputeTarget.create(ws, cluster_name, compute_config)
compute_target.wait_for_completion(show_output=True, min_node_count=None, timeout_in_minutes=20)
For more information on compute targets, see the what is a compute target article.
Create a scikit-learn estimator

The scikit-learn estimator provides a simple way of launching a scikit-learn training job on a compute target. It is
implemented through the SKLearn class, which can be used to support single-node CPU training.
If your training script needs additional pip or conda packages to run, you can have the packages installed on the
resulting docker image by passing their names through the pip_packages and conda_packages arguments.
from azureml.train.sklearn import SKLearn
script_params = {
'--kernel': 'linear',
'--penalty': 1.0,
}
estimator = SKLearn(source_directory=project_folder,
entry_script='train_iris.py'
pip_packages=['joblib']
)
Submit a run
The Run object provides the interface to the run history while the job is running and after it has completed.
As the Run is executed, it goes through the following stages:

Preparing: A docker image is created according to the TensorFlow estimator. The image is uploaded to the
workspace's container registry and cached for later runs. Logs are also streamed to the run history and can
be viewed to monitor progress.
Scaling: The cluster attempts to scale up if the Batch AI cluster requires more nodes to execute the run than
are currently available.
Running: All scripts in the script folder are uploaded to the compute target, data stores are mounted or
copied, and the entry_script is executed. Outputs from stdout and the ./logs folder are streamed to the run
history and can be used to monitor the run.
Post-Processing: The ./outputs folder of the run is copied over to the run history.
Save and register the model

Once you've trained the model, you can save and register it to your workspace. Model registration lets you store
and version your models in your workspace to simplify model management and deployment.
Add the following code to your training script, train_iris.py, to save the model.
import joblib
joblib.dump(svm_model_linear, 'model.joblib')
Register the model to your workspace with the following code.
model = run.register_model(model_name='sklearn-iris', model_path='model.joblib')
Next steps
In this article, you trained and registered a Keras model on Azure Machine Learning service. To learn how to deploy
a model, continue on to our model deployment article.
How and where to deploy models
Reference architecture for distributed deep learning training in Azure
Build a TensorFlow deep learning model at scale with
Azure Machine Learning
This article shows you how to run your TensorFlow training scripts at scale using Azure Machine Learning's
TensorFlow estimator class. This example trains and registers a TensorFlow model to classify handwritten digits
using a deep neural network (DNN ).
Whether you're developing a TensorFlow model from the ground-up or you're bringing anexisting model into the
cloud, you can use Azure Machine Learning to scale out open-source training jobs to build, deploy, version, and
monitor production-grade models.
Learn more about deep learning vs machine learning.
Prerequisites
In the samples deep learning folder on the notebook server, find a completed and expanded notebook
by navigating to this directory: how-to-use-azureml > training-with-deep-learning > train-
hyperparameter-tune-deploy-with-tensorflow folder.
Download the sample script files mnist-tf.py and utils.py
notebook includes expanded sections covering intelligent hyperparameter tuning, model deployment, and
notebook widgets.

This section sets up the training experiment by loading the required Python packages, initializing a workspace,
Import packages
import os
import urllib
import shutil
import azureml


Create a deep learning experiment

Create an experiment and a folder to hold your training scripts. In this example, create an experiment called "tf-
mnist".
script_folder = './tf-mnist'
exp = Experiment(workspace=ws, name='tf-mnist')

1. Download the MNIST dataset locally.
os.makedirs('./data/mnist', exist_ok=True)
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz', filename =
'./data/mnist/train-images.gz')
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz', filename =
'./data/mnist/train-labels.gz')
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz', filename =
'./data/mnist/test-images.gz')
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz', filename =
'./data/mnist/test-labels.gz')
2. Upload the MNIST dataset to the default datastore.
ds.upload(src_dir='./data/mnist', target_path='mnist', overwrite=True, show_progress=True)
3. Upload the TensorFlow training script, tf_mnist.py , and the helper file, utils.py .
shutil.copy('./tf_mnist.py', script_folder)
shutil.copy('./utils.py', script_folder)

Create a compute target for your TensorFlow job to run on. In this example, create a GPU -enabled Azure Machine
Learning compute cluster.
cluster_name = "gpucluster"
try:
compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_NC6',
max_nodes=4)
Create a TensorFlow estimator

The TensorFlow estimator provides a simple way of launching a TensorFlow training job on a compute target.
The TensorFlow estimator is implemented through the generic estimator class, which can be used to support any
framework. For more information about training models using the generic estimator, see train models with Azure
Machine Learning using estimator
resulting Docker image by passing their names through the pip_packages and conda_packages arguments.
script_params = {
'--data-folder': ws.get_default_datastore().as_mount(),
'--batch-size': 50,
'--first-layer-neurons': 300,
'--second-layer-neurons': 100,
'--learning-rate': 0.01
}
est = TensorFlow(source_directory=script_folder,
entry_script='tf_mnist.py',
use_gpu=True)
Submit a run
run = exp.submit(est)
Preparing: A Docker image is created according to the TensorFlow estimator. The image is uploaded to the
Scaling: The cluster attempts to scale up if the Batch AI cluster requires more nodes to execute the run
than are currently available.
Register or download a model

Once you've trained the model, you can register it to your workspace. Model registration lets you store and
version your models in your workspace to simplify model management and deployment.
model = run.register_model(model_name='tf-dnn-mnist', model_path='outputs/model')
You can also download a local copy of the model by using the Run object. In the training script mnist-tf.py , a
TensorFlow saver object persists the model to a local folder (local to the compute target). You can use the Run
object to download a copy.
# Create a model folder in the current directory

os.makedirs('./model', exist_ok=True)
for f in run.get_file_names():
if f.startswith('outputs/model'):
output_file_path = os.path.join('./model', f.split('/')[-1])
print('Downloading from {} to {} ...'.format(f, output_file_path))
run.download_file(name=f, output_file_path=output_file_path)
Distributed training
The TensorFlow estimator also supports distributed training across CPU and GPU clusters. You can easily run
distributed TensorFlow jobs and Azure Machine Learning service will manage the orchestration for you.
Azure Machine Learning service supports two methods of distributed training in TensorFlow:
MPI-based distributed training using the Horovod framework
Native distributed TensorFlow using the parameter server method
Horovod
Horovod is an open-source framework for distributed training developed by Uber. It offers an easy path to
distributed GPU TensorFlow jobs.
To use Horovod, specify an MpiConfiguration object for the distributed_training parameter in the TensorFlow
constructor. This parameter ensures that Horovod library is installed for you to use in your training script.
from azureml.core.runconfig import MpiConfiguration
from azureml.train.dnn import TensorFlow
# Tensorflow constructor
estimator= TensorFlow(source_directory=project_folder,
entry_script='script.py',
node_count=2,
distributed_training=MpiConfiguration(),
framework_version='1.13',
use_gpu=True)
Parameter server
You can also run native distributed TensorFlow, which uses the parameter server model. In this method, you train
across a cluster of parameter servers and workers. The workers calculate the gradients during training, while the
parameter servers aggregate the gradients.
To use the parameter server method, specify a TensorflowConfiguration object for the distributed_training
parameter in the TensorFlow constructor.
distributed_training = TensorflowConfiguration()
distributed_training.worker_count = 2
# Tensorflow constructor
estimator= TensorFlow(source_directory=project_folder,
node_count=2,
distributed_training=distributed_training,
use_gpu=True)
# submit the TensorFlow job

run = exp.submit(tf_est)
Define cluster specifications in 'TF_CONFIG`

You also need the network addresses and ports of the cluster for the tf.train.ClusterSpec , so Azure Machine
Learning sets the TF_CONFIG environment variable for you.
The TF_CONFIG environment variable is a JSON string. Here is an example of the variable for a parameter server:
TF_CONFIG='{
"cluster": {
"ps": ["host0:2222", "host1:2222"],
"worker": ["host2:2222", "host3:2222", "host4:2222"],
},
"task": {"type": "ps", "index": 0},
"environment": "cloud"
}'
For TensorFlow's high level tf.estimator API, TensorFlow parses the TF_CONFIG variable and builds the cluster
spec for you.
For TensorFlow's lower-level core APIs for training, parse the TF_CONFIG variable and build the
tf.train.ClusterSpec in your training code.
import os, json

import tensorflow as tf
tf_config = os.environ.get('TF_CONFIG')
if not tf_config or tf_config == "":
raise ValueError("TF_CONFIG not found.")
tf_config_json = json.loads(tf_config)
cluster_spec = tf.train.ClusterSpec(cluster)
Next steps
In this article, you trained and registered a TensorFlow model. To learn how to deploy a model to a GPU -enabled
cluster, continue on to our GPU model deployment article.
Train and register a Keras classification model with
Azure Machine Learning service
This article shows you how to train and register a Keras classification model built on TensorFlow using Azure
Machine Learning service. It uses the popular MNIST dataset to classify handwritten digits using a deep neural
network (DNN ) built using the Keras Python library running on top of TensorFlow.
Keras is a high-level neural network API capable of running top of other popular DNN frameworks to simplify
development. With Azure Machine Learning service, you can rapidly scale out training jobs using elastic cloud
compute resources. You can also track your training runs, version models, deploy models, and much more.
Whether you're developing a Keras model from the ground-up or you're bringing an existing model into the cloud,
Azure Machine Learning service can help you build production-ready models.
See the conceptual article for information on the differences between machine learning and deep learning.
Prerequisites
In the samples folder on the notebook server, find a completed and expanded notebook by navigating to
this directory: how-to-use-azureml > training-with-deep-learning > train-hyperparameter-tune-
deploy-with-keras folder.
Download the sample script files mnist-keras.py and utils.py
notebook widgets.

Import packages
import os
import urllib
import shutil
import azureml


Create an experiment and a folder to hold your training scripts. In this example, create an experiment called "keras-
mnist".
script_folder = './keras-mnist'
exp = Experiment(workspace=ws, name='keras-mnist')

1. Download the MNIST dataset locally.
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz', filename =
'./data/mnist/train-images.gz')
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz', filename =
'./data/mnist/train-labels.gz')
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz', filename =
'./data/mnist/test-images.gz')
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz', filename =
'./data/mnist/test-labels.gz')
2. Upload the MNIST dataset to the default datastore.
ds.upload(src_dir='./data/mnist', target_path='mnist', overwrite=True, show_progress=True)
3. Upload the Keras training script, keras_mnist.py , and the helper file, utils.py .
shutil.copy('./keras_mnist.py', script_folder)
shutil.copy('./utils.py', script_folder)

Create a compute target for your TensorFlow job to run on. In this example, create a GPU -enabled Azure Machine
try:
max_nodes=4)
Create a TensorFlow estimator and import Keras

The TensorFlow estimator provides a simple way of launching TensorFlow training jobs on compute target. Since
Keras runs on top of TensorFlow, you can use the TensorFlow estimator and import the Keras library using the
pip_packages argument.
The TensorFlow estimator is implemented through the generic estimator class, which can be used to support any
framework. Additionally, create a dictionary script_params that contains the DNN hyperparameter settings. For
more information about training models using the generic estimator, see train models with Azure Machine
Learning using estimator
script_params = {
'--data-folder': ds.path('mnist').as_mount(),
'--batch-size': 50,
'--first-layer-neurons': 300,
'--second-layer-neurons': 100,
'--learning-rate': 0.001
}
est = TensorFlow(source_directory=script_folder,
entry_script='keras_mnist.py',
pip_packages=['keras', 'matplotlib'],
use_gpu=True)
Submit a run
Preparing: A docker image is created according to the TensorFlow estimator. The image is uploaded to the
Register the model

Once you've trained the DNN model, you can register it to your workspace. Model registration lets you store and
model = run.register_model(model_name='keras-dnn-mnist', model_path='outputs/model')
You can also download a local copy of the model. This can be useful for doing additional model validation work
locally. In the training script, mnist-keras.py , a TensorFlow saver object persists the model to a local folder (local to
the compute target). You can use the Run object to download a copy from datastore.

Next steps
In this article, you trained and registered a Keras model on Azure Machine Learning service. To learn how to
deploy a model, continue on to our model deployment article.
Train Pytorch deep learning models at scale with
Azure Machine Learning
In this article, learn how to run your PyTorch training scripts at enterprise scale using Azure Machine Learning's
PyTorch estimator class.
The example scripts in this article are used to classify chicken and turkey images to build a deep learning neural
network based on PyTorch's transfer learning tutorial.
Whether you're training a deep learning PyTorch model from the ground-up or you're bringing an existing model
into the cloud, you can use Azure Machine Learning to scale out open-source training jobs using elastic cloud
compute resources. You can build, deploy, version, and monitor production-grade models with Azure Machine
Learning.
Prerequisites
In the samples deep learning folder on the notebook server, find a completed and expanded notebook
by navigating to this directory: how-to-use-azureml > training-with-deep-learning > train-
hyperparameter-tune-deploy-with-pytorch folder.
Download the sample script files pytorch_train.py
notebook widgets.

Import packages
import os
import shutil


from azureml.train.dnn import PyTorch

Create an experiment and a folder to hold your training scripts. In this example, create an experiment called
"pytorch-birds".
project_folder = './pytorch-birds'
experiment_name = 'pytorch-birds'
experiment = Experiment(ws, name=experiment_name)
Get the data

The dataset consists of about 120 training images each for turkeys and chickens, with 100 validation images for
each class. We will download and extract the dataset as part of our training script pytorch_train.py . The images
are a subset of the Open Images v5 Dataset.
Prepare training scripts
In this tutorial, the training script, pytorch_train.py , is already provided. In practice, you can take any custom
training script, as is, and run it with Azure Machine Learning service.
Upload the Pytorch training script, pytorch_train.py .
shutil.copy('pytorch_train.py', project_folder)
However, if you would like to use Azure Machine Learning service tracking and metrics capabilities, you will have
to add a small amount code inside your training script. Examples of metrics tracking can be found in
pytorch_train.py .

Create a compute target for your PyTorch job to run on. In this example, create a GPU -enabled Azure Machine
try:
max_nodes=4)
Create a PyTorch estimator

The PyTorch estimator provides a simple way of launching a PyTorch training job on a compute target.
The PyTorch estimator is implemented through the generic estimator class, which can be used to support any
resulting docker image by passing their names through the pip_packages and conda_packages arguments.
script_params = {
'--num_epochs': 30,
'--output_dir': './outputs'
}
estimator = PyTorch(source_directory=project_folder,
entry_script='pytorch_train.py',
use_gpu=True,
pip_packages=['pillow==5.4.1'])
Submit a run

Preparing: A docker image is created according to the PyTorch estimator. The image is uploaded to the
Scaling: The cluster attempts to scale up if the Batch AI cluster requires more nodes to execute the run
than are currently available.
Register or download a model

Once you've trained the model, you can register it to your workspace. Model registration lets you store and
model = run.register_model(model_name='pt-dnn', model_path='outputs/')
You can also download a local copy of the model by using the Run object. In the training script pytorch_train.py ,
a PyTorch save object persists the model to a local folder (local to the compute target). You can use the Run object
to download a copy.

Distributed training
The PyTorch estimator also supports distributed training across CPU and GPU clusters. You can easily run
distributed PyTorch jobs and Azure Machine Learning service will manage the orchestration for you.
Horovod
Horovod is an open-source, all reduce framework for distributed training developed by Uber. It offers an easy
path to distributed GPU PyTorch jobs.
To use Horovod, specify an MpiConfiguration object for the distributed_training parameter in the PyTorch
constructor. This parameter ensures that Horovod library is installed for you to use in your training script.
from azureml.train.dnn import PyTorch
estimator= PyTorch(source_directory=project_folder,
node_count=2,
distributed_training=MpiConfiguration(),
framework_version='1.13',
use_gpu=True)
Horovod and its dependencies will be installed for you, so you can import it in your training script train.py as
follows:
import torch
import horovod
Export to ONNX
To optimize inference with the ONNX Runtime, convert your trained PyTorch model to the ONNX format.
Inference, or model scoring, is the phase where the deployed model is used for prediction, most commonly on
production data. See the tutorial for an example.
Next steps
In this article, you trained and registered a deep learning, neural network using PyTorch on Azure Machine
Learning service. To learn how to deploy a model, continue on to our model deployment article.
Train and register Chainer models at scale with Azure
Machine Learning service
In this article, learn how to run your Chainer training scripts at enterprise scale using Azure Machine Learning's
Chainer estimator class. The example training script in this article uses the popular MNIST dataset to classify
handwritten digits using a deep neural network (DNN ) built using the Chainer Python library running on top of
numpy.
Whether you're training a deep learning Chainer model from the ground-up or you're bringing an existing model
into the cloud, you can use Azure Machine Learning to scale out open-source training jobs using elastic cloud
compute resources. You can build, deploy, version, and monitor production-grade models with Azure Machine
Learning.
Prerequisites
In the samples deep learning folder on the notebook server, find a completed notebook and files in the
how-to-use-azureml/training-with-deep-learning/train-hyperparameter-tune-deploy-with-
chainer folder. The notebook includes expanded sections covering intelligent hyperparameter tuning,
model deployment, and notebook widgets.
Download the sample script file chainer_mnist.py.
You can also find a completed Jupyter Notebook version of this guide on GitHub samples page. The
notebook includes expanded sections covering intelligent hyperparameter tuning, model deployment,
and notebook widgets.

Import packages
First, import the azureml.core Python library and display the version number.
# Check core SDK version number
import azureml.core
print("SDK version:", azureml.core.VERSION)
Create a workspace object by reading the config.json file created in the prerequisites section:
Create a project directory

Create a directory that will contain all the necessary code from your local machine that you will need access to on
the remote resource. This includes the training script and any additional files your training script depends on.
import os
project_folder = './chainer-mnist'
Prepare training script

In this tutorial, the training script chainer_mnist.py is already provided for you. In practice, you should be able to
take any custom training script as is and run it with Azure ML without having to modify your code.
To use Azure ML's tracking and metrics capabilities, add a small amount of Azure ML code inside your training
script. The training script chainer_mnist.py shows how to log some metrics to your Azure ML run using the Run
object within the script.
The provided training script uses example data from the chainer datasets.mnist.get_mnist function. For your own
data, you may need to use steps such as Upload dataset and scripts to make data available during training.
Copy the training script chainer_mnist.py into your project directory.
import shutil
shutil.copy('chainer_mnist.py', project_folder)

Create an experiment. In this example, create an experiment called "chainer-mnist".
experiment_name = 'chainer-mnist'
experiment = Experiment(ws, name=experiment_name)
Create or get a compute target

You need a compute target for training your model. In this example, you use Azure ML managed compute
(AmlCompute) for your remote training compute resource.
Creation of AmlCompute takes approximately 5 minutes. If the AmlCompute with that name is already in
your workspace, this code skips the creation process.

# choose a name for your cluster

cluster_name = "gpu-cluster"
try:
print('Found existing compute target.')
max_nodes=4)

compute_target.wait_for_completion(show_output=True)
# use get_status() to get a detailed status for the current cluster.

print(compute_target.get_status().serialize())
Create a Chainer estimator

The Chainer estimator provides a simple way of launching Chainer training jobs on your compute target.
The Chainer estimator is implemented through the generic estimator class, which can be used to support any
from azureml.train.dnn import Chainer
script_params = {
'--epochs': 10,
'--batchsize': 128,
'--output_dir': './outputs'
}
estimator = Chainer(source_directory=project_folder,
pip_packages=['numpy', 'pytest'],
entry_script='chainer_mnist.py',
use_gpu=True)
Submit a run

Preparing: A docker image is created according to the Chainer estimator. The image is uploaded to the
Save and register the model

Once you've trained the model, you can save and register it to your workspace. Model registration lets you store
and version your models in your workspace to simplify model management and deployment.
After the model training has completed, register the model to your workspace with the following code.
model = run.register_model(model_name='chainer-dnn-mnist', model_path='outputs/model.npz')
TIP
If you receive an error that the model is not found, give it a minute and try again. Sometimes there is a slight lag between
the end of the training run and the availability of the model in the outputs directory.
You can also download a local copy of the model. This can be useful for doing additional model validation work
locally. In the training script, chainer_mnist.py , a saver object persists the model to a local folder (local to the
compute target). You can use the Run object to download a copy from datastore.

Next steps
In this article, you trained and registered a deep learning, neural network using Chainer on Azure Machine
Learning service. To learn how to deploy a model, continue on to our model deployment article.
View our reference architecture for distributed deep learning training in Azure
Tune hyperparameters for your model with Azure
Machine Learning service
Efficiently tune hyperparameters for your model using Azure Machine Learning service. Hyperparameter tuning
includes the following steps:
Define the parameter search space
Specify a primary metric to optimize
Specify early termination criteria for poorly performing runs
Allocate resources for hyperparameter tuning
Launch an experiment with the above configuration
Visualize the training runs
Select the best performing configuration for your model
What are hyperparameters?

Hyperparameters are adjustable parameters you choose to train a model that govern the training process itself.
For example, to train a deep neural network, you decide the number of hidden layers in the network and the
number of nodes in each layer prior to training the model. These values usually stay constant during the training
process.
In deep learning / machine learning scenarios, model performance depends heavily on the hyperparameter
values selected. The goal of hyperparameter exploration is to search across various hyperparameter
configurations to find a configuration that results in the best performance. Typically, the hyperparameter
exploration process is painstakingly manual, given that the search space is vast and evaluation of each
configuration can be expensive.
Azure Machine Learning allows you to automate hyperparameter exploration in an efficient manner, saving you
significant time and resources. You specify the range of hyperparameter values and a maximum number of
training runs. The system then automatically launches multiple simultaneous runs with different parameter
configurations and finds the configuration that results in the best performance, measured by the metric you
choose. Poorly performing training runs are automatically early terminated, reducing wastage of compute
resources. These resources are instead used to explore other hyperparameter configurations.
Define search space

Automatically tune hyperparameters by exploring the range of values defined for each hyperparameter.
Types of hyperparameters
Each hyperparameter can either be discrete or continuous and has a distribution of values described by a
parameter expression.
Discrete hyperparameters
Discrete hyperparameters are specified as a choice among discrete values. choice can be:
one or more comma-separated values
a range object
any arbitrary list object
{
"batch_size": choice(16, 32, 64, 128)
"number_of_hidden_layers": choice(range(1,5))
}
In this case, batch_size takes on one of the values [16, 32, 64, 128] and number_of_hidden_layers takes on one
of the values [1, 2, 3, 4].
Advanced discrete hyperparameters can also be specified using a distribution. The following distributions are
supported:
quniform(low, high, q) - Returns a value like round(uniform(low, high) / q) * q
qloguniform(low, high, q) - Returns a value like round(exp(uniform (low, high)) / q) * q
qnormal(mu, sigma, q) - Returns a value like round(normal(mu, sigma) / q) * q
qlognormal(mu, sigma, q) - Returns a value like round(exp(normal(mu, sigma)) / q) * q
Continuous hyperparameters
Continuous hyperparameters are specified as a distribution over a continuous range of values. Supported
distributions include:
uniform(low, high) - Returns a value uniformly distributed between low and high
loguniform(low, high) - Returns a value drawn according to exp(uniform (low, high)) so that the logarithm of
the return value is uniformly distributed
normal(mu, sigma) - Returns a real value that's normally distributed with mean mu and standard deviation
sigma
lognormal(mu, sigma) - Returns a value drawn according to exp(normal(mu, sigma)) so that the logarithm of
the return value is normally distributed
An example of a parameter space definition:
{
"learning_rate": normal(10, 3),
"keep_probability": uniform(0.05, 0.1)
}
This code defines a search space with two parameters - learning_rate and keep_probability . learning_rate
has a normal distribution with mean value 10 and a standard deviation of 3. keep_probability has a uniform
distribution with a minimum value of 0.05 and a maximum value of 0.1.
Sampling the hyperparameter space
You can also specify the parameter sampling method to use over the hyperparameter space definition. Azure
Machine Learning service supports random sampling, grid sampling, and Bayesian sampling.
Random sampling
In random sampling, hyperparameter values are randomly selected from the defined search space. Random
sampling allows the search space to include both discrete and continuous hyperparameters.
from azureml.train.hyperdrive import RandomParameterSampling

param_sampling = RandomParameterSampling( {
"learning_rate": normal(10, 3),
"keep_probability": uniform(0.05, 0.1),
}
)
Grid sampling
Grid sampling performs a simple grid search over all feasible values in the defined search space. It can only be
used with hyperparameters specified using choice . For example, the following space has a total of six samples:
from azureml.train.hyperdrive import GridParameterSampling

param_sampling = GridParameterSampling( {
"num_hidden_layers": choice(1, 2, 3),
"batch_size": choice(16, 32)
}
)
Bayesian sampling
Bayesian sampling is based on the Bayesian optimization algorithm and makes intelligent choices on the
hyperparameter values to sample next. It picks the sample based on how the previous samples performed, such
that the new sample improves the reported primary metric.
When you use Bayesian sampling, the number of concurrent runs has an impact on the effectiveness of the
tuning process. Typically, a smaller number of concurrent runs can lead to better sampling convergence, since
the smaller degree of parallelism increases the number of runs that benefit from previously completed runs.
Bayesian sampling only supports choice , uniform , and quniform distributions over the search space.
from azureml.train.hyperdrive import BayesianParameterSampling

param_sampling = BayesianParameterSampling( {
"learning_rate": uniform(0.05, 0.1),
}
)
NOTE
Bayesian sampling does not support any early termination policy (See Specify an early termination policy). When using
Bayesian parameter sampling, set early_termination_policy = None , or leave off the early_termination_policy
parameter.
Specify primary metric

Specify the primary metric you want the hyperparameter tuning experiment to optimize. Each training run is
evaluated for the primary metric. Poorly performing runs (where the primary metric does not meet criteria set
by the early termination policy) will be terminated. In addition to the primary metric name, you also specify the
goal of the optimization - whether to maximize or minimize the primary metric.
primary_metric_name : The name of the primary metric to optimize. The name of the primary metric needs to
exactly match the name of the metric logged by the training script. See Log metrics for hyperparameter
tuning.
primary_metric_goal : It can be either PrimaryMetricGoal.MAXIMIZE or PrimaryMetricGoal.MINIMIZE and
determines whether the primary metric will be maximized or minimized when evaluating the runs.
primary_metric_name="accuracy",
primary_metric_goal=PrimaryMetricGoal.MAXIMIZE
Optimize the runs to maximize "accuracy". Make sure to log this value in your training script.
Log metrics for hyperparameter tuning
The training script for your model must log the relevant metrics during model training. When you configure the
hyperparameter tuning, you specify the primary metric to use for evaluating run performance. (See Specify a
primary metric to optimize.) In your training script, you must log this metric so it is available to the
hyperparameter tuning process.
Log this metric in your training script with the following sample snippet:

run_logger = Run.get_context()
run_logger.log("accuracy", float(val_accuracy))
The training script calculates the val_accuracy and logs it as "accuracy", which is used as the primary metric.
Each time the metric is logged it is received by the hyperparameter tuning service. It is up to the model
developer to determine how frequently to report this metric.
Specify early termination policy

Terminate poorly performing runs automatically with an early termination policy. Termination reduces wastage
of resources and instead uses these resources for exploring other parameter configurations.
When using an early termination policy, you can configure the following parameters that control when a policy
is applied:
evaluation_interval : the frequency for applying the policy. Each time the training script logs the primary
metric counts as one interval. Thus an evaluation_interval of 1 will apply the policy every time the training
script reports the primary metric. An evaluation_interval of 2 will apply the policy every other time the
training script reports the primary metric. If not specified, evaluation_interval is set to 1 by default.
delay_evaluation : delays the first policy evaluation for a specified number of intervals. It is an optional
parameter that allows all configurations to run for an initial minimum number of intervals, avoiding
premature termination of training runs. If specified, the policy applies every multiple of evaluation_interval
that is greater than or equal to delay_evaluation.
Azure Machine Learning service supports the following Early Termination Policies.
Bandit policy
Bandit is a termination policy based on slack factor/slack amount and evaluation interval. The policy early
terminates any runs where the primary metric is not within the specified slack factor / slack amount with respect
to the best performing training run. It takes the following configuration parameters:
slack_factor or slack_amount : the slack allowed with respect to the best performing training run.
slack_factor specifies the allowable slack as a ratio. slack_amount specifies the allowable slack as an
absolute amount, instead of a ratio.
For example, consider a Bandit policy being applied at interval 10. Assume that the best performing run
at interval 10 reported a primary metric 0.8 with a goal to maximize the primary metric. If the policy was
specified with a slack_factor of 0.2, any training runs, whose best metric at interval 10 is less than 0.66
(0.8/(1+ slack_factor )) will be terminated. If instead, the policy was specified with a slack_amount of 0.2,
any training runs, whose best metric at interval 10 is less than 0.6 (0.8 - slack_amount ) will be terminated.
evaluation_interval : the frequency for applying the policy (optional parameter).
delay_evaluation : delays the first policy evaluation for a specified number of intervals (optional
parameter).
from azureml.train.hyperdrive import BanditPolicy
early_termination_policy = BanditPolicy(slack_factor = 0.1, evaluation_interval=1, delay_evaluation=5)
In this example, the early termination policy is applied at every interval when metrics are reported, starting at
evaluation interval 5. Any run whose best metric is less than (1/(1+0.1) or 91% of the best performing run will
be terminated.
Median stopping policy
Median stopping is an early termination policy based on running averages of primary metrics reported by the
runs. This policy computes running averages across all training runs and terminates runs whose performance is
worse than the median of the running averages. This policy takes the following configuration parameters:
evaluation_interval : the frequency for applying the policy (optional parameter).
delay_evaluation : delays the first policy evaluation for a specified number of intervals (optional parameter ).
from azureml.train.hyperdrive import MedianStoppingPolicy

early_termination_policy = MedianStoppingPolicy(evaluation_interval=1, delay_evaluation=5)
In this example, the early termination policy is applied at every interval starting at evaluation interval 5. A run
will be terminated at interval 5 if its best primary metric is worse than the median of the running averages over
intervals 1:5 across all training runs.
Truncation selection policy
Truncation selection cancels a given percentage of lowest performing runs at each evaluation interval. Runs are
compared based on their performance on the primary metric and the lowest X% are terminated. It takes the
following configuration parameters:
truncation_percentage : the percentage of lowest performing runs to terminate at each evaluation interval.
Specify an integer value between 1 and 99.
evaluation_interval : the frequency for applying the policy (optional parameter ).
delay_evaluation : delays the first policy evaluation for a specified number of intervals (optional parameter ).
from azureml.train.hyperdrive import TruncationSelectionPolicy

early_termination_policy = TruncationSelectionPolicy(evaluation_interval=1, truncation_percentage=20,
delay_evaluation=5)
In this example, the early termination policy is applied at every interval starting at evaluation interval 5. A run
will be terminated at interval 5 if its performance at interval 5 is in the lowest 20% of performance of all runs at
interval 5.
No termination policy
If you want all training runs to run to completion, set policy to None. This will have the effect of not applying any
early termination policy.
policy=None
Default policy
If no policy is specified, the hyperparameter tuning service will let all training runs execute to completion.
NOTE
If you are looking for a conservative policy that provides savings without terminating promising jobs, you can use a
Median Stopping Policy with evaluation_interval 1 and delay_evaluation 5. These are conservative settings, that
can provide approximately 25%-35% savings with no loss on primary metric (based on our evaluation data).
Allocate resources
Control your resource budget for your hyperparameter tuning experiment by specifying the maximum total
number of training runs. Optionally specify the maximum duration for your hyperparameter tuning experiment.
max_total_runs : Maximum total number of training runs that will be created. Upper bound - there may be
fewer runs, for instance, if the hyperparameter space is finite and has fewer samples. Must be a number
between 1 and 1000.
max_duration_minutes : Maximum duration in minutes of the hyperparameter tuning experiment. Parameter is
optional, and if present, any runs that would be running after this duration are automatically canceled.
NOTE
If both max_total_runs and max_duration_minutes are specified, the hyperparameter tuning experiment terminates
when the first of these two thresholds is reached.
Additionally, specify the maximum number of training runs to run concurrently during your hyperparameter
tuning search.
max_concurrent_runs : Maximum number of runs to run concurrently at any given moment. If none specified,
all max_total_runs will be launched in parallel. If specified, must be a number between 1 and 100.
NOTE
The number of concurrent runs is gated on the resources available in the specified compute target. Hence, you need to
ensure that the compute target has the available resources for the desired concurrency.
Allocate resources for hyperparameter tuning:
max_total_runs=20,
max_concurrent_runs=4
This code configures the hyperparameter tuning experiment to use a maximum of 20 total runs, running four
configurations at a time.
Configure experiment
Configure your hyperparameter tuning experiment using the defined hyperparameter search space, early
termination policy, primary metric, and resource allocation from the sections above. Additionally, provide an
estimator that will be called with the sampled hyperparameters. The estimator describes the training script
you run, the resources per job (single or multi-gpu), and the compute target to use. Since concurrency for your
hyperparameter tuning experiment is gated on the resources available, ensure that the compute target specified
in the estimator has sufficient resources for your desired concurrency. (For more information on estimators,
see how to train models.)
Configure your hyperparameter tuning experiment:
from azureml.train.hyperdrive import HyperDriveConfig
hyperdrive_run_config = HyperDriveConfig(estimator=estimator,
hyperparameter_sampling=param_sampling,
policy=early_termination_policy,
primary_metric_name="accuracy",
primary_metric_goal=PrimaryMetricGoal.MAXIMIZE,
max_total_runs=100,
max_concurrent_runs=4)
Submit experiment
Once you define your hyperparameter tuning configuration, submit an experiment:

experiment = Experiment(workspace, experiment_name)
hyperdrive_run = experiment.submit(hyperdrive_run_config)
experiment_name is the name you assign to your hyperparameter tuning experiment, and workspace is the
workspace in which you want to create the experiment (For more information on experiments, see How does
Azure Machine Learning service work?)
Visualize experiment
The Azure Machine Learning SDK provides a Notebook widget that visualizes the progress of your training
runs. The following snippet visualizes all your hyperparameter tuning runs in one place in a Jupyter notebook:

RunDetails(hyperdrive_run).show()
This code displays a table with details about the training runs for each of the hyperparameter configurations.
You can also visualize the performance of each of the runs as training progresses.
Additionally, you can visually identify the correlation between performance and values of individual
hyperparameters using a Parallel Coordinates Plot.
You can visualize all your hyperparameter tuning runs in the Azure web portal as well. For more information on
how to view an experiment in the web portal, see how to track experiments.
Find the best model

Once all of the hyperparameter tuning runs have completed, identify the best performing configuration and the
corresponding hyperparameter values:
best_run = hyperdrive_run.get_best_run_by_primary_metric()
best_run_metrics = best_run.get_metrics()
parameter_values = best_run.get_details()['runDefinition']['Arguments']
print('Best Run Id: ', best_run.id)

print('\n Accuracy:', best_run_metrics['accuracy'])
print('\n learning rate:',parameter_values[3])
print('\n keep probability:',parameter_values[5])
print('\n batch size:',parameter_values[7])
Sample notebook
Refer to train-hyperparameter-* notebooks in this folder:
how -to-use-azureml/training-with-deep-learning
Next steps
Track an experiment
Use secrets in training runs
In this article, you learn how to use secrets in training runs securely. For example, to connect to an external
database to query training data, you would need to pass username and password to the remote run context.
Coding such values into training scripts in clear text is insecure as it would expose the secret.
Instead, your Azure Machine Learning Workspace has Azure Key Vault as associated resource. This Key Vault can
be used for passing secrets to remote runs securely through a set of APIs in Azure Machine Learning Python SDK
The basic flow for using secrets is:
1. On local computer, log in to Azure and connect to your Workspace
2. On local computer, set a secret in Workspace Key Vault
3. Submit a remote run.
4. Within remote run, get the secret from Key Value and use it.
Set secrets
In Azure Machine Learning Python SDK, the Keyvault class contains methods for setting secrets. In your local
Python session, first obtain a reference to Workspace Key Vault, and then use set_secret method to set a secret by
name and value.

import os
my_secret = os.environ.get("MY_SECRET")
keyvault = ws.get_default_keyvault()
keyvault.set_secret(name="mysecret", value = my_secret)
Do not put the secret value in Python code as it is insecure to store it in file as cleartext. Instead, obtain the secret
value from environment variable, for example Azure DevOps build secret, or from interactive user input.
You can list secret names using list_secrets method. The set_secret method updates the secret value if the name
already exists.
Get secrets
In your local code, you can use Keyvault.get_secret method to get the secret value by name.
In runs submitted using Experiment.submit, use Run.get_secret method. Because submitted run is aware of its
Workspace, this method shortcuts the Workspace instantiation and returns the secret value directly.
# Code in submitted run

secret_value = run.get_secret(name="mysecret")
Be careful not to expose the secret value by writing or printing it out.

The set and get methods also have batch versions set_secrets and get_secrets for accessing multiple secrets at
once.
Next steps
View example notebook
Learn about enterprise security with Azure Machine Learning service
Start, monitor, and cancel training runs in Python
The Azure Machine Learning SDK for Python and Machine Learning CLI provide various methods to monitor,
organize, and manage your runs for training and experimentation.
This article shows examples of the following tasks:
Monitor run performance.
Cancel or fail runs.
Create child runs.
Tag and find runs.
Prerequisites
You'll need the following items:
The Azure Machine Learning SDK for Python (version 1.0.21 or later). To install or update to the latest
version of the SDK, see Install or update the SDK.
To check your version of the Azure Machine Learning SDK, use the following code:
print(azureml.core.VERSION)
The Azure CLI and CLI extension for Azure Machine Learning service.
Start a run and its logging process

Using the SDK
Set up your experiment by importing the Workspace, Experiment, Run, and ScriptRunConfig classes from the
azureml.core package.
import azureml.core
from azureml.core import Workspace, Experiment, Run
exp = Experiment(workspace=ws, name="explore-runs")
Start a run and its logging process with the start_logging() method.
notebook_run = exp.start_logging()
notebook_run.log(name="message", value="Hello from run!")
Using the CLI

To start a run of your experiment, use the following steps:
1. From a shell or command prompt, use the Azure CLI to authenticate to your Azure subscription:
az login
2. Attach a workspace configuration to the folder that contains your training script. Replace myworkspace with
your Azure Machine Learning service workspace. Replace myresourcegroup with the Azure resource group
that contains your workspace:
az ml folder attach -w myworkspace -g myresourcegroup
This command creates a .azureml subdirectory that contains example runconfig and conda environment
files. It also contains a config.json file that is used to communicate with your Azure Machine Learning
workspace.
For more information, see az ml folder attach.
3. To start the run, use the following command. When using this command, specify the name of the runconfig
file (the text before *.runconfig if you are looking at your file system) against the -c parameter.
az ml run submit-script -c sklearn -e testexperiment train.py
TIP
The az ml folder attach command created a .azureml subdirectory, which contains two example runconfig files.
If you have a Python script that creates a run configuration object programmatically, you can use RunConfig.save() to
save it as a runconfig file.
For more example runconfig files, see https://github.com/MicrosoftDocs/pipelines-azureml/tree/master/.azureml.
For more information, see az ml run submit-script.
Monitor the status of a run

Using the SDK
Get the status of a run with the get_status() method.
print(notebook_run.get_status())
To get the run ID, execution time, and additional details about the run, use the get_details() method.
print(notebook_run.get_details())
When your run finishes successfully, use the complete() method to mark it as completed.
notebook_run.complete()
If you use Python's with...as design pattern, the run will automatically mark itself as completed when the run is
out of scope. You don't need to manually mark the run as completed.
with exp.start_logging() as notebook_run:
notebook_run.log(name="message", value="Hello from run!")
Using the CLI

1. To view a list of runs for your experiment, use the following command. Replace experiment with the name
of your experiment:
az ml run list --experiment-name experiment
This command returns a JSON document that lists information about runs for this experiment.
For more information, see az ml experiment list.
2. To view information on a specific run, use the following command. Replace runid with the ID of the run:
az ml run show -r runid
This command returns a JSON document that lists information about the run.
For more information, see az ml run show.
Cancel or fail runs

If you notice a mistake or if your run is taking too long to finish, you can cancel the run.
Using the SDK
To cancel a run using the SDK, use the cancel() method:
run_config = ScriptRunConfig(source_directory='.', script='hello_with_delay.py')

local_script_run = exp.submit(run_config)
print(local_script_run.get_status())
local_script_run.cancel()
If your run finishes, but it contains an error (for example, the incorrect training script was used), you can use the
fail() method to mark it as failed.
local_script_run.fail()
Using the CLI

To cancel a run using the CLI, use the following command. Replace runid with the ID of the run
az ml run cancel -r runid
For more information, see az ml run cancel.

Create child runs
Create child runs to group together related runs, such as for different hyperparameter-tuning iterations.
NOTE
Child runs can only be created using the SDK.
This code example uses the hello_with_children.py script to create a batch of five child runs from within a
submitted run by using the child_run() method:
!more hello_with_children.py
run_config = ScriptRunConfig(source_directory='.', script='hello_with_children.py')
local_script_run.wait_for_completion(show_output=True)
with exp.start_logging() as parent_run:

for c,count in enumerate(range(5)):
with parent_run.child_run() as child:
child.log(name="Hello from child run", value=c)
NOTE
As they move out of scope, child runs are automatically marked as completed.
To create many child runs efficiently, use the create_children() method. Because because each creation results in a
network call, creating a batch of runs is more efficient than creating them one by one.
Submit child runs
Child runs can also be submitted from a parent run. This allows you to create hierarchies of parent and child runs,
each running on different compute targets, connected by common parent run ID.
Use the 'submit_child()' method to submit a child run from within a parent run. To do this in the parent run script,
get the run context and submit the child run using the ''submit_child''' method of the context instance.
## In parent run script

parent_run = Run.get_context()
child_run_config = ScriptRunConfig(source_directory='.', script='child_script.py')
parent_run.submit_child(child_run_config)
Within a child run, you can view the parent run ID:
## In child run script

child_run = Run.get_context()
child_run.parent.id
Query child runs

To query the child runs of a specific parent, use the get_children() method. The '''recursive = True''' argument
allows you to query a nested tree of children and grandchildren.
print(parent_run.get_children())
Tag and find runs
In Azure Machine Learning service, you can use properties and tags to help organize and query your runs for
important information.
Add properties and tags
Using the SDK
To add searchable metadata to your runs, use the add_properties() method. For example, the following code adds
the "author" property to the run:
local_script_run.add_properties({"author":"azureml-user"})
print(local_script_run.get_properties())
Properties are immutable, so they create a permanent record for auditing purposes. The following code example
results in an error, because we already added "azureml-user" as the "author" property value in the preceding
code:
try:
local_script_run.add_properties({"author":"different-user"})
except Exception as e:
print(e)
Unlike properties, tags are mutable. To add searchable and meaningful information for consumers of your
experiment, use the tag() method.
local_script_run.tag("quality", "great run")

print(local_script_run.get_tags())
local_script_run.tag("quality", "fantastic run")

You can also add simple string tags. When these tags appear in the tag dictionary as keys, they have a value of
None .
local_script_run.tag("worth another look")

Using the CLI
NOTE
Using the CLI, you can only add or update tags.
To add or update a tag, use the following command:
az ml run update -r runid --add-tag quality='fantastic run'
For more information, see az ml run update.

Query properties and tags
You can query runs within an experiment to return a list of runs that match specific properties and tags.
Using the SDK
list(exp.get_runs(properties={"author":"azureml-user"},tags={"quality":"fantastic run"}))
list(exp.get_runs(properties={"author":"azureml-user"},tags="worth another look"))
Using the CLI

The Azure CLI supports JMESPath queries, which can be used to filter runs based on properties and tags. To use a
JMESPath query with the Azure CLI, specify it with the --query parameter. The following examples show basic
queries using properties and tags:
# list runs where the author property = 'azureml-user'

az ml run list --experiment-name experiment [?properties.author=='azureml-user']
# list runs where the tag contains a key that starts with 'worth another look'
az ml run list --experiment-name experiment [?tags.keys(@)[?starts_with(@, 'worth another look')]]
# list runs where the author property = 'azureml-user' and the 'quality' tag starts with 'fantastic run'
az ml run list --experiment-name experiment [?properties.author=='azureml-user' && tags.quality=='fantastic
run']
For more information on querying Azure CLI results, see Query Azure CLI command output.
Example notebooks
The following notebooks demonstrate the concepts in this article:
To learn more about the logging APIs, see the logging API notebook.
For more information about managing runs with the Azure Machine Learning SDK, see the manage runs
notebook.
Next steps
To learn how to log metrics for your experiments, see Log metrics during training runs.
Log metrics during training runs
Enhance the model creation process by tracking your experiments and monitoring metrics. In this article, learn
how to add logging to your training script, submit an experiment run, monitor the run, and view the results of a
run in Azure Machine Learning service.
NOTE
Azure Machine Learning service may also log information from other sources during training, such as AutoML or the
Docker container that runs the training job. These logs are not documented. If you encounter problems and contact
Microsoft support, they may be able to use these logs during troubleshooting.
List of training metrics

The following metrics can be added to a run while training an experiment. To view a more detailed list of what
can be tracked on a run, see the Run class reference documentation.
TYPE PYTHON FUNCTION NOTES
Scalar values Function: Log a numerical or string value to the

run.log(name, value, run with the given name. Logging a
description='') metric to a run causes that metric to
be stored in the run record in the
Example: experiment. You can log the same
run.log("accuracy", 0.95) metric multiple times within a run, the
result being considered a vector of that
metric.
Lists Function: Log a list of values to the run with the

run.log_list(name, value, given name.
description='')
Example:
run.log_list("accuracies", [0.6, 0.7, 0.87])
Row Function: Using log_row creates a metric with

run.log_row(name, multiple columns as described in
description=None, **kwargs) kwargs. Each named parameter
Example: generates a column with the value
run.log_row("Y over X", x=1, y=0.4) specified. log_row can be called once to
log an arbitrary tuple, or multiple times
in a loop to generate a complete table.
Table Function: Log a dictionary object to the run with

run.log_table(name, value, the given name.
description='')
Example:
run.log_table("Y over X", {"x":[1, 2, 3],
"y":[0.6, 0.7, 0.89]})
TYPE PYTHON FUNCTION NOTES
Images Function: Log an image to the run record. Use

run.log_image(name, path=None, log_image to log an image file or a
plot=None) matplotlib plot to the run. These
images will be visible and comparable
Example: in the run record.
run.log_image("ROC", plt)
Tag a run Function: Tag the run with a string key and
run.tag(key, value=None) optional string value.
Example:
run.tag("selected", "yes")
Upload file or directory Function: Upload a file to the run record. Runs
run.upload_file(name, automatically capture file in the
path_or_stream) specified output directory, which
defaults to "./outputs" for most run
Example: types. Use upload_file only when
run.upload_file("best_model.pkl", additional files need to be uploaded or
"./model.pkl") an output directory is not specified. We
suggest adding outputs to the name
so that it gets uploaded to the outputs
directory. You can list all of the files
that are associated with this run record
by called run.get_file_names()
NOTE
Metrics for scalars, lists, rows, and tables can have type: float, integer, or string.
Start logging metrics

If you want to track or monitor your experiment, you must add code to start logging when you submit the run.
The following are ways to trigger the run submission:
Run.start_logging - Add logging functions to your training script and start an interactive logging session in
the specified experiment. start_logging creates an interactive run for use in scenarios such as notebooks.
Any metrics that are logged during the session are added to the run record in the experiment.
ScriptRunConfig - Add logging functions to your training script and load the entire script folder with the
run. ScriptRunConfig is a class for setting up configurations for script runs. With this option, you can add
monitoring code to be notified of completion or to get a visual widget to monitor.
Set up the workspace

Before adding logging and submitting an experiment, you must set up the workspace.
1. Load the workspace. To learn more about setting the workspace configuration, see workspace
configuration file.
from azureml.core import Experiment, Run, Workspace

import azureml.core
Option 1: Use start_logging
start_logging creates an interactive run for use in scenarios such as notebooks. Any metrics that are logged
during the session are added to the run record in the experiment.
The following example trains a simple sklearn Ridge model locally in a local Jupyter notebook. To learn more
about submitting experiments to different environments, see Set up compute targets for model training with
1. Create a training script in a local Jupyter notebook.
# load diabetes dataset, a well-known small dataset that comes with scikit-learn
columns = ['age', 'gender', 'bmi', 'bp', 's1', 's2', 's3', 's4', 's5', 's6']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)
data = {
"train":{"X": X_train, "y": y_train},
"test":{"X": X_test, "y": y_test}
}
reg = Ridge(alpha = 0.03)
reg.fit(data['train']['X'], data['train']['y'])
preds = reg.predict(data['test']['X'])
print('Mean Squared Error is', mean_squared_error(preds, data['test']['y']))
joblib.dump(value = reg, filename = 'model.pkl');
2. Add experiment tracking using the Azure Machine Learning service SDK, and upload a persisted model
into the experiment run record. The following code adds tags, logs, and uploads a model file to the
experiment run.
# Get an experiment object from Azure Machine Learning

experiment = Experiment(workspace=ws, name="train-within-notebook")
# Create a run object in the experiment

run = experiment.start_logging()
# Log the algorithm parameter alpha to the run
run.log('alpha', 0.03)
# Create, fit, and test the scikit-learn Ridge regression model

regression_model = Ridge(alpha=0.03)
regression_model.fit(data['train']['X'], data['train']['y'])
preds = regression_model.predict(data['test']['X'])
# Output the Mean Squared Error to the notebook and to the run
print('Mean Squared Error is', mean_squared_error(data['test']['y'], preds))
run.log('mse', mean_squared_error(data['test']['y'], preds))
# Save the model to the outputs directory for capture

model_file_name = 'outputs/model.pkl'
joblib.dump(value = regression_model, filename = model_file_name)
# upload the model file explicitly into artifacts

run.upload_file(name = model_file_name, path_or_stream = model_file_name)
# Complete the run

run.complete()
The script ends with run.complete() , which marks the run as completed. This function is typically used in
interactive notebook scenarios.
Option 2: Use ScriptRunConfig

ScriptRunConfig is a class for setting up configurations for script runs. With this option, you can add
monitoring code to be notified of completion or to get a visual widget to monitor.
This example expands on the basic sklearn Ridge model from above. It does a simple parameter sweep to sweep
over alpha values of the model to capture metrics and trained models in runs under the experiment. The example
runs locally against a user-managed environment.
1. Create a training script train.py .
# train.py
import os
import numpy as np
#os.makedirs('./outputs', exist_ok = True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)

data = {"train": {"X": X_train, "y": y_train},
"test": {"X": X_test, "y": y_test}}
# list of numbers from 0.0 to 1.0 with a 0.05 interval

alphas = mylib.get_alphas()
for alpha in alphas:

# Use Ridge algorithm to create a regression model
reg = Ridge(alpha = alpha)
reg.fit(data["train"]["X"], data["train"]["y"])
preds = reg.predict(data["test"]["X"])
mse = mean_squared_error(preds, data["test"]["y"])
# log the alpha and mse values
run.log('alpha', alpha)
run.log('mse', mse)
model_file_name = 'ridge_{0:.2f}.pkl'.format(alpha)
# save model in the outputs folder so it automatically get uploaded
with open(model_file_name, "wb") as file:
joblib.dump(value = reg, filename = model_file_name)
# upload the model file explicitly into artifacts

run.upload_file(name = model_file_name, path_or_stream = model_file_name)
# register the model

#run.register_model(file_name = model_file_name)
print('alpha is {0:.2f}, and mse is {1:0.2f}'.format(alpha, mse))

2. The train.py script references mylib.py which allows you to get the list of alpha values to use in the
ridge model.
# mylib.py
import numpy as np
def get_alphas():
# list of numbers from 0.0 to 1.0 with a 0.05 interval
return np.arange(0.0, 1.0, 0.05)
3. Configure a user-managed local environment.
# Editing a run configuration property on-fly.

user_managed_env = Environment("user-managed-env")
user_managed_env.python.user_managed_dependencies = True
# You can choose a specific Python environment by pointing to a Python path

#user_managed_env.python.interpreter_path = '/home/johndoe/miniconda3/envs/myenv/bin/python'
4. Submit the train.py script to run in the user-managed environment. This whole script folder is
submitted for training, including the mylib.py file.
exp = Experiment(workspace=ws, name="train-on-local")

src = ScriptRunConfig(source_directory='./', script='train.py')
src.run_config.environment = user_managed_env
Manage a run
The Start, monitor, and cancel training runs article highlights specific Azure Machine Learning workflows for how
to manage your experiments.
View run details

Monitor run with Jupyter notebook widget
When you use the ScriptRunConfig method to submit runs, you can watch the progress of the run with a
Jupyter widget. Like the run submission, the widget is asynchronous and provides live updates every 10-15
seconds until the job completes.
1. View the Jupyter widget while waiting for the run to complete.

You can also get a link to the same display in your workspace.
2. [For automated machine learning runs] To access the charts from a previous run. Replace
<<experiment_name>> with the appropriate experiment name:

experiment = Experiment (workspace, <<experiment_name>>)

run_id = 'autoML_my_runID' #replace with run_ID
run = Run(experiment, run_id)
To view further details of a pipeline click on the Pipeline you would like to explore in the table, and the charts will
render in a pop-up from the Azure portal.
Get log results upon completion
Model training and monitoring occur in the background so that you can run other tasks while you wait. You can
also wait until the model has completed training before running more code. When you use ScriptRunConfig,
you can use run.wait_for_completion(show_output = True) to show when the model training is complete. The
show_output flag gives you verbose output.
Query run metrics

You can view the metrics of a trained model using run.get_metrics() . You can now get all of the metrics that
were logged in the example above to determine the best model.
View the experiment in the Azure portal

When an experiment has finished running, you can browse to the recorded experiment run record. You can
access the history in two ways:
Get the URL to the run directly print(run.get_portal_url())
View the run details by submitting the name of the run (in this case, run ). This way points you to the
experiment name, ID, type, status, details page, a link to the Azure portal, and a link to documentation.
The link for the run brings you directly to the run details page in the Azure portal. Here you can see any
properties, tracked metrics, images, and charts that are logged in the experiment. In this case, we logged MSE
and the alpha values.
You can also view any outputs or logs for the run, or download the snapshot of the experiment you submitted so
you can share the experiment folder with others.
Viewing charts in run details
There are various ways to use the logging APIs to record different types of metrics during a run and view them
as charts in the Azure portal.
LOGGED VALUE EXAMPLE CODE VIEW IN PORTAL
Log an array of numeric values run.log_list(name='Fibonacci', single-variable line chart

value=[0, 1, 1, 2, 3, 5, 8, 13,
21, 34, 55, 89])
Log a single numeric value with the for i in tqdm(range(-10, 10)): Single-variable line chart
same metric name repeatedly used run.log(name='Sigmoid', value=1
/ (1 + np.exp(-i))) angle = i /
(like from within a for loop) 2.0
Log a row with 2 numerical columns run.log_row(name='Cosine Wave', Two-variable line chart
repeatedly angle=angle, cos=np.cos(angle))
sines['angle'].append(angle)
sines['sine'].append(np.sin(angle))
Log table with 2 numerical columns run.log_table(name='Sine Wave', Two-variable line chart
value=sines)
Example notebooks
The following notebooks demonstrate concepts in this article:
how -to-use-azureml/training/train-within-notebook
how -to-use-azureml/training/train-on-local
how -to-use-azureml/training/logging-api/logging-api.ipynb
Next steps
Try these next steps to learn how to use the Azure Machine Learning SDK for Python:
See an example of how to register the best model and deploy it in the tutorial, Train an image
classification model with Azure Machine Learning.
Learn how to Train PyTorch Models with Azure Machine Learning.
Track metrics and deploy models with MLflow and
Azure Machine Learning service (Preview)
This article demonstrates how to enable MLflow's tracking URI and logging API, collectively known as MLflow
Tracking, with Azure Machine Learning service. Doing so enables you to:
Track and log your experiment metrics and artifacts in your Azure Machine Learning service workspace. If
you already use MLflow Tracking for your experiments, the workspace provides a centralized, secure, and
scalable location to store your training metrics and models.
Deploy your MLflow experiments as an Azure Machine Learning web service. By deploying as a web
service, you can apply the Azure Machine Learning monitoring and data drift detection functionalities to
your production models.
MLflow is an open-source library for managing the life cycle of your machine learning experiments. MLFlow
Tracking is a component of MLflow that logs and tracks your training run metrics and model artifacts, no matter
your experiment's environment--locally, on a virtual machine, remote compute cluster, even on Azure Databricks.
The following diagram illustrates that with MLflow Tracking, you can take any experiment--whether it's on a
remote compute target on a virtual machine, locally on your computer, or on an Azure Databricks cluster--and
track its run metrics and store model artifacts in your Azure Machine Learning workspace.
Compare MLflow and Azure Machine Learning clients

The below table summarizes the different clients that can use Azure Machine Learning service, and their respective
function capabilities.
MLflow Tracking offers metric logging and artifact storage functionalities that are only otherwise available via the
Azure Machine Learning Python SDK.
AZURE MACHINE
MLFLOW TRACKING & LEARNING PYTHON AZURE MACHINE
DEPLOYMENT SDK LEARNING CLI AZURE PORTAL
Manage workspace ✓ ✓ ✓
Use data stores ✓ ✓
Log metrics ✓ ✓
Upload artifacts ✓ ✓
View metrics ✓ ✓ ✓ ✓
Manage compute ✓ ✓ ✓
Deploy models ✓ ✓ ✓ ✓
Monitor model ✓
performance
Detect data drift ✓ ✓
Prerequisites
Install MLflow.
Install the Azure Machine Learning SDK on your local computer The SDK provides the connectivity for MLflow
to access your workspace.
Create an Azure Machine Learning Workspace.
Track local runs

MLflow Tracking with Azure Machine Learning service lets you store the logged metrics and artifacts from your
local runs into your Azure Machine Learning workspace.
Install the azureml-contrib-run package to use MLflow Tracking with Azure Machine Learning on your
experiments locally run in a Jupyter Notebook or code editor.
pip install azureml-contrib-run
NOTE
The azureml.contrib namespace changes frequently, as we work to improve the service. As such, anything in this namespace
should be considered as a preview, and not fully supported by Microsoft.
Import the mlflow and Workspace classes to access MLflow's tracking URI and configure your workspace.
In the following code, the get_mlflow_tracking_uri() method assigns a unique tracking URI address to the
workspace, ws , and set_tracking_uri() points the MLflow tracking URI to that address.
import mlflow
mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri())
NOTE
The tracking URI is valid up to an hour or less. If you restart your script after some idle time, use the get_mlflow_tracking_uri
API to get a new URI.
Set the MLflow experiment name with set_experiment() and start your training run with start_run() . Then use
log_metric() to activate the MLflow logging API and begin logging your training run metrics.
experiment_name = 'experiment_with_mlflow'
mlflow.set_experiment(experiment_name)
with mlflow.start_run():
mlflow.log_metric('alpha', 0.03)
Track remote runs

remote runs into your Azure Machine Learning workspace.
Remote runs let you train your models on more powerful computes, such as GPU enabled virtual machines, or
Machine Learning Compute clusters. See Set up compute targets for model training to learn about different
compute options.
Configure your compute and training run environment with the Environment class. Include mlflow and
azure-contrib-run pip packages in environment's CondaDependencies section. Then construct ScriptRunConfig
with your remote compute as the compute target.

exp = Experiment(workspace = 'my_workspace',

name='my_experiment')
mlflow_env = Environment(name='mlflow-env')
cd = CondaDependencies.create(pip_packages=['mlflow', 'azureml-contrib-run'])
mlflow_env.python.conda_dependencies = cd
src = ScriptRunConfig(source_directory='./my_script_location', script='my_training_script.py')
src.run_config.target = 'my-remote-compute-compute'
src.run_config.environment = mlflow_env
In your training script, import mlflow to use the MLflow logging APIs, and start logging your run metrics.
import mlflow
with mlflow.start_run():
mlflow.log_metric('example', 1.23)
With this compute and training run configuration, use the Experiment.submit('train.py') method to submit a run.
This automatically sets the MLflow tracking URI and directs the logging from MLflow to your Workspace.
Track Azure Databricks runs

Databricks runs in your Azure Machine Learning workspace.
To run your Mlflow experiments with Azure Databricks, you need to first create an Azure Databricks workspace and
cluster
In your cluster, be sure to install the azureml-mlflow library from PyPi, to ensure that your cluster has access to the
necessary functions and classes.
Install libraries
To install libraries on your cluster, navigate to the Libraries tab and click Install New
In the Package field, type azureml-mlflow and then click install. Repeat this step as necessary to install other
additional packages to your cluster for your experiment.
Set up your notebook and workspace
Once your cluster is set up, import your experiment notebook, open it and attach your cluster to it.
The following code should be in your experiment notebook. This gets the details of your Azure subscription to
instantiate your workspace. This assumes you have an existing resource group and Azure Machine Learning
workspace, otherwise you can create them.
import mlflow
import mlflow.azureml
import azureml.mlflow
import azureml.core

from azureml.mlflow import get_portal_url
subscription_id = 'subscription_id'
# Azure Machine Learning resource group NOT the managed resource group
resource_group = 'resource_group_name'
#Azure Machine Learning workspace name, NOT Azure Databricks workspace

workspace_name = 'workspace_name'
# Instantiate Azure Machine Learning workspace

ws = Workspace.get(name=workspace_name,
subscription_id=subscription_id,
resource_group=resource_group)
Link MLflow tracking to your workspace

After you instantiate your workspace, set the MLflow tracking URI. By doing so, you link the MLflow tracking to
Azure Machine Learning workspace. After this, all your experiments will land in the managed Azure Machine
Learning tracking service.
uri = ws.get_mlflow_tracking_uri()
mlflow.set_tracking_uri(uri)
In your training script, import mlflow to use the MLflow logging APIs, and start logging your run metrics. The
following example, logs the epoch loss metric.
import mlflow
mlflow.log_metric('epoch_loss', loss.item())
View metrics and artifacts in your workspace

The metrics and artifacts from MLflow logging are kept in your workspace. To view them anytime, navigate to your
workspace and find the experiment by name on the Azure portal or by running the below code.
run.get_metrics()
ws.get_details()
Deploy MLflow models as a web service

Deploying your MLflow experiments as an Azure Machine Learning web service allows you to leverage the Azure
Machine Learning model management and data drift detection capabilities and apply them to your production
models.
The following diagram demonstrates that with the MLflow deploy API you can deploy your existing MLflow
models as an Azure Machine Learning web service, despite their frameworks--PyTorch, Tensorflow, scikit-learn,
ONNX, etc., and manage your production models in your workspace.
Log your model

Before you can deploy, be sure that your model is saved so you can reference it and its path location for
deployment. In your training script, there should be code similar to the following mlflow.sklearn.log_model()
method, that saves your model to the specified outputs directory.
# change sklearn to pytorch, tensorflow, etc. based on your experiment's framework
import mlflow.sklearn
# Save the model to the outputs directory for capture

mlflow.sklearn.log_model(regression_model, model_save_path)
NOTE
Include the conda_env parameter to pass a dictionary representation of the dependencies and environment this model
should be run in.
Retrieve model from previous run

To retrieve the desired run you need the run ID and the path in run history of where the model was saved.
# gets the list of runs for your experiment as an array

experiment_name = 'experiment-with-mlflow'
exp = ws.experiments[experiment_name]
runs = list(exp.get_runs())
# get the run ID and the path in run history

runid = runs[0].id
model_save_path = 'model'
Create Docker image

The mlflow.azureml.build_image() function builds a Docker image from the saved model in a framework-aware
manner. It automatically creates the framework-specific inferencing wrapper code and specifies package
dependencies for you. Specify the model path, your workspace, run ID and other parameters.
The following code builds a docker image using runs:/<run.id>/model as the model_uri path for a Scikit-learn
experiment.
import mlflow.azureml
azure_image, azure_model = mlflow.azureml.build_image(model_uri='runs:/{}/{}'.format(runid, model_save_path),

workspace=ws,
model_name='sklearn-model',
image_name='sklearn-image',
synchronous=True)
The creation of the Docker image can take several minutes.

Deploy the Docker image
After the image is created, use the Azure Machine Learning SDK to deploy the image as a web service.
First, specify the deployment configuration. Azure Container Instance (ACI) is a suitable choice for a quick dev-test
deployment, while Azure Kubernetes Service (AKS ) is suitable for scalable production deployments.
Deploy to ACI
Set up your deployment configuration with the deploy_configuration() method. You can also add tags and
descriptions to help keep track of your web service.
# Configure
aci_config = AciWebservice.deploy_configuration(cpu_cores=1,
memory_gb=1,
tags={'method' : 'sklearn'},
description='Diabetes model',
location='eastus2')
Then, deploy the image using Azure Machine Learning SDK's deploy_from_image() method.
webservice = Webservice.deploy_from_image( image=azure_image,

workspace=ws,
name='diabetes-model-1',
deployment_config=aci_config)
webservice.wait_for_deployment(show_output=True)
Deploy to AKS
To deploy to AKS you need to create an AKS cluster and bring over the Docker image you want to deploy. For this
example, bring over the previously created image from the ACI deployment.
To get the image from the previous ACI deployment use the Image class.
from azureml.core.image import Image
# Get the image by name, you can change this based on the image you want to deploy
myimage = Image(workspace=ws, name='sklearn-image')
Create AKS compute it may take 20-25 minutes to create a new cluster
from azureml.core.compute import AksCompute, ComputeTarget
# Use the default configuration (can also provide parameters to customize)

prov_config = AksCompute.provisioning_configuration()
aks_name = 'aks-mlflow'
# Create the cluster

name=aks_name,
provisioning_configuration=prov_config)
aks_target.wait_for_completion(show_output = True)
print(aks_target.provisioning_state)
print(aks_target.provisioning_errors)
Set up your deployment configuration with the deploy_configuration() method. You can also add tags and
descriptions to help keep track of your web service.
from azureml.core.webservice import Webservice, AksWebservice
from azureml.core.image import ContainerImage
# Set the web service configuration (using default here with app insights)
aks_config = AksWebservice.deploy_configuration(enable_app_insights=True)
# Unique service name

service_name ='aks-service'
Then, deploy the image using Azure Machine Learning SDK's deploy_from_image() method.
# Webservice creation using single command

aks_service = Webservice.deploy_from_image( workspace=ws,
name=service_name,
deployment_config = aks_config
image = myimage,
deployment_target = aks_target)
aks_service.wait_for_deployment(show_output=True)
The service deployment can take several minutes.
Clean up resources
If you don't plan to use the logged metrics and artifacts in your workspace, the ability to delete them individually is
currently unavailable. Instead, delete the resource group that contains the storage account and workspace, so you
don't incur any charges:

Example notebooks
The MLflow with Azure ML notebooks demonstrate and expand upon concepts presented in this article.
Next steps
Manage your models.
Monitor your production models for data drift.
Visualize experiment runs and metrics with
TensorBoard and Azure Machine Learning
In this article, you learn how to view your experiment runs and metrics in TensorBoard using the tensorboard
package in the main Azure Machine Learning service SDK. Once you've inspected your experiment runs, you can
better tune and retrain your machine learning models.
TensorBoard is a suite of web applications for inspecting and understanding your experiment structure and
performance.
How you launch TensorBoard with Azure Machine Learning experiments depends on the type of experiment:
If your experiment natively outputs log files that are consumable by TensorBoard, such as PyTorch, Chainer
and TensorFlow experiments, then you can launch TensorBoard directly from experiment's run history.
For experiments that don't natively output TensorBoard consumable files, such as like Scikit-learn or Azure
Machine Learning experiments, use the export_to_tensorboard() method to export the run histories as
TensorBoard logs and launch TensorBoard from there.
Prerequisites
To launch TensorBoard and view your experiment run histories, your experiments need to have previously
enabled logging to track its metrics and performance.
The code in this how -to can be run in either of the following environments:
Complete the Tutorial: Setup environment and workspace to create a dedicated notebook
server pre-loaded with the SDK and the sample repository.
In the samples folder on the notebook server, find two completed and expanded notebooks by
navigating to this directory: how-to-use-azureml > training-with-deep-learning.
export-run-history-to-run-history.ipynb
tensorboard.ipynb
Your own Juptyer notebook server * Install the Azure Machine Learning SDK with the tensorboard
extra * Create an Azure Machine Learning service workspace.
* Create a workspace configuration file.
Option 1: Directly view run history in TensorBoard

This option works for experiments that natively outputs log files consumable by TensorBoard, such as PyTorch,
Chainer and TensorFlow experiments. If that is not the case of your experiment, use the export_to_tensorboard()
method instead.
The following example code uses the MNIST demo experiment from TensorFlow's repository in a remote compute
target, Azure Machine Learning Compute. Next, we train our model with the SDK's custom TensorFlow estimator,
and then start TensorBoard against this TensorFlow experiment, that is, an experiment that natively outputs
TensorBoard event files.
Set experiment name and create project folder
Here we name the experiment and create its folder.
from os import path, makedirs

experiment_name = 'tensorboard-demo'
# experiment folder
exp_dir = './sample_projects/' + experiment_name
if not path.exists(exp_dir):
makedirs(exp_dir)
Download TensorFlow demo experiment code

TensorFlow's repository has an MNIST demo with extensive TensorBoard instrumentation. We do not, nor need to,
alter any of this demo's code for it to work with the Azure Machine Learning service. In the following code, we
download the MNIST code and save it in our newly created experiment folder.
import requests
import os
tf_code =
requests.get("https://raw.githubusercontent.com/tensorflow/tensorflow/r1.8/tensorflow/examples/tutorials/mnist/
mnist_with_summaries.py")
with open(os.path.join(exp_dir, "mnist_with_summaries.py"), "w") as file:
file.write(tf_code.text)
Throughout the MNIST code file, mnist_with_summaries.py, notice that there are lines that call
tf.summary.scalar() , tf.summary.histogram() , tf.summary.FileWriter() etc. These methods group, log, and tag key
metrics of your experiments into run history. The tf.summary.FileWriter() is especially important as it serializes
the data from your logged experiment metrics, which allows for TensorBoard to generate visualizations off of them.
In the following, we configure our experiment and set up directories for logs and data. These logs will be uploaded
to the Artifact Service, which TensorBoard accesses later.
NOTE
For this TensorFlow example, you will need to install TensorFlow on your local machine. Further, the TensorBoard module (that
is, the one included with TensorFlow) must be accessible to this notebook's kernel, as the local machine is what runs
TensorBoard.
import azureml.core
# create directories for experiment logs and dataset

logs_dir = os.path.join(os.curdir, "logs")
data_dir = os.path.abspath(os.path.join(os.curdir, "mnist_data"))
if not path.exists(data_dir):
makedirs(data_dir)
os.environ["TEST_TMPDIR"] = data_dir
# Writing logs to ./logs results in their being uploaded to Artifact Service,

# and thus, made accessible to our TensorBoard instance.
arguments_list = ["--log_dir", logs_dir]
# Create an experiment
exp = Experiment(ws, experiment_name)
Create a cluster for your experiment

We create an AmlCompute cluster for this experiment, however your experiments can be created in any
environment and you are still able to launch TensorBoard against the experiment run history.
cluster_name = "cpucluster"
cts = ws.compute_targets
found = False
if cluster_name in cts and cts[cluster_name].type == 'AmlCompute':
found = True
print('Found existing compute target.')
compute_target = cts[cluster_name]
if not found:
max_nodes=4)

compute_target.wait_for_completion(show_output=True, min_node_count=None)
# use get_status() to get a detailed status for the current cluster.

# print(compute_target.get_status().serialize())
Submit run with TensorFlow estimator

The TensorFlow estimator provides a simple way of launching a TensorFlow training job on a compute target. It's
implemented through the generic estimator class, which can be used to support any framework. For more
information about training models using the generic estimator, see train models with Azure Machine Learning
using estimator
script_params = {"--log_dir": "./logs"}
tf_estimator = TensorFlow(source_directory=exp_dir,
entry_script='mnist_with_summaries.py',
script_params=script_params)
run = exp.submit(tf_estimator)
Launch TensorBoard
You can launch TensorBoard during your run or after it completes. In the following, we create a TensorBoard object
instance, tb , that takes the experiment run history loaded in the run , and then launches TensorBoard with the
start() method.
The TensorBoard constructor takes an array of runs, so be sure and pass it in as a single-element array.
from azureml.tensorboard import Tensorboard
tb = Tensorboard([run])
# If successful, start() returns a string with the URI of the instance.

tb.start()
# After your job completes, be sure to stop() the streaming otherwise it will continue to run.
tb.stop()
Option 2: Export history as log to view in TensorBoard

The following code sets up a sample experiment, begins the logging process using the Azure Machine Learning run
history APIs, and exports the experiment run history into logs consumable by TensorBoard for visualization.
Set up experiment
The following code sets up a new experiment and names the run directory root_run .
from azureml.core import Workspace, Experiment

import azuremml.core
# set experiment name and run name

experiment_name = 'export-to-tensorboard'
exp = Experiment(ws, experiment_name)
root_run = exp.start_logging()
Here we load the diabetes dataset-- a built-in small dataset that comes with scikit-learn, and split it into test and
training sets.
X, y = load_diabetes(return_X_y=True)
columns = ['age', 'gender', 'bmi', 'bp', 's1', 's2', 's3', 's4', 's5', 's6']
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
data = {
"train":{"x":x_train, "y":y_train},
"test":{"x":x_test, "y":y_test}
}
Run experiment and log metrics

For this code, we train a linear regression model and log key metrics, the alpha coefficient, alpha and mean
squared error, mse , in run history.
from tqdm import tqdm

alphas = [.1, .2, .3, .4, .5, .6 , .7]
# try a bunch of alpha values in a Linear Regression (aka Ridge regression) mode
for alpha in tqdm(alphas):
# create child runs and fit lines for the resulting models
with root_run.child_run("alpha" + str(alpha)) as run
reg = Ridge(alpha=alpha)
reg.fit(data["train"]["x"], data["train"]["y"])
preds = reg.predict(data["test"]["x"])
mse = mean_squared_error(preds, data["test"]["y"])
# End train and eval
# log alpha, mean_squared_error and feature names in run history

root_run.log("alpha", alpha)
root_run.log("mse", mse)
Export runs to TensorBoard

With the SDK's export_to_tensorboard() method, we can export the run history of our Azure machine learning
experiment into TensorBoard logs, so we can view them via TensorBoard.
In the following code, we create the folder logdir in our current working directory. This folder is where we will
export our experiment run history and logs from root_run and then mark that run as completed.
from azureml.tensorboard.export import export_to_tensorboard

import os
logdir = 'exportedTBlogs'
log_path = os.path.join(os.getcwd(), logdir)
try:
os.stat(log_path)
except os.error:
os.mkdir(log_path)
print(logdir)
# export run history for the project

export_to_tensorboard(root_run, logdir)
root_run.complete()
NOTE
You can also export a particular run to TensorBoard by specifying the name of the run
export_to_tensorboard(run_name, logdir)
Start and stop TensorBoard Once our run history for this experiment is exported, we can launch TensorBoard with
the start() method.
from azureml.tensorboard import Tensorboard
# The TensorBoard constructor takes an array of runs, so be sure and pass it in as a single-element array here
tb = Tensorboard([], local_root=logdir, port=6006)
# If successful, start() returns a string with the URI of the instance.

tb.start()
When you're done, make sure to call the stop() method of the TensorBoard object. Otherwise, TensorBoard will
continue to run until you shut down the notebook kernel.
tb.stop()
Next steps
In this how -to you, created two experiments and learned how to launch TensorBoard against their run histories to
identify areas for potential tuning and retraining.
If you are satisfied with your model, head over to our How to deploy a model article.
Learn more about hyperparameter tuning.
Configure automated ML experiments in Python
In this guide, learn how to define various configuration settings of your automated machine learning experiments
with the Azure Machine Learning SDK. Automated machine learning picks an algorithm and hyperparameters for
you and generates a model ready for deployment. There are several options that you can use to configure
automated machine learning experiments.
To view examples of an automated machine learning experiments , see Tutorial: Train a classification model with
automated machine learning or Train models with automated machine learning in the cloud.
Configuration options available in automated machine learning:
Select your experiment type: Classification, Regression or Time Series Forecasting
Data source, formats, and fetch data
Choose your compute target: local or remote
Automated machine learning experiment settings
Run an automated machine learning experiment
Explore model metrics
Register and deploy model
If you prefer a no code experience, you can also Create your automated machine learning experiments in the
Azure portal.
Select your experiment type

Before you begin your experiment, you should determine the kind of machine learning problem you are solving.
Automated machine learning supports task types of classification, regression and forecasting.
Automated machine learning supports the following algorithms during the automation and tuning process. As a
user, there is no need for you to specify the algorithm.
CLASSIFICATION REGRESSION TIME SERIES FORECASTING
Logistic Regression Elastic Net Elastic Net
Light GBM Light GBM Light GBM
Gradient Boosting Gradient Boosting Gradient Boosting
Decision Tree Decision Tree Decision Tree
K Nearest Neighbors K Nearest Neighbors K Nearest Neighbors
Linear SVC LARS Lasso LARS Lasso
C-Support Vector Classification (SVC) Stochastic Gradient Descent (SGD) Stochastic Gradient Descent (SGD)
Random Forest Random Forest Random Forest

Extremely Randomized Trees Extremely Randomized Trees Extremely Randomized Trees
Xgboost Xgboost Xgboost
DNN Classifier DNN Regressor DNN Regressor
DNN Linear Classifier Linear Regressor Linear Regressor
Naive Bayes
Stochastic Gradient Descent (SGD)
Use the task parameter in the AutoMLConfig constructor to specify your experiment type.
# task can be one of classification, regression, forecasting

automl_config = AutoMLConfig(task="classification")
Data source and format

Automated machine learning supports data that resides on your local desktop or in the cloud such as Azure Blob
Storage. The data can be read into scikit-learn supported data formats. You can read the data into:
Numpy arrays X (features) and y (target variable or also known as label)
Pandas dataframe
IMPORTANT
Requirements for training data:
Data must be in tabular form.
The value you want to predict (target column) must be present in the data.
Examples:
Numpy arrays
digits = datasets.load_digits()
X_digits = digits.data
y_digits = digits.target
Pandas dataframe
import pandas as pd
df = pd.read_csv("https://automldemods.blob.core.windows.net/datasets/PlayaEvents2016,_1.6MB,_3.4k-
rows.cleaned.2.tsv", delimiter="\t", quotechar='"')
# get integer labels
y = df["Label"]
df = df.drop(["Label"], axis=1)
df_train, _, y_train, _ = train_test_split(df, y, test_size=0.1, random_state=42)
Fetch data for running experiment on remote compute
For remote executions, you need to make the data accessible from the remote compute. This can be done by
uploading the data to DataStore.
Here is an example of using datastore :
import pandas as pd
from sklearn import datasets
data_train = datasets.load_digits()
pd.DataFrame(data_train.data[100:,:]).to_csv("data/X_train.csv", index=False)
pd.DataFrame(data_train.target[100:]).to_csv("data/y_train.csv", index=False)
ds.upload(src_dir='./data', target_path='digitsdata', overwrite=True, show_progress=True)
Define dprep references

Define X and y as dprep reference, which will be passed to automated machine learning AutoMLConfig object
similar to below:
X = dprep.auto_read_file(path=ds.path('digitsdata/X_train.csv'))
y = dprep.auto_read_file(path=ds.path('digitsdata/y_train.csv'))
automl_config = AutoMLConfig(task = 'classification',

debug_log = 'automl_errors.log',
path = project_folder,
run_configuration=conda_run_config,
X = X,
y = y,
**automl_settings
)
Train and validation data

You can specify separate train and validation set directly in the AutoMLConfig method.
K -Folds Cross Validation
Use n_cross_validations setting to specify the number of cross validations. The training data set will be randomly
split into n_cross_validations folds of equal size. During each cross validation round, one of the folds will be used
for validation of the model trained on the remaining folds. This process repeats for n_cross_validations rounds
until each fold is used once as validation set. The average scores across all n_cross_validations rounds will be
reported, and the corresponding model will be retrained on the whole training data set.
Monte Carlo Cross Validation (Repeated Random Sub-Sampling)
Use validation_size to specify the percentage of the training dataset that should be used for validation, and use
n_cross_validations to specify the number of cross validations. During each cross validation round, a subset of
size validation_size will be randomly selected for validation of the model trained on the remaining data. Finally,
the average scores across all n_cross_validations rounds will be reported, and the corresponding model will be
retrained on the whole training data set. Monte Carlo is not supported for time series forecasting.
Custom validation dataset
Use custom validation dataset if random split is not acceptable, usually time series data or imbalanced data. You
can specify your own validation dataset. The model will be evaluated against the validation dataset specified
instead of random dataset.
Compute to run experiment

Next determine where the model will be trained. An automated machine learning training experiment can run on
the following compute options:
Your local machine such as a local desktop or laptop – Generally when you have small dataset and you are still
in the exploration stage.
A remote machine in the cloud – Azure Machine Learning Managed Compute is a managed service that
enables the ability to train machine learning models on clusters of Azure virtual machines.
See the GitHub site for example notebooks with local and remote compute targets.
An Azure Databricks cluster in your Azure subscription. You can find more details here - Setup Azure
Databricks cluster for Automated ML
See the GitHub site for example notebooks with Azure Databricks.
Configure your experiment settings

There are several options that you can use to configure your automated machine learning experiment. These
parameters are set by instantiating an AutoMLConfig object. See the AutoMLConfig class for a full list of
parameters.
Some examples include:
1. Classification experiment using AUC weighted as the primary metric with a max time of 12,000 seconds
per iteration, with the experiment to end after 50 iterations and 2 cross validation folds.
automl_classifier = AutoMLConfig(
task='classification',
primary_metric='AUC_weighted',
max_time_sec=12000,
iterations=50,
blacklist_models='XGBoostClassifier',
X=X,
y=y,
n_cross_validations=2)
2. Below is an example of a regression experiment set to end after 100 iterations, with each iteration lasting
up to 600 seconds with 5 validation cross folds.
automl_regressor = AutoMLConfig(
task='regression',
max_time_sec=600,
iterations=100,
whitelist_models='kNN regressor'
primary_metric='r2_score',
X=X,
y=y,
n_cross_validations=5)
The three different task parameter values determine the list of algorithms to apply. Use the whitelist or
blacklist parameters to further modify iterations with the available algorithms to include or exclude. The list of
supported models can be found on SupportedAlgorithms Class.
Primary Metric
The primary metric; as shown in the examples above determines the metric to be used during model training for
optimization. The primary metric you can select is determined by the task type you choose. Below is a list of
available metrics.
Learn about the specific definitions of these in Understand automated machine learning results.
accuracy spearman_correlation spearman_correlation
AUC_weighted normalized_root_mean_squared_error normalized_root_mean_squared_error
average_precision_score_weighted r2_score r2_score
norm_macro_recall normalized_mean_absolute_error normalized_mean_absolute_error
precision_score_weighted
Data preprocessing & featurization

In every automated machine learning experiment, your data is automatically scaled and normalized to help
algorithms perform well. However, you can also enable additional preprocessing/featurization, such as missing
values imputation, encoding, and transforms. Learn more about what featurization is included.
To enable this featurization, specify "preprocess": True for the AutoMLConfig class.
NOTE
Time Series Forecasting

For time series forecasting task type you have additional parameters to define.
1. time_column_name - This is a required parameter which defines the name of the column in your training data
containing date/time series.
2. max_horizon - This defines the length of time you want to predict out based on the periodicity of the training
data. For example if you have training data with daily time grains, you define how far out in days you want the
model to train for.
3. grain_column_names - This defines the name of columns which contain individual time series data in your
training data. For example, if you are forecasting sales of a particular brand by store, you would define store
and brand columns as your grain columns.
See example of these settings being used below, notebook example is available here.
# Setting Store and Brand as grains for training.

grain_column_names = ['Store', 'Brand']
nseries = data.groupby(grain_column_names).ngroups
# View the number of time series data with defined grains

print('Data contains {0} individual time-series.'.format(nseries))
time_series_settings = {
'time_column_name': time_column_name,
'grain_column_names': grain_column_names,
'drop_column_names': ['logQuantity'],
'max_horizon': n_test_periods
}
automl_config = AutoMLConfig(task='forecasting',
debug_log='automl_oj_sales_errors.log',
primary_metric='normalized_root_mean_squared_error',
iterations=10,
X=X_train,
y=y_train,
n_cross_validations=5,
path=project_folder,
**time_series_settings)
Ensemble configuration
Ensemble models are enabled by default, and appear as the final run iterations in an automated machine learning
run. Currently supported ensemble methods are voting and stacking. Voting is implemented as soft-voting using
weighted averages, and the stacking implementation is using a 2 layer implementation, where the first layer has
the same models as the voting ensemble, and the second layer model is used to find the optimal combination of
the models from the first layer. If you are using ONNX models, or have model-explainability enabled, stacking will
be disabled and only voting will be utilized.
There are multiple default arguments that can be provided as kwargs in an AutoMLConfig object to alter the
default stack ensemble behavior.
stack_meta_learner_type : the meta-learner is a model trained on the output of the individual heterogenous
models. Default meta-learners are LogisticRegression for classification tasks (or LogisticRegressionCV if
cross-validation is enabled) and ElasticNet for regression/forecasting tasks (or ElasticNetCV if cross-
validation is enabled). This parameter can be one of the following strings: LogisticRegression ,
LogisticRegressionCV , LightGBMClassifier , ElasticNet , ElasticNetCV , LightGBMRegressor , or
LinearRegression .
stack_meta_learner_train_percentage : specifies the proportion of the training set (when choosing train and
validation type of training) to be reserved for training the meta-learner. Default value is 0.2 .
stack_meta_learner_kwargs : optional parameters to pass to the initializer of the meta-learner. These parameters
and parameter types mirror those from the corresponding model constructor, and are forwarded to the model
constructor.
The following code shows an example of specifying custom ensemble behavior in an AutoMLConfig object.
ensemble_settings = {
"stack_meta_learner_type": "LogisticRegressionCV",
"stack_meta_learner_train_percentage": 0.3,
"stack_meta_learner_kwargs": {
"refit": True,
"fit_intercept": False,
"class_weight": "balanced",
"multi_class": "auto",
"n_jobs": -1
}
}
iterations=20,
X=X_train,
y=y_train,
**ensemble_settings
)
Ensemble training is enabled by default, but it can be disabled by using the enable_voting_ensemble and
enable_stack_ensemble boolean parameters.
iterations=20,
X=X_train,
y=y_train,
enable_voting_ensemble=False,
enable_stack_ensemble=False
)
Run experiment
For automated ML you create an Experiment object, which is a named object in a Workspace used to run
experiments.
# Choose a name for the experiment and specify the project folder.
experiment_name = 'automl-classification'
project_folder = './sample_projects/automl-classification'
experiment = Experiment(ws, experiment_name)
Submit the experiment to run and generate a model. Pass the AutoMLConfig to the submit method to generate
the model.
run = experiment.submit(automl_config, show_output=True)

NOTE
Dependencies are first installed on a new machine. It may take up to 10 minutes before output is shown. Setting
show_output to True results in output being shown on the console.
Exit Criteria
There a few options you can define to complete your experiment.
1. No Criteria - If you do not define any exit parameters the experiment will continue until no further progress is
made on your primary metric.
2. Number of iterations - You define the number of iterations for the experiment to run. You can optional add
iteration_timeout_minutes to define a time limit in minutes per each iteration.
3. Exit after a length of time - Using experiment_timeout_minutes in your settings you can define how long in
minutes should an experiment continue in run.
4. Exit after a score has been reached - Using experiment_exit_score you can choose to complete the experiment
after a score based on your primary metric has been reached.
Explore model metrics
You can view your training results in a widget or inline if you are in a notebook. See Track and evaluate models for
more details.
Understand automated ML models

Any model produced using automated ML includes the following steps:
Automated feature engineering (if preprocess=True)
Scaling/Normalization and algorithm with hypermeter values
We make it transparent to get this information from the fitted_model output from automated ML.
automl_config = AutoMLConfig(…)
automl_run = experiment.submit(automl_config …)
best_run, fitted_model = automl_run.get_output()
Automated feature engineering

See the list of preprocessing and automated feature engineering that happens when preprocess=True.
Consider this example:
There are 4 input features: A (Numeric), B (Numeric), C (Numeric), D (DateTime)
Numeric feature C is dropped because it is an ID column with all unique values
Numeric features A and B have missing values and hence are imputed by mean
DateTime feature D is featurized into 11 different engineered features
Use these 2 APIs on the first step of fitted model to understand more. See this sample notebook.
API 1: get_engineered_feature_names() returns a list of engineered feature names.
Usage:
fitted_model.named_steps['timeseriestransformer']. get_engineered_feature_names ()
Output: ['A', 'B', 'A_WASNULL', 'B_WASNULL', 'year', 'half', 'quarter', 'month', 'day', 'hour',
'am_pm', 'hour12', 'wday', 'qday', 'week']
This list includes all engineered feature names.
NOTE
Use 'timeseriestransformer' for task=’forecasting’, else use 'datatransformer' for ‘regression’ or ‘classification’ task.
API 2: get_featurization_summary() returns featurization summary for all the input features.
Usage:
fitted_model.named_steps['timeseriestransformer'].get_featurization_summary()
NOTE
Use 'timeseriestransformer' for task=’forecasting’, else use 'datatransformer' for ‘regression’ or ‘classification’ task.
Output:
[{'RawFeatureName': 'A',
'TypeDetected': 'Numeric',
'Dropped': 'No',
'EngineeredFeatureCount': 2,
'Tranformations': ['MeanImputer', 'ImputationMarker']},
{'RawFeatureName': 'B',
'Dropped': 'No',
'Tranformations': ['MeanImputer', 'ImputationMarker']},
{'RawFeatureName': 'C',
'Dropped': 'Yes',
'Tranformations': []},
{'RawFeatureName': 'D',
'TypeDetected': 'DateTime',
'Dropped': 'No',
'Tranformations':
['DateTime','DateTime','DateTime','DateTime','DateTime','DateTime','DateTime','DateTime','DateTime','Da
teTime','DateTime']}]
Where:
OUTPUT DEFINITION
RawFeatureName Input feature/column name from the dataset provided.
TypeDetected Detected datatype of the input feature.
Dropped Indicates if the input feature was dropped or used.

OUTPUT DEFINITION
EngineeringFeatureCount Number of features generated through automated

feature engineering transforms.
Transformations List of transformations applied to input features to

generate engineered features.
Scaling/Normalization and algorithm with hypermeter values:

To understand the scaling/normalization and algorithm/hyperparameter values for a pipeline, use
fitted_model.steps. Learn more about scaling/normalization. Here is a sample output:
[('RobustScaler', RobustScaler(copy=True, quantile_range=[10, 90], with_centering=True, with_scaling=True)),

('LogisticRegression', LogisticRegression(C=0.18420699693267145, class_weight='balanced', dual=False,
fit_intercept=True, intercept_scaling=1, max_iter=100, multi_class='multinomial', n_jobs=1, penalty='l2',
random_state=None, solver='newton-cg', tol=0.0001, verbose=0, warm_start=False))
To get more details, use this helper function shown in this sample notebook.
from pprint import pprint
def print_model(model, prefix=""):

for step in model.steps:
print(prefix + step[0])
if hasattr(step[1], 'estimators') and hasattr(step[1], 'weights'):
pprint({'estimators': list(
e[0] for e in step[1].estimators), 'weights': step[1].weights})
print()
for estimator in step[1].estimators:
print_model(estimator[1], estimator[0] + ' - ')
else:
pprint(step[1].get_params())
print()
print_model(fitted_model)
The following is sample output for a pipeline using a specific algorithm (LogisticRegression with RobustScalar, in
this case).
RobustScaler
{'copy': True,
'quantile_range': [10, 90],
'with_centering': True,
'with_scaling': True}
LogisticRegression
{'C': 0.18420699693267145,
'class_weight': 'balanced',
'dual': False,
'fit_intercept': True,
'intercept_scaling': 1,
'max_iter': 100,
'multi_class': 'multinomial',
'n_jobs': 1,
'penalty': 'l2',
'random_state': None,
'solver': 'newton-cg',
'tol': 0.0001,
'verbose': 0,
'warm_start': False}
Explain the model (interpretability)

Automated machine learning allows you to understand feature importance. During the training process, you can
get global feature importance for the model. For classification scenarios, you can also get class-level feature
importance. You must provide a validation dataset (X_valid) to get feature importance.
There are two ways to generate feature importance.
Once an experiment is complete, you can use explain_model method on any iteration.
from azureml.train.automl.automlexplainer import explain_model
shap_values, expected_values, overall_summary, overall_imp, per_class_summary, per_class_imp = \

explain_model(fitted_model, X_train, X_test)
#Overall feature importance

print(overall_imp)
print(overall_summary)
#Class-level feature importance

print(per_class_imp)
print(per_class_summary)
To view feature importance for all iterations, set model_explainability flag to True in AutoMLConfig.
automl_config = AutoMLConfig(task = 'classification',

debug_log = 'automl_errors.log',
primary_metric = 'AUC_weighted',
max_time_sec = 12000,
iterations = 10,
verbosity = logging.INFO,
X = X_train,
y = y_train,
X_valid = X_test,
y_valid = y_test,
model_explainability=True,
path=project_folder)
Once done, you can use retrieve_model_explanation method to retrieve feature importance for a specific
iteration.
from azureml.train.automl.automlexplainer import retrieve_model_explanation

retrieve_model_explanation(best_run)
#Overall feature importance

print(overall_imp)
#Class-level feature importance

You can visualize the feature importance chart in your workspace in the Azure portal. Display the URL using the
run object:
automl_run.get_portal_url()
You can visualize the feature importance chart in your workspace in the Azure portal. The chart is also shown
when using the RunDetails Jupyter widget in a notebook. To learn more about the charts refer to Understand
automated machine learning results.

RunDetails(automl_run).show()
For more information on how model explanations and feature importance can be enabled in other areas of the
SDK outside of automated machine learning, see the concept article on interpretability.
Next steps
Learn more about how and where to deploy a model.
Learn more about how to train a regression model with Automated machine learning or how to train using
Automated machine learning on a remote resource.
Create, explore and deploy automated machine
learning experiments in the Azure portal (Preview)
In this article, you learn how to create, explore, and deploy automated machine learning experiments in the Azure
portal without a single line of code. Automated machine learning automates the process of selecting the best
algorithm to use for your specific data, so you can generate a machine learning model quickly. Learn more about
automated machine learning.
If you prefer a more code-based experience, you can also configure your automated machine learning
experiments in Python with the Azure Machine Learning SDK.
Prerequisites
An Azure Machine Learning service workspace. See Create an Azure Machine Learning service workspace.
Get started
Navigate to the left pane of your workspace. Select Automated Machine Learning under the Authoring (Preview )
section.
If this is your first time doing any experiments, you'll see the Welcome to Automated Machine Learning
screen.
Otherwise, you'll see your Automated machine learning dashboard with an overview of all of your automated
machine learning experiments, including those created with the SDK. Here you can filter and explore your runs by
date, experiment name, and run status.
Select Create Experiment and populate the Create a new automated machine learning experiment form.
1. Enter a unique experiment name.
2. Select a compute for the data profiling and training job. A list of your existing computes is available in the
dropdown. To create a new compute, follow the instructions in step 3.
3. Select Create a new compute to configure your compute context for this experiment.
FIELD DESCRIPTION
Compute name Enter a unique name that identifies your compute

context.
Virtual machine size Select the virtual machine size for your compute.
Additional settings Min node: Enter the minimum number of nodes for your
compute. The minimum number of nodes for AML
compute is 0. To enable data profiling, you must have 1
or more nodes.
Max node: Enter the maximum number of nodes for your
compute. The default is 6 nodes for an AML Compute.
Select Create. Creation of a new compute can take a few minutes.
NOTE
Your compute name will indicate if the compute you select/create is profiling enabled. (See 7b for more details on
data profiling).
4. Select a storage account for your data.

5. Select a storage container.
6. Select a data file from your storage container, or upload a file from your local computer to the container.
Public preview only supports local file uploads and Azure Blob Storage accounts.
IMPORTANT
Requirements for training data:
Data must be in tabular form.
The value you want to predict (target column) must be present in the data.
7. Use the preview and profile tabs to further configure your data for this experiment.
a. On the Preview tab, indicate if your data includes headers, and select the features (columns) for
training using the Included switch buttons in each feature column.
b. On the Profile tab, you can view the data profile by feature, as well as the distribution, type, and
summary statistics (mean, median, max/min, and so on) of each.
NOTE
The following error message will appear if your compute context is not profiling enabled: Data profiling is
only available for compute targets that are already running.
8. Select the training job type: classification, regression, or forecasting.

9. Select target column; this is the column that you would like to do predictions on.
10. For forecasting:
a. Select time column: This column contains the time data to be used.
b. Select forecast horizon: Indicate how many time units (minutes/hours/days/weeks/months/years)
will the model be able to predict to the future. The further the model is required to predict into the
future, the less accurate it will become. Learn more about forecasting and forecast horizon.
11. (Optional) Advanced settings: additional settings you can use to better control the training job.
ADVANCED SETTINGS DESCRIPTION
Primary metric Main metric used for scoring your model. Learn more
about model metrics.
Exit criteria When any of these criteria are met, the training job ends
before full completion.
Training job time (minutes ): How long to allow the
training job to run.
Max number of iterations: Maximum number of pipelines
(iterations) to test in the training job. The job will not run
more than the specified number of iterations.
Metric score threshold: Minimum metric score for all
pipelines. This ensures that if you have a defined target
metric you want to reach, you do not spend more time
on the training job than necessary.
Preprocessing Select to enable or disable the preprocessing done by

automated machine learning. Preprocessing includes
automatic data cleansing, preparing, and transformation
to generate synthetic features. Learn more about
preprocessing.
Validation Select one of the cross validation options to use in the

training job. Learn more about cross validation.
Concurrency Select the multi-core limits you would like to use when
using multi-core compute.
Blocked algorithm Select algorithms you want to exclude from the training
job.
Data profiling & summary stats

You can get a vast variety of summary statistics across your data set to verify whether your data set is ML -ready.
For non-numeric columns, they include only basic statistics like min, max, and error count. For numeric columns,
you can also review their statistical moments and estimated quantiles. Specifically, our data profile includes:
NOTE
Blank entries appear for features with irrelevant types.
STATISTIC DESCRIPTION
Feature Name of the column that is being summarized.
Profile In-line visualization based on the type inferred. For example,

strings, booleans, and dates will have value counts, while
decimals (numerics) have approximated histograms. This
allows you to gain a quick understanding of the distribution
of the data.
Type distribution In-line value count of types within a column. Nulls are their
own type, so this visualization is useful for detecting odd or
missing values.
Type Inferred type of the column. Possible values include: strings,

booleans, dates, and decimals.
Min Minimum value of the column. Blank entries appear for

features whose type does not have an inherent ordering (e.g.
booleans).
Max Maximum value of the column.
Count Total number of missing and non-missing entries in the

column.
Not missing count Number of entries in the column that are not missing. Empty
strings and errors are treated as values, so they will not
contribute to the "not missing count."
Quantiles Approximated values at each quantile to provide a sense of

the distribution of the data.
Mean Arithmetic mean or average of the column.
Standard deviation Measure of the amount of dispersion or variation of this

column's data.
Variance Measure of how far spread out this column's data is from its
average value.
Skewness Measure of how different this column's data is from a normal

distribution.
Kurtosis Measure of how heavily tailed this column's data is compared

to a normal distribution.
Advanced preprocessing options

When configuring your experiments, you can enable the advanced setting Preprocess . Doing so means that the
following data preprocessing and featurization steps are performed automatically.
PREPROCESSING STEPS DESCRIPTION
Drop high cardinality or no variance features Drop these from training and validation sets, including
features with all values missing, same value across all rows or
with extremely high cardinality (for example, hashes, IDs, or
GUIDs).
Impute missing values For numerical features, impute with average of values in the
column.
For categorical features, impute with most frequent value.
Generate additional features For DateTime features: Year, Month, Day, Day of week, Day of
year, Quarter, Week of the year, Hour, Minute, Second.
For Text features: Term frequency based on unigrams, bi-

grams, and tri-character-grams.
Transform and encode Numeric features with few unique values are transformed into
categorical features.
One-hot encoding is performed for low cardinality categorical;

for high cardinality, one-hot-hash encoding.
Word embeddings Text featurizer that converts vectors of text tokens into
sentence vectors using a pre-trained model. Each word’s
embedding vector in a document is aggregated together to
produce a document feature vector.
Target encodings For categorical features, maps each category with averaged
target value for regression problems, and to the class
probability for each class for classification problems.
Frequency-based weighting and k-fold cross validation is
applied to reduce over fitting of the mapping and noise
caused by sparse data categories.
Text target encoding For text input, a stacked linear model with bag-of-words is
used to generate the probability of each class.
Weight of Evidence (WoE) Calculates WoE as a measure of correlation of categorical

columns to the target column. It is calculated as the log of
the ratio of in-class vs out-of-class probabilities. This step
outputs one numerical feature column per class and removes
the need to explicitly impute missing values and outlier
treatment.
Cluster Distance Trains a k-means clustering model on all numerical columns.

Outputs k new features, one new numerical feature per
cluster, containing the distance of each sample to the centroid
of each cluster.
Run experiment and view results

Select Start to run your experiment. The experiment preparing process takes a couple of minutes.
View experiment details
Once the experiment preparation phase is done, you'll see the Run Detail screen begin to populate. This screen
gives you a full list of the models created. By default, the model that scores the highest based on the chosen
metric is at the top of the list. As the training job tries out more models, they are added to the iteration list and
chart. Use the iteration chart to get a quick comparison of the metrics for the models produced so far.
Training jobs can take a while for each pipeline to finish running.
View training run details

Drill down on any of the output models to see training run details, like performance metrics and distribution
charts. Learn more about charts.
Deploy your model

Once you have the best model at hand, it is time to deploy it as a web service to predict on new data.
Automated ML helps you with deploying the model without writing code:
1. You have a couple options for deployment.
Option 1: To deploy the best model (according to the metric criteria you defined), select Deploy Best
Model from the Run Detail page.
Option 2: To deploy a specific model iteration from this experiment, drill down on the model to open
its run detail page and select Deploy Model.
2. Populate the Deploy Model pane,
FIELD VALUE
Deployment name Enter a unique name for your deployment.
Deployment description Enter a description to better identify what this

deployment is for.
Scoring script Autogenerate or upload your own scoring file. Learn more
about scoring script
Environment script Autogenerate or upload your own environment file.
IMPORTANT
File names must be under 32 characters and must begin and end with alphanumerics. May include dashes,
underscores, dots, and alphanumerics between. Spaces are not allowed.
3. Select Deploy. Deployment can take about 20 minutes to complete.

The following message appears when deployment successfully completes.
Now you have an operational web service to generate predictions!
Next steps
Try the end to end tutorial for creating your first automated ML experiment with Azure Machine Learning.
Learn more about automated machine learning and Azure Machine Learning.
Understand automated machine learning results.
Learn how to consume a web service.
Train models with automated machine learning in the
cloud
In Azure Machine Learning, you train your model on different types of compute resources that you manage. The
compute target could be a local computer or a resource in the cloud.
You can easily scale up or scale out your machine learning experiment by adding additional compute targets, such
as Azure Machine Learning Compute (AmlCompute). AmlCompute is a managed-compute infrastructure that
allows you to easily create a single or multi-node compute.
In this article, you learn how to build a model using automated ML with AmlCompute.
How does remote differ from local?

The tutorial "Train a classification model with automated machine learning" teaches you how to use a local
computer to train a model with automated ML. The workflow when training locally also applies to remote targets
as well. However, with remote compute, automated ML experiment iterations are executed asynchronously. This
functionality allows you to cancel a particular iteration, watch the status of the execution, or continue to work on
other cells in the Jupyter notebook. To train remotely, you first create a remote compute target such as
AmlCompute. Then you configure the remote resource and submit your code there.
This article shows the extra steps needed to run an automated ML experiment on a remote AmlCompute target.
The workspace object, ws , from the tutorial is used throughout the code here.
Create resource
Create the AmlCompute target in your workspace ( ws ) if it doesn't already exist.
Time estimate: Creation of the AmlCompute target takes approximately 5 minutes.

amlcompute_cluster_name = "automlcl" # Name your cluster

provisioning_config = AmlCompute.provisioning_configuration(vm_size="STANDARD_D2_V2",
# for GPU, use "STANDARD_NC6"
# vm_priority = 'lowpriority', # optional
max_nodes=6)
ws, amlcompute_cluster_name, provisioning_config)
# Can poll for a minimum number of nodes and for a specific timeout.
# If no min_node_count is provided, it will use the scale settings for the cluster.
You can now use the compute_target object as the remote compute target.
Cluster name restrictions include:
Must be shorter than 64 characters.
Cannot include any of the following characters: \ ~ ! @ # $ % ^ & * ( ) = + _ [ ] { } \\ | ; : ' \" , < > / ?.`
Access data using TabularDataset function

Defined X and y as TabularDataset s, which are passed to Automated ML in the AutoMLConfig.
from_delimited_files by default sets the infer_column_types to true, which will infer the columns type
automatically.
If you do wish to manually set the column types, you can set the set_column_types argument to manually set the
type of each columns. In the following code sample, the data comes from the sklearn package.
# Create a project_folder if it doesn't exist

if not os.path.isdir('data'):
os.mkdir('data')
if not os.path.exists(project_folder):
os.makedirs(project_folder)
from sklearn import datasets

from azureml.core.dataset import Dataset
from scipy import sparse
import numpy as np
import pandas as pd
data_train = datasets.load_digits()
pd.DataFrame(data_train.data[100:,:]).to_csv("data/X_train.csv", index=False)
pd.DataFrame(data_train.target[100:]).to_csv("data/y_train.csv", index=False)
ds.upload(src_dir='./data', target_path='digitsdata', overwrite=True, show_progress=True)
X = Dataset.Tabular.from_delimited_files(path=ds.path('digitsdata/X_train.csv'))
y = Dataset.Tabular.from_delimited_files(path=ds.path('digitsdata/y_train.csv'))
Create run configuration

To make dependencies available to the get_data.py script, define a RunConfiguration object with defined
CondaDependencies . Use this object for the run_configuration parameter in AutoMLConfig .

run_config = RunConfiguration(framework="python")
run_config.target = compute_target
run_config.environment.docker.enabled = True
run_config.environment.docker.base_image = azureml.core.runconfig.DEFAULT_CPU_IMAGE
dependencies = CondaDependencies.create(
pip_packages=["scikit-learn", "scipy", "numpy"])
run_config.environment.python.conda_dependencies = dependencies
See this sample notebook for an additional example of this design pattern.
Specify the settings for AutoMLConfig . (See a full list of parameters and their possible values.)
import time
import logging
automl_settings = {
"name": "AutoML_Demo_Experiment_{0}".format(time.time()),
"iteration_timeout_minutes": 10,
"iterations": 20,
"n_cross_validations": 5,
"primary_metric": 'AUC_weighted',
"preprocess": False,
"max_concurrent_iterations": 10,
"verbosity": logging.INFO
}
automl_config = AutoMLConfig(task='classification',
debug_log='automl_errors.log',
run_configuration=run_config,
X = X,
y = y,
**automl_settings,
)
Enable model explanations

Set the optional model_explainability parameter in the AutoMLConfig constructor. Additionally, a validation
dataframe object must be passed as a parameter X_valid to use the model explainability feature.
automl_config = AutoMLConfig(task='classification',
debug_log='automl_errors.log',
run_configuration=run_config,
X = X,
y = y,
**automl_settings,
model_explainability=True,
X_valid=X_test
)
Submit training experiment

Now submit the configuration to automatically select the algorithm, hyper parameters, and train the model.

experiment = Experiment(ws, 'automl_remote')
remote_run = experiment.submit(automl_config, show_output=True)
You will see output similar to the following example:

Running on remote compute: mydsvmParent Run ID: AutoML_015ffe76-c331-406d-9bfd-0fd42d8ab7f6
***********************************************************************************************
ITERATION: The iteration being evaluated.
PIPELINE: A summary description of the pipeline being evaluated.
DURATION: Time taken for the current iteration.
METRIC: The result of computing score on the fitted pipeline.
BEST: The best observed score thus far.
***********************************************************************************************
ITERATION PIPELINE DURATION METRIC BEST

2 Standardize SGD classifier 0:02:36 0.954 0.954
7 Normalizer DT 0:02:22 0.161 0.954
0 Scale MaxAbs 1 extra trees 0:02:45 0.936 0.954
4 Robust Scaler SGD classifier 0:02:24 0.867 0.954
1 Normalizer kNN 0:02:44 0.984 0.984
9 Normalizer extra trees 0:03:15 0.834 0.984
5 Robust Scaler DT 0:02:18 0.736 0.984
8 Standardize kNN 0:02:05 0.981 0.984
6 Standardize SVM 0:02:18 0.984 0.984
10 Scale MaxAbs 1 DT 0:02:18 0.077 0.984
11 Standardize SGD classifier 0:02:24 0.863 0.984
3 Standardize gradient boosting 0:03:03 0.971 0.984
12 Robust Scaler logistic regression 0:02:32 0.955 0.984
14 Scale MaxAbs 1 SVM 0:02:15 0.989 0.989
13 Scale MaxAbs 1 gradient boosting 0:02:15 0.971 0.989
15 Robust Scaler kNN 0:02:28 0.904 0.989
17 Standardize kNN 0:02:22 0.974 0.989
16 Scale 0/1 gradient boosting 0:02:18 0.968 0.989
18 Scale 0/1 extra trees 0:02:18 0.828 0.989
19 Robust Scaler kNN 0:02:32 0.983 0.989
Explore results
You can use the same Jupyter widget as shown in the training tutorial to see a graph and table of results.

RunDetails(remote_run).show()
Here is a static image of the widget. In the notebook, you can click on any line in the table to see run properties
and output logs for that run. You can also use the dropdown above the graph to view a graph of each available
metric for each iteration.
The widget displays a URL you can use to see and explore the individual run details.
If you aren't in a Jupyter notebook, you can display the URL from the run itself:
remote_run.get_portal_url()
The same information is available in your workspace. To learn more about these results, see Understand
automated machine learning results.
View logs
Find logs on the DSVM under /tmp/azureml_run/{iterationid}/azureml-logs .
Best model explanation

Retrieving model explanation data allows you to see detailed information about the models to increase
transparency into what's running on the back-end. In this example, you run model explanations only for the best fit
model. If you run for all models in the pipeline, it will result in significant run time. Model explanation information
includes:
shap_values: The explanation information generated by shap lib.
expected_values: The expected value of the model applied to set of X_train data.
overall_summary: The model level feature importance values sorted in descending order.
overall_imp: The feature names sorted in the same order as in overall_summary.
per_class_summary: The class level feature importance values sorted in descending order. Only available for
the classification case.
per_class_imp: The feature names sorted in the same order as in per_class_summary. Only available for the
classification case.
Use the following code to select the best pipeline from your iterations. The get_output method returns the best
run and the fitted model for the last fit invocation.
best_run, fitted_model = remote_run.get_output()
Import the retrieve_model_explanation function and run on the best model.

from azureml.train.automl.automlexplainer import retrieve_model_explanation

retrieve_model_explanation(best_run)
Print results for the best_run explanation variables you want to view.
print(overall_imp)
Printing the best_run explanation summary variables results in the following output.
You can also visualize feature importance through the widget UI as well as the web UI on Azure portal inside your
workspace.
Example
The how -to-use-azureml/automated-machine-learning/remote-amlcompute/auto-ml-remote-amlcompute.ipynb
notebook demonstrates concepts in this article.
Next steps
Learn how to configure settings for automatic training.
Auto-train a time-series forecast model
In this article, you learn how to train a time-series forecasting regression model using automated machine
learning in Azure Machine Learning service. Configuring a forecasting model is similar to setting up a standard
regression model using automated machine learning, but certain configuration options and pre-processing steps
exist for working with time-series data. The following examples show you how to:
Prepare data for time series modeling
Configure specific time-series parameters in an AutoMLConfig object
Run predictions with time-series data
You can use automated ML to combine techniques and approaches and get a recommended, high-quality time-
series forecast. An automated time-series experiment is treated as a multivariate regression problem. Past time-
series values are “pivoted” to become additional dimensions for the regressor together with other predictors.
This approach, unlike classical time series methods, has an advantage of naturally incorporating multiple
contextual variables and their relationship to one another during training. In real-world forecasting applications,
multiple factors can influence a forecast. For example, when forecasting sales, interactions of historical trends,
exchange rate and price all jointly drive the sales outcome. A further benefit is that all recent innovations in
regression models apply immediately to forecasting.
You can configure how far into the future the forecast should extend (the forecast horizon), as well as lags and
more. Automated ML learns a single, but often internally branched model for all items in the dataset and
prediction horizons. More data is thus available to estimate model parameters and generalization to unseen series
becomes possible.
Features extracted from the training data play a critical role. And, automated ML performs standard pre-
processing steps and generates additional time-series features to capture seasonal effects and maximize predictive
accuracy.
Prerequisites
An Azure Machine Learning service workspace. To create the workspace, see Create an Azure Machine
This article assumes basic familiarity with setting up an automated machine learning experiment. Follow the
tutorial or how -to to see the basic automated machine learning experiment design patterns.
Preparing data
The most important difference between a forecasting regression task type and regression task type within
automated machine learning is including a feature in your data that represents a valid time series. A regular time
series has a well-defined and consistent frequency and has a value at every sample point in a continuous time
span. Consider the following snapshot of a file sample.csv .
day_datetime,store,sales_quantity,week_of_year
9/3/2018,A,2000,36
9/3/2018,B,600,36
9/4/2018,A,2300,36
9/4/2018,B,550,36
9/5/2018,A,2100,36
9/5/2018,B,650,36
9/6/2018,A,2400,36
9/6/2018,B,700,36
9/7/2018,A,2450,36
9/7/2018,B,650,36
This data set is a simple example of daily sales data for a company that has two different stores, A and B.
Additionally, there is a feature for week_of_year that will allow the model to detect weekly seasonality. The field
day_datetime represents a clean time series with daily frequency, and the field sales_quantity is the target
column for running predictions. Read the data into a Pandas dataframe, then use the to_datetime function to
ensure the time series is a datetime type.
import pandas as pd
data = pd.read_csv("sample.csv")
data["day_datetime"] = pd.to_datetime(data["day_datetime"])
In this case the data is already sorted ascending by the time field day_datetime . However, when setting up an
experiment, ensure the desired time column is sorted in ascending order to build a valid time series. Assume the
data contains 1,000 records, and make a deterministic split in the data to create training and test data sets. Then
separate the target field sales_quantity to create the prediction train and test sets.
X_train = data.iloc[:950]
X_test = data.iloc[-50:]
y_train = X_train.pop("sales_quantity").values
y_test = X_test.pop("sales_quantity").values
NOTE
When training a model for forecasting future values, ensure all the features used in training can be used when running
predictions for your intended horizon. For example, when creating a demand forecast, including a feature for current stock
price could massively increase training accuracy. However, if you intend to forecast with a long horizon, you may not be able
to accurately predict future stock values corresponding to future time-series points, and model accuracy could suffer.
Configure and run experiment

For forecasting tasks, automated machine learning uses pre-processing and estimation steps that are specific to
time-series data. The following pre-processing steps will be executed:
Detect time-series sample frequency (e.g. hourly, daily, weekly) and create new records for absent time points
to make the series continuous.
Impute missing values in the target (via forward-fill) and feature columns (using median column values)
Create grain-based features to enable fixed effects across different series
Create time-based features to assist in learning seasonal patterns
Encode categorical variables to numeric quantities
The AutoMLConfig object defines the settings and data necessary for an automated machine learning task. Similar
to a regression problem, you define standard training parameters like task type, number of iterations, training
data, and number of cross-validations. For forecasting tasks, there are additional parameters that must be set that
affect the experiment. The following table explains each parameter and its usage.
PARAM DESCRIPTION REQUIRED
time_column_name Used to specify the datetime column in ✓

the input data used for building the
time series and inferring its frequency.
grain_column_names Name(s) defining individual series

groups in the input data. If grain is not
defined, the data set is assumed to be
one time-series.
max_horizon Defines the maximum desired forecast ✓

horizon in units of time-series
frequency. Units are based on the time
interval of your training data, e.g.
monthly, weekly that the forecaster
should predict out.
target_lags n periods to forward-lag target values

prior to model training.
target_rolling_window_size n historical periods to use to generate

forecasted values, <= training set size.
If omitted, n is the full training set size.
Create the time-series settings as a dictionary object. Set the time_column_name to the day_datetime field in the
data set. Define the grain_column_names parameter to ensure that two separate time-series groups are created
for the data; one for store A and B. Lastly, set the max_horizon to 50 in order to predict for the entire test set. Set a
forecast window to 10 periods with target_rolling_window_size , and lag the target values 2 periods ahead with
the target_lags parameter.
time_series_settings = {
"time_column_name": "day_datetime",
"grain_column_names": ["store"],
"max_horizon": 50,
"target_lags": 2,
"target_rolling_window_size": 10,
"preprocess": True,
}
NOTE
Automated machine learning pre-processing steps (feature normalization, handling missing data, converting text to numeric,
etc.) become part of the underlying model. When using the model for predictions, the same pre-processing steps applied
during training are applied to your input data automatically.
Now create a standard AutoMLConfig object, specifying the forecasting task type, and submit the experiment.
After the model finishes, retrieve the best run iteration.
import logging
automl_config = AutoMLConfig(task='forecasting',
primary_metric='normalized_root_mean_squared_error',
iterations=10,
X=X_train,
y=y_train,
enable_ensembling=False,
**time_series_settings)
experiment = Experiment(ws, "forecasting_example")
local_run = experiment.submit(automl_config, show_output=True)
best_run, fitted_model = local_run.get_output()
NOTE
For the cross-validation (CV) procedure, time-series data can violate the basic statistical assumptions of the canonical K-Fold
cross-validation strategy, so automated machine learning implements a rolling origin validation procedure to create cross-
validation folds for time-series data. To use this procedure, specify the n_cross_validations parameter in the
AutoMLConfig object. You can bypass validation and use your own validation sets with the X_valid and y_valid
parameters.
View feature engineering summary

For time-series task types in automated machine learning, you can view details from the feature engineering
process. The following code shows each raw feature along with the following attributes:
Raw feature name
Number of engineered features formed out of this raw feature
Type detected
Whether feature was dropped
List of feature transformations for the raw feature
fitted_model.named_steps['timeseriestransformer'].get_featurization_summary()
Forecasting with best model

Use the best model iteration to forecast values for the test data set.
y_predict = fitted_model.predict(X_test)
y_actual = y_test.flatten()
Alternatively, you can use the forecast() function instead of predict() , which will allow specifications of when
predictions should start. In the following example, you first replace all values in y_pred with NaN . The forecast
origin will be at the end of training data in this case, as it would normally be when using predict() . However, if
you replaced only the second half of y_pred with NaN , the function would leave the numerical values in the first
half unmodified, but forecast the NaN values in the second half. The function returns both the forecasted values
and the aligned features.
You can also use the forecast_destination parameter in the forecast() function to forecast values up until a
specified date.
y_query = y_test.copy().astype(np.float)
y_query.fill(np.nan)
y_fcst, X_trans = fitted_pipeline.forecast(
X_test, y_query, forecast_destination=pd.Timestamp(2019, 1, 8))
Calculate RMSE (root mean squared error) between the y_test actual values, and the forecasted values in
y_pred .

from math import sqrt
rmse = sqrt(mean_squared_error(y_actual, y_predict))

rmse
Now that the overall model accuracy has been determined, the most realistic next step is to use the model to
forecast unknown future values. Simply supply a data set in the same format as the test set X_test but with
future datetimes, and the resulting prediction set is the forecasted values for each time-series step. Assume the
last time-series records in the data set were for 12/31/2018. To forecast demand for the next day (or as many
periods as you need to forecast, <= max_horizon ), create a single time series record for each store for 01/01/2019.
day_datetime,store,week_of_year
01/01/2019,A,1
01/01/2019,A,1
Repeat the necessary steps to load this future data to a dataframe and then run best_run.predict(X_test) to
predict future values.
NOTE
Values cannot be predicted for number of periods greater than the max_horizon . The model must be re-trained with a
larger horizon to predict future values beyond the current horizon.
Next steps
Follow the tutorial to learn how to create experiments with automated machine learning.
View the Azure Machine Learning SDK for Python reference documentation.
Understand automated machine learning results
In this article, you learn how to view and understand the charts and metrics for each of your automated machine
learning runs.
Learn more about:
Metrics, charts, and curves for classification models
Metrics, charts, and graphs for regression models
Model interpretability and feature importance
Prerequisites
Create an automated machine learning experiment, either with the SDK or in the Azure portal.
Use the SDK to build a classification model or regression model
Use the Azure portal to create a classification or regression model by uploading the appropriate data.
View the run

After running an automated machine learning experiment, a history of the runs can be found in your machine
learning service workspace.
1. Go to your workspace.
2. In the left panel of the workspace, select Experiments.
3. In the list of experiments, select the one you want to explore.
4. In the bottom table, select the Run Number.

)
5. In the Iterations table, select the Iteration Number for the model that you want to explore further.
You also see these same results during a run when you use the RunDetails Jupyter widget.
Classification results
Thee following metrics and charts are available for every classification model that you build using the automated
machine learning capabilities of Azure Machine Learning
Metrics
Confusion matrix
Precision-Recall chart
Receiver operating characteristics (or ROC )
Lift curve
Gains curve
Calibration plot
Classification metrics
The following metrics are saved in each run iteration for a classification task.
METRIC DESCRIPTION CALCULATION EX TRA PARAMETERS
AUC_Macro AUC is the Area under the Calculation average="macro"

Receiver Operating
Characteristic Curve. Macro
is the arithmetic mean of
the AUC for each class.
AUC_Micro AUC is the Area under the Calculation average="micro"

Receiver Operating
Characteristic Curve. Micro
is computed globally by
combining the true positives
and false positives from
each class.
AUC_Weighted AUC is the Area under the Calculation average="weighted"

Receiver Operating
Characteristic Curve.
Weighted is the arithmetic
mean of the score for each
class, weighted by the
number of true instances in
each class.
accuracy Accuracy is the percent of Calculation None

predicted labels that exactly
match the true labels.
average_precision_score_ma Average precision Calculation average="macro"

cro summarizes a precision-
recall curve as the weighted
mean of precisions achieved
at each threshold, with the
increase in recall from the
previous threshold used as
the weight. Macro is the
arithmetic mean of the
average precision score of
each class.
average_precision_score_mic Average precision Calculation average="micro"

ro summarizes a precision-
the weight. Micro is
computed globally by
combining the true positives
and false positives at each
cutoff.
average_precision_score_wei Average precision Calculation average="weighted"

ghted summarizes a precision-
the weight. Weighted is the
arithmetic mean of the
average precision score for
each class, weighted by the
number of true instances in
each class.
balanced_accuracy Balanced accuracy is the Calculation average="macro"

arithmetic mean of recall for
each class.
f1_score_macro F1 score is the harmonic Calculation average="macro"

mean of precision and recall.
Macro is the arithmetic
mean of F1 score for each
class.
f1_score_micro F1 score is the harmonic Calculation average="micro"

Micro is computed globally
by counting the total true
positives, false negatives,
and false positives.
f1_score_weighted F1 score is the harmonic Calculation average="weighted"

Weighted mean by class
frequency of F1 score for
each class
log_loss This is the loss function Calculation None

used in (multinomial) logistic
regression and extensions of
it such as neural networks,
defined as the negative log-
likelihood of the true labels
given a probabilistic
classifier’s predictions. For a
single sample with true label
yt in {0,1} and estimated
probability yp that yt = 1,
the log loss is -log P(yt|yp)
= -(yt log(yp) + (1 - yt)
log(1 - yp)).
norm_macro_recall Normalized Macro Recall is Calculation average = "macro"

Macro Recall normalized so
that random performance
has a score of 0 and perfect
performance has a score of
1. This is achieved by
norm_macro_recall :=
(recall_score_macro - R)/(1 -
R), where R is the expected
value of recall_score_macro
for random predictions (i.e.,
R=0.5 for binary
classification and R=(1/C)
for C-class classification
problems).
precision_score_macro Precision is the percent of Calculation average="macro"

elements labeled as a
certain class that actually
are in that class. Macro is
the arithmetic mean of
precision for each class.
precision_score_micro Precision is the percent of Calculation average="micro"

are in that class. Micro is
counting the total true
positives and false positives.
precision_score_weighted Precision is the percent of Calculation average="weighted"

are in that class. Weighted is
the arithmetic mean of
precision for each class,
weighted by number of true
instances in each class.
recall_score_macro Recall is the percent of Calculation average="macro"

elements actually in a
certain class that are
correctly labeled. Macro is
the arithmetic mean of recall
for each class.
recall_score_micro Recall is the percent of Calculation average="micro"

correctly labeled. Micro is
counting the total true
positives, false negatives
and false positives
recall_score_weighted Recall is the percent of Calculation average="weighted"

correctly labeled. Weighted
is the arithmetic mean of
recall for each class,
weighted by number of true
instances in each class.
weighted_accuracy Weighted accuracy is Calculation sample_weight is a vector

accuracy where the weight equal to the proportion of
given to each example is that class for each element
equal to the proportion of in the target
true instances in that
example's true class.
Confusion matrix
A confusion matrix is used to describe the performance of a classification model. Each row displays the instances
of the true class, and each column represents the instances of the predicted class. The confusion matrix shows the
correctly classified labels and the incorrectly classified labels for a given model.
For classification problems, Azure Machine Learning automatically provides a confusion matrix for each model
that is built. For each confusion matrix, automated ML will show the frequency of each predicted label and each
true label intersection. The darker the color, the higher the count in that particular part of the matrix. Ideally, the
darkest colors would be along the diagonal of the matrix.
Example 1: A classification model with poor accuracy
Example 2: A classification model with high accuracy (ideal)

Precision-recall chart
With this chart, you can compare the precision-recall curves for each model to determine which model has an
acceptable relationship between precision and recall for your particular business problem. This chart shows
Macro Average Precision-Recall, Micro Average Precision-Recall, and the precision-recall associated with all
classes for a model.
The term Precision represents that ability for a classifier to label all instances correctly. Recall represents the
ability for a classifier to find all instances of a particular label. The precision-recall curve shows the relationship
between these two concepts. Ideally, the model would have 100% precision and 100% accuracy.
Example 1: A classification model with low precision and low recall
Example 2: A classification model with ~100% precision and ~100% recall (ideal)
ROC
Receiver operating characteristic (or ROC ) is a plot of the correctly classified labels vs. the incorrectly classified
labels for a particular model. The ROC curve can be less informative when training models on datasets with high
bias, as it will not show the false positive labels.
Example 1: A classification model with low true labels and high false labels
Example 2: A classification model with high true labels and low false labels
Lift curve
You can compare the lift of the model built automatically with Azure Machine Learning to the baseline in order to
view the value gain of that particular model.
Lift charts are used to evaluate the performance of a classification model. It shows how much better you can
expect to do with a model compared to without a model.
Example 1: Model performs worse than a random selection model
Example 2: Model performs better than a random selection model

Gains curve
A gains chart evaluates the performance of a classification model by each portion of the data. It shows for each
percentile of the data set, how much better you can expect to perform compared against a random selection
model.
Use the cumulative gains chart to help you choose the classification cutoff using a percentage that corresponds to
a desired gain from the model. This information provides another way of looking at the results in the
accompanying lift chart.
Example 1: A classification model with minimal gain
Example 2: A classification model with significant gain

Calibration plot
For all classification problems, you can review the calibration line for micro-average, macro-average, and each
class in a given predictive model.
A calibration plot is used to display the confidence of a predictive model. It does this by showing the relationship
between the predicted probability and the actual probability, where “probability” represents the likelihood that a
particular instance belongs under some label. A well calibrated model aligns with the y=x line, where it is
reasonably confident in its predictions. An over-confident model aligns with the y=0 line, where the predicted
probability is present but there is no actual probability.
Example 1: A more well-calibrated model
Example 2: An over-confident model

Regression results
Thee following metrics and charts are available for every regression model that you build using the automated
machine learning capabilities of Azure Machine Learning
Metrics
Predicted vs. True
Histogram of residuals
Regression metrics
The following metrics are saved in each run iteration for a regression or forecasting task.
explained_variance Explained variance is the Calculation None

proportion to which a
mathematical model
accounts for the variation of
a given data set. It is the
percent decrease in variance
of the original data to the
variance of the errors. When
the mean of the errors is 0,
it is equal to explained
variance.
r2_score R2 is the coefficient of Calculation None

determination or the
percent reduction in
squared errors compared to
a baseline model that
outputs the mean.
spearman_correlation Spearman correlation is a Calculation None

nonparametric measure of
the monotonicity of the
relationship between two
datasets. Unlike the Pearson
correlation, the Spearman
correlation does not assume
that both datasets are
normally distributed. Like
other correlation
coefficients, this one varies
between -1 and +1 with 0
implying no correlation.
Correlations of -1 or +1
imply an exact monotonic
relationship. Positive
correlations imply that as x
increases, so does y.
Negative correlations imply
that as x increases, y
decreases.
mean_absolute_error Mean absolute error is the Calculation None

expected value of absolute
value of difference between
the target and the
prediction
normalized_mean_absolute_ Normalized mean absolute Calculation Divide by range of the data

error error is mean Absolute Error
divided by the range of the
data
median_absolute_error Median absolute error is the Calculation None

median of all absolute
differences between the
target and the prediction.
This loss is robust to
outliers.
normalized_median_absolut Normalized median absolute Calculation Divide by range of the data

e_error error is median absolute
error divided by the range
of the data
root_mean_squared_error Root mean squared error is Calculation None

the square root of the
expected squared difference
between the target and the
prediction
normalized_root_mean_squa Normalized root mean Calculation Divide by range of the data

red_error squared error is root mean
squared error divided by the
range of the data
root_mean_squared_log_err Root mean squared log Calculation None

or error is the square root of
the expected squared
logarithmic error
normalized_root_mean_squa Normalized Root mean Calculation Divide by range of the data

red_log_error squared log error is root
mean squared log error
divided by the range of the
data
Predicted vs. True

Predicted vs. True shows the relationship between a predicted value and its correlating true value for a regression
problem. This graph can be used to measure performance of a model as the closer to the y=x line the predicted
values are, the better the accuracy of a predictive model.
After each run, you can see a predicted vs. true graph for each regression model. To protect data privacy, values
are binned together and the size of each bin is shown as a bar graph on the bottom portion of the chart area. You
can compare the predictive model, with the lighter shade area showing error margins, against the ideal value of
where the model should be.
Example 1: A regression model with low accuracy in predictions
Example 2: A regression model with high accuracy in its predictions
Histogram of residuals
A residual represents an observed y – the predicted y. To show a margin of error with low bias, the histogram of
residuals should be shaped as a bell curve, centered around 0.
Example 1: A regression model with bias in its errors
Example 2: A regression model with more even distribution of errors

Model interpretability and feature importance
Feature importance allows you to see how valuable each feature was in the construction of a model. This
calculation is turned off by default, as it can significantly increase run time. You can enable model explanation for
all models or explain only the best fit model.
You can review the feature importance score for the model overall as well as per class on a predictive model. You
can see per feature how the importance compares against each class and overall.
For more information on enabling interpretability features, see Configure automated ML experiments in Python.
For an example that explains the best model, see Best model explanation.
Next steps
Learn more about automated ml in Azure Machine Learning.
Try the Automated Machine Learning Model Explanation sample notebook.
Deploy models with the Azure Machine
Learning service
Learn how to deploy your machine learning model as a web service in the Azure cloud, or to IoT Edge
devices.
The workflow is similar regardless of where you deploy your model:
1. Register the model.
2. Prepare to deploy (specify assets, usage, compute target).
3. Deploy the model to the compute target.
4. Test the deployed model, also called web service.
For more information on the concepts involved in the deployment workflow, see Manage, deploy, and
monitor models with Azure Machine Learning Service.
Prerequisites
An Azure Machine Learning service workspace. For more information, see Create an Azure
Machine Learning service workspace.
A model. If you do not have a trained model, you can use the model & dependency files
provided in this tutorial.
The Azure CLI extension for Machine Learning service, Azure Machine Learning Python SDK,
or the Azure Machine Learning Visual Studio Code extension.
Connect to your workspace

The following code demonstrates how to connect to an Azure Machine Learning service workspace
using information cached to the local development environment:
Using the SDK

ws = Workspace.from_config(path=".file-path/ws_config.json")
For more information on using the SDK to connect to a workspace, see the Azure Machine Learning
SDK for Python.
Using the CLI
When using the CLI, use the -w or --workspace-name parameter to specify the workspace for the
command.
Using VS Code
When using VS Code, the workspace is selected using a graphical interface. For more information, see
Deploy and manage models in the VS Code extension documentation.
Register your model
A registered model is a logical container for one or more files that make up your model. For example, if
you have a model that is stored in multiple files, you can register them as a single model in the
workspace. After registration, you can then download or deploy the registered model and receive all
the files that were registered.
TIP
When registering a model, you provide either a path to a cloud location (from a training run) or a local
directory. This path is just to locate the files for upload as part of the registration process; it does not need to
match the path used in the entry script. For more information, see What is get_model_path.
Machine learning models are registered in your Azure Machine Learning workspace. The model can
come from Azure Machine Learning or can come from somewhere else. The following examples
demonstrate how to register a model:
Register a model from an Experiment Run
The code snippets in this section demonstrate registering a model from a training run:
IMPORTANT
These snippets assume that you have previously performed a training run and have access to the run object
(SDK example) or run ID value (CLI example). For more information on training models, see Create and use
compute targets for model training.
Using the SDK

When using the SDK to train a model, you can receive either a Run or AutoMLRun object,
depending on how you trained the model. Each object can be used to register a model created
by an experiment run.
Register a model from an azureml.core.Run object:
model = run.register_model(model_name='sklearn_mnist',
model_path='outputs/sklearn_mnist_model.pkl')
print(model.name, model.id, model.version, sep='\t')
The model_path refers to the cloud location of the model. In this example, the path to a
single file is used. To include multiple files in the model registration, set model_path to
the directory that contains the files. For more information, see the Run.register_model
reference.
Register a model from an azureml.train.automl.run.AutoMLRun object:
description = 'My AutoML Model'

model = run.register_model(description = description)
print(run.model_id)
In this example, the metric and iteration parameters are not specified, which causes
the iteration with the best primary metric to be registered. The model_id value returned
from the run is used instead of a model name.
For more information, see the AutoMLRun.register_model reference.
Using the CLI
az ml model register -n sklearn_mnist --asset-path outputs/sklearn_mnist_model.pkl --

experiment-name myexperiment --run-id myrunid
TIP
The --asset-path refers to the cloud location of the model. In this example, the path to a single
file is used. To include multiple files in the model registration, set --asset-path to the directory
that contains the files.
Using VS Code
Register models using any model files or folders with the VS Code extension.
Register a model from a local file
You can register a model by providing a local path to the model. You can provide either a folder or a
single file. You can use this method to register both models trained with Azure Machine Learning
service and then downloaded, or models trained outside Azure Machine Learning.
IMPORTANT
You should only use models that you create or obtain from a trusted source. Serialized models should be
treated as code, as security vulnerabilities have been discovered in a number of popular formats. Further,
models may be intentionally trained with malicious intent to provide biased or inaccurate output.
ONNX example with the Python SDK:
import os
from azureml.core import Model
# Download model
onnx_model_url = "https://www.cntk.ai/OnnxModels/mnist/opset_7/mnist.tar.gz"
urllib.request.urlretrieve(onnx_model_url, filename="mnist.tar.gz")
os.system('tar xvzf mnist.tar.gz')
# Register model
model = Model.register(workspace = ws,
model_path ="mnist/model.onnx",
model_name = "onnx_mnist",
tags = {"onnx": "demo"},
description = "MNIST image classification CNN from ONNX Model Zoo",)
To include multiple files in the model registration, set model_path to the directory that contains
the files.
Using the CLI
az ml model register -n onnx_mnist -p mnist/model.onnx

To include multiple files in the model registration, set -p to the directory that contains the files.
Time estimate: Approximately 10 seconds.

For more information, see the reference documentation for the Model class.
For more information on working with models trained outside Azure Machine Learning service, see
How to deploy an existing model.
Choose a compute target

The following compute targets, or compute resources, can be used to host your web service
deployment.
Local web service Testing/debug Good for limited

testing and
troubleshooting.
Hardware
acceleration
depends on using
libraries in the local
system.
Notebook Testing/debug Good for limited

VM web service testing and
troubleshooting.
Azure Kubernetes Real-time inference yes yes Good for high-

Service (AKS) scale production
deployments.
Provides fast
response time and
autoscaling of the
deployed service.
Cluster autoscaling
is not supported
through the Azure
Machine Learning
SDK. To change the
nodes in the AKS
cluster, use the UI
for your AKS
cluster in the Azure
portal. AKS is the
only option
available for the
visual interface.
Azure Container Testing or dev Good for low scale,

Instances (ACI) CPU-based
workloads
requiring <48-GB
RAM
Azure Machine (Preview) Run batch scoring

Learning Compute Batch inference on serverless
compute. Supports
normal and low-
priority VMs.
Azure IoT Edge (Preview) Deploy & serve ML

IoT module models on IoT
devices.
Azure Data Box via IoT Edge yes Deploy & serve ML
Edge models on IoT
devices.
NOTE
While compute targets like local, Notebook VM, and Azure Machine Learning Compute support GPU for
training and experimentation, using GPU for inference is only supported on Azure Kubernetes Service.
Prepare to deploy
Deploying the model requires several things:
An entry script. This script accepts requests, scores the request using the model, and returns
the results.
IMPORTANT
The entry script is specific to your model; it must understand the format of the incoming request data,
the format of the data expected by your model, and the format of the data returned to clients.
If the request data is in a format that is not usable by your model, the script can transform it into an
acceptable format. It may also transform the response before returning to it to the client.
IMPORTANT
The Azure Machine Learning SDK does not provide a way for web service or IoT Edge deployments to
access your datastore or data sets. If you need the deployed model to access data stored outside the
deployment, such as in an Azure Storage account, you must develop a custom code solution using the
relevant SDK. For example, the Azure Storage SDK for Python.
Another alternative that may work for your scenario is batch predictions, which does provide access to
datastores when scoring.
Dependencies, such as helper scripts or Python/Conda packages required to run the entry
script or model
The deployment configuration for the compute target that hosts the deployed model. This
configuration describes things like memory and CPU requirements needed to run the model.
These entities are encapsulated into an inference configuration, and a deployment configuration.
The inference configuration references the entry script and other dependencies. These configurations
are defined programmatically when using the SDK, and as JSON files when using the CLI to perform
the deployment.
1. Define your entry script & dependencies
The entry script receives data submitted to a deployed web service, and passes it to the model. It then
takes the response returned by the model and returns that to the client. The script is specific to your
model; it must understand the data that the model expects and returns.
The script contains two functions that load and run the model:
: Typically this function loads the model into a global object. This function is run only
init()
once when the Docker container for your web service is started.
run(input_data) : This function uses the model to predict a value based on the input data. Inputs
and outputs to the run typically use JSON for serialization and de-serialization. You can also
work with raw binary data. You can transform the data before sending to the model, or before
returning to the client.
What is get_model_path?
When you register a model, you provide a model name used for managing the model in the registry.
You use this name with the Model.get_model_path() to retrieve the path of the model file(s) on the
local file system. If you register a folder or a collection of files, this API returns the path to the directory
that contains those files.
When you register a model, you give it a name, which corresponds to where the model is placed, either
locally or during service deployment.
IMPORTANT
If you trained a model using automated machine learning, a model_id value is used as the model name. For
an example of registering and deploying a model trained with automated ml, see
https://github.com/Azure/MachineLearningNotebooks/tree/master/how-to-use-azureml/automated-machine-
learning/classification-with-deployment.
The below example will return a path to a single file called sklearn_mnist_model.pkl (which was
registered with the name sklearn_mnist ):
model_path = Model.get_model_path('sklearn_mnist')
(Optional) Automatic schema generation

To automatically generate a schema for your web service, provide a sample of the input and/or output
in the constructor for one of the defined type objects, and the type and sample are used to
automatically create the schema. Azure Machine Learning service then creates an OpenAPI (Swagger)
specification for the web service during deployment.
The following types are currently supported:
pandas
numpy
pyspark
standard Python object
To use schema generation, include the inference-schema package in your conda environment file.
Ex a m p l e d e p e n d e n c i e s fi l e
The following YAML is an example of a Conda dependencies file for inference:

dependencies:
- python=3.6.2
- pip:
- azureml-defaults
- scikit-learn==0.20.0
- inference-schema[numpy-support]
If you want to use automatic schema generation, your entry script must import the inference-schema
packages.
Define the input and output sample formats in the input_sample and output_sample variables, which
represent the request and response formats for the web service. Use these samples in the input and
output function decorators on the run() function. The scikit-learn example below uses schema
generation.
Ex a m p l e e n t r y sc r i p t
The following example demonstrates how to accept and return JSON data:
#example: scikit-learn and Swagger

import json
import numpy as np
from inference_schema.schema_decorators import input_schema, output_schema

from inference_schema.parameter_types.numpy_parameter_type import NumpyParameterType
def init():
global model
# note here "sklearn_regression_model.pkl" is the name of the model registered under
# this is a different behavior than before when the code is run locally, even though the code
is the same.
model_path = Model.get_model_path('sklearn_regression_model.pkl')
# deserialize the model file back into a sklearn model
input_sample = np.array([[10, 9, 8, 7, 6, 5, 4, 3, 2, 1]])

output_sample = np.array([3726.995])
@input_schema('data', NumpyParameterType(input_sample))
@output_schema(NumpyParameterType(output_sample))
def run(data):
try:
result = model.predict(data)
# you can return any datatype as long as it is JSON-serializable
return result.tolist()
error = str(e)
return error
The following example demonstrates how to define the input data as a <key: value> dictionary, using
a Dataframe. This method is supported for consuming the deployed web service from Power BI (Learn
more about how to consume the web service from Power BI):
import json
import pickle
import numpy as np
import pandas as pd
import azureml.train.automl
from inference_schema.schema_decorators import input_schema, output_schema

from inference_schema.parameter_types.numpy_parameter_type import NumpyParameterType
from inference_schema.parameter_types.pandas_parameter_type import PandasParameterType
def init():
global model
# replace model_name with your actual model name, if needed
model_path = Model.get_model_path('model_name')
# deserialize the model file back into a sklearn model
input_sample = pd.DataFrame(data=[{
# This is a decimal type sample. Use the data type that reflects this column in your data
"input_name_1": 5.1,
# This is a string type sample. Use the data type that reflects this column in your data
"input_name_2": "value2",
# This is a integer type sample. Use the data type that reflects this column in your data
"input_name_3": 3
}])
# This is a integer type sample. Use the data type that reflects the expected result
output_sample = np.array([0])
@input_schema('data', PandasParameterType(input_sample))
@output_schema(NumpyParameterType(output_sample))
def run(data):
try:
# you can return any datatype as long as it is JSON-serializable
return result.tolist()
error = str(e)
return error
For more example scripts, see the following examples:

Pytorch: https://github.com/Azure/MachineLearningNotebooks/blob/master/how -to-use-
azureml/training-with-deep-learning/train-hyperparameter-tune-deploy-with-pytorch
TensorFlow: https://github.com/Azure/MachineLearningNotebooks/tree/master/how -to-use-
azureml/training-with-deep-learning/train-hyperparameter-tune-deploy-with-tensorflow
Keras: https://github.com/Azure/MachineLearningNotebooks/tree/master/how -to-use-
azureml/training-with-deep-learning/train-hyperparameter-tune-deploy-with-keras
ONNX: https://github.com/Azure/MachineLearningNotebooks/blob/master/how -to-use-
azureml/deployment/onnx/
Binary data
If your model accepts binary data, such as an image, you must modify the score.py file used for your
deployment to accept raw HTTP requests. To accept raw data, use the AMLRequest class in your entry
script and add the @rawhttp decorator to the run() function.
Here's an example of a score.py that accepts binary data:
from azureml.contrib.services.aml_request import AMLRequest, rawhttp
from azureml.contrib.services.aml_response import AMLResponse
def init():
print("This is init()")
@rawhttp
def run(request):
print("This is run()")
print("Request: [{0}]".format(request))
if request.method == 'GET':
# For this example, just return the URL for GETs
respBody = str.encode(request.full_path)
return AMLResponse(respBody, 200)
elif request.method == 'POST':
reqBody = request.get_data(False)
# For a real world solution, you would load the data from reqBody
# and send to the model. Then return the response.
# For demonstration purposes, this example just returns the posted data as the response.
return AMLResponse(reqBody, 200)
else:
return AMLResponse("bad request", 500)
IMPORTANT
The AMLRequest class is in the azureml.contrib namespace. Things in this namespace change frequently as
we work to improve the service. As such, anything in this namespace should be considered as a preview, and
not fully supported by Microsoft.
If you need to test this on your local development environment, you can install the components by using the
following command:
pip install azureml-contrib-services
Cross-origin resource sharing (CORS)

Cross-origin resource sharing is a way to allow resources on a web page to be requested from another
domain. CORS works based on HTTP headers sent with the client request and returned with the
service response. For more information on CORS and valid headers, see Cross-origin resource sharing
on Wikipedia.
To configure your model deployment to support CORS, use the AMLResponse class in your entry script.
This class allows you to set the headers on the response object.
The following example sets the Access-Control-Allow-Origin header for the response from the entry
script:
from azureml.contrib.services.aml_response import AMLResponse
def init():
print("This is init()")
def run(request):
print("This is run()")
print("Request: [{0}]".format(request))
if request.method == 'GET':
# For this example, just return the URL for GETs
respBody = str.encode(request.full_path)
return AMLResponse(respBody, 200)
elif request.method == 'POST':
reqBody = request.get_data(False)
# For a real world solution, you would load the data from reqBody
# and send to the model. Then return the response.
# For demonstration purposes, this example

# adds a header and returns the request body
resp = AMLResponse(reqBody, 200)
resp.headers['Access-Control-Allow-Origin'] = "http://www.example.com"
return resp
else:
return AMLResponse("bad request", 500)
IMPORTANT
The AMLResponse class is in the azureml.contrib namespace. Things in this namespace change frequently
as we work to improve the service. As such, anything in this namespace should be considered as a preview, and
not fully supported by Microsoft.
If you need to test this on your local development environment, you can install the components by using the
following command:
pip install azureml-contrib-services
2. Define your InferenceConfig

The inference configuration describes how to configure the model to make predictions. This
configuration is not part of your entry script; it references your entry script and is used to locate all the
resources required by the deployment. It is used later when actually deploying the model.
Inference configuration can use Azure Machine Learning environments to define the software
dependencies needed for your deployment. Environments allow you to create, manage, and reuse the
software dependencies required for training and deployment. The following example demonstrates
loading an environment from your workspace, and then using it with the inference configuration:

deploy_env = Environment.get(workspace=ws,name="myenv",version="1")
inference_config = InferenceConfig(entry_script="x/y/score.py",
environment=deploy_env)
For more information on environments, see Create and manage environments for training and
deployment.
You can also directly specify the dependencies without using an environment. The following example
demonstrates how to create an inference configuration that loads software dependencies from a conda
file:
inference_config = InferenceConfig(runtime="python",
entry_script="x/y/score.py",
conda_file="env/myenv.yml")
For more information, see the InferenceConfig class reference.

For information on using a custom Docker image with inference configuration, see How to deploy a
model using a custom Docker image.
CLI example of InferenceConfig
The entries in the inferenceconfig.json document map to the parameters for the InferenceConfig
class. The following table describes the mapping between entities in the JSON document and the
parameters for the method:
JSON ENTITY METHOD PARAMETER DESCRIPTION
entryScript entry_script Path to local file that contains the

code to run for the image.
runtime runtime Which runtime to use for the

image. Current supported runtimes
are 'spark-py' and 'python'.
condaFile conda_file Optional. Path to local file

containing a conda environment
definition to use for the image.
extraDockerFileSteps extra_docker_file_steps Optional. Path to local file

containing additional Docker steps
to run when setting up image.
sourceDirectory source_directory Optional. Path to folders that

contains all files to create the
image.
enableGpu enable_gpu Optional. Whether to enable GPU

support in the image. The GPU
image must be used on Microsoft
Azure Service. For example, Azure
Container Instances, Azure
Machine Learning Compute, Azure
Virtual Machines, and Azure
Kubernetes Service. Defaults to
False.
baseImage base_image Optional. A custom image to be

used as base image. If no base
image is given, then the base
image will be used based off of
given runtime parameter.
baseImageRegistry base_image_registry Optional. Image registry that

contains the base image.
cudaVersion cuda_version Optional. Version of CUDA to

install for images that need GPU
support. The GPU image must be
used on Microsoft Azure Services.
For example, Azure Container
Instances, Azure Machine Learning
Compute, Azure Virtual Machines,
and Azure Kubernetes Service.
Supported versions are 9.0, 9.1,
and 10.0. If 'enable_gpu' is set,
defaults to '9.1'.
description description A description for this image.
The following JSON is an example inference configuration for use with the CLI:
{
"entryScript": "score.py",
"runtime": "python",
"condaFile": "myenv.yml",
"extraDockerfileSteps": null,
"sourceDirectory": null,
"enableGpu": false,
"baseImage": null,
"baseImageRegistry": null
}
The following command demonstrates how to deploy a model using the CLI:
az ml model deploy -n myservice -m mymodel:1 --ic inferenceconfig.json
In this example, the configuration contains the following items:

That this model requires Python
The entry script, which is used to handle web requests sent to the deployed service
The conda file that describes the Python packages needed to inference
For information on using a custom Docker image with inference configuration, see How to deploy a
model using a custom Docker image.
3. Define your Deployment configuration
Before deploying, you must define the deployment configuration. The deployment configuration is
specific to the compute target that will host the web service. For example, when deploying
locally you must specify the port where the service accepts requests. The deployment configuration is
not part of your entry script. It is used to define the characteristics of the compute target that will host
the model and entry script.
You may also need to create the compute resource. For example, if you do not already have an Azure
Kubernetes Service associated with your workspace.
The following table provides an example of creating a deployment configuration for each compute
target:
COMPUTE TARGET DEPLOYMENT CONFIGURATION EXAMPLE
Local deployment_config =
LocalWebservice.deploy_configuration(port=8890)
Azure Container Instance deployment_config =

AciWebservice.deploy_configuration(cpu_cores =
1, memory_gb = 1)
Azure Kubernetes Service deployment_config =

AksWebservice.deploy_configuration(cpu_cores =
1, memory_gb = 1)
Each of these classes for local, ACI, and AKS web services can be imported from
azureml.core.webservice :
from azureml.core.webservice import AciWebservice, AksWebservice, LocalWebservice
Profiling
Prior to deploying your model as a service, you may want to profile it to determine optimal CPU and
memory requirements. You can profile your model using either the SDK or CLI. The following
examples show how to use profiling from the SDK:
IMPORTANT
When using profiling, the inference configuration that you provide cannot reference an Azure Machine Learning
environment. Instead, define the software dependencies using the conda_file parameter of the
InferenceConfig object.
import json
test_sample = json.dumps({'data': [
[1,2,3,4,5,6,7,8,9,10]
]})
profile = Model.profile(ws, "profilemymodel", [model], inference_config, test_data)

profile.wait_for_profiling(true)
profiling_results = profile.get_results()
print(profiling_results)
This code displays a result similar to the following text:
{'cpu': 1.0, 'memoryInGB': 0.5}
Model profiling results are emitted as a Run object.

For information on using profiling from the CLI, see az ml model profile.
For more information, see the following reference documents:
ModelProfile
profile()
Inference configuration file schema
Deploy to target
Deployment uses the inference configuration deployment configuration to deploy the models. The
deployment process is similar regardless of the compute target. Deploying to AKS is slightly different,
as you must provide a reference to the AKS cluster.
Local deployment
To deploy locally, you need to have Docker installed on your local machine.
Using the SDK
from azureml.core.webservice import LocalWebservice, Webservice
deployment_config = LocalWebservice.deploy_configuration(port=8890)
service = Model.deploy(ws, "myservice", [model], inference_config, deployment_config)
service.wait_for_deployment(show_output = True)
print(service.state)
For more information, see the reference documentation for LocalWebservice, Model.deploy(), and
Webservice.
Using the CLI
To deploy using the CLI, use the following command. Replace mymodel:1 with the name and version of
the registered model:
az ml model deploy -m mymodel:1 -ic inferenceconfig.json -dc deploymentconfig.json
The entries in the deploymentconfig.json document map to the parameters for

LocalWebservice.deploy_configuration. The following table describes the mapping between the entities
in the JSON document and the parameters for the method:
computeType NA The compute target. For local, the

value must be local .
port port The local port on which to expose

the service's HTTP endpoint.
The following JSON is an example deployment configuration for use with the CLI:
{
"computeType": "local",
"port": 32267
}
For more information, see the az ml model deploy reference.

NotebookVM web service (DEVTEST )
See Deploy a model to Notebook VMs.
Azure Container Instances (DEVTEST )
See Deploy to Azure Container Instances.
Azure Kubernetes Service (DEVTEST & PRODUCTION )
See Deploy to Azure Kubernetes Service.
Consume web services
Every deployed web service provides a REST API, so you can create client applications in a variety of
programming languages. If you have enabled key authentication for your service, you need to provide
a service key as a token in your request header. If you have enabled token authentication for your
service, you need to provide an Azure Machine Learning JWT token as a bearer token in your request
header.
TIP
You can retrieve the schema JSON document after deploying the service. Use the swagger_uri property from
the deployed web service, such as service.swagger_uri , to get the URI to the local web service's Swagger
file.
Request-response consumption
Here is an example of how to invoke your service in Python:
import requests
import json
if service.auth_enabled:
headers['Authorization'] = 'Bearer '+service.get_keys()[0]
elif service.token_auth_enabled:
headers['Authorization'] = 'Bearer '+service.get_token()[0]
print(headers)
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
[10, 9, 8, 7, 6, 5, 4, 3, 2, 1]
]})
response = requests.post(
service.scoring_uri, data=test_sample, headers=headers)
print(response.status_code)
print(response.elapsed)
print(response.json())
For more information, see Create client applications to consume webservices.

Web service schema (OpenAPI specification)
If you used the automatic schema generation with the deployment, you can get the address of the
OpenAPI specification for the service by using the swagger_uri property. For example,
print(service.swagger_uri) . Use a GET request (or open the URI in a browser ) to retrieve the
specification.
The following JSON document is an example of a schema (OpenAPI specification) generated for a
deployment:
{
"swagger": "2.0",
"info": {
"title": "myservice",
"description": "API specification for the Azure Machine Learning service myservice",
"version": "1.0"
},
"schemes": [
"https"
"https"
],
"consumes": [
"application/json"
],
"produces": [
"application/json"
],
"securityDefinitions": {
"Bearer": {
"type": "apiKey",
"name": "Authorization",
"in": "header",
"description": "For example: Bearer abc123"
}
},
"paths": {
"/": {
"get": {
"operationId": "ServiceHealthCheck",
"description": "Simple health check endpoint to ensure the service is up at any
given point.",
"responses": {
"200": {
"description": "If service is up and running, this response will be
returned with the content 'Healthy'",
"schema": {
"type": "string"
},
"examples": {
"application/json": "Healthy"
}
},
"default": {
"description": "The service failed to execute due to an error.",
"schema": {
"$ref": "#/definitions/ErrorResponse"
}
}
}
}
},
"/score": {
"post": {
"operationId": "RunMLService",
"description": "Run web service's model and get the prediction output",
"security": [
{
"Bearer": []
}
],
"parameters": [
{
"name": "serviceInputPayload",
"in": "body",
"description": "The input payload for executing the real-time machine
learning service.",
"schema": {
"$ref": "#/definitions/ServiceInput"
}
}
],
"responses": {
"200": {
"description": "The service processed the input correctly and provided a
result prediction, if applicable.",
"schema": {
"$ref": "#/definitions/ServiceOutput"
}
},
"default": {
"description": "The service failed to execute due to an error.",
"schema": {
"$ref": "#/definitions/ErrorResponse"
}
}
}
}
}
},
"definitions": {
"ServiceInput": {
"type": "object",
"properties": {
"data": {
"type": "array",
"items": {
"type": "array",
"items": {
"type": "integer",
"format": "int64"
}
}
}
},
"example": {
"data": [
[ 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 ]
]
}
},
"ServiceOutput": {
"type": "array",
"items": {
"type": "number",
"format": "double"
},
"example": [
3726.995
]
},
"ErrorResponse": {
"type": "object",
"properties": {
"status_code": {
"type": "integer",
"format": "int32"
},
"message": {
"type": "string"
}
}
}
}
}
For more information on the specification, see the Open API specification.
For a utility that can create client libraries from the specification, see swagger-codegen.
Batch inference
Azure Machine Learning Compute targets are created and managed by the Azure Machine Learning
service. They can be used for batch prediction from Azure Machine Learning Pipelines.
For a walkthrough of batch inference with Azure Machine Learning Compute, read the How to Run
Batch Predictions article.
IoT Edge inference
Support for deploying to the edge is in preview. For more information, see the Deploy Azure Machine
Learning as an IoT Edge module article.
Update web services

To update the web service, use the update method. You can update the web service to use a new
model, entry script, or dependencies that can be specified using an inference configuration. For more
information, see the reference for Webservice.update.
IMPORTANT
When you create a new version of a model, you must manually update each service that you want to use it.
Using the SDK

The following code demonstrates how to use the SDK to update the model, environment, and entry
script for a web service:

from azureml.core.model import Model, InferenceConfig
# register new model

new_model = Model.register(model_path="outputs/sklearn_mnist_model.pkl",
model_name="sklearn_mnist",
tags={"key": "0.1"},
description="test",
workspace=ws)
# Use version 3 of the environment

inference_config = InferenceConfig(entry_script="score.py",
service_name = 'myservice'
# Retrieve existing service
service = Webservice(name=service_name, workspace=ws)
# Update to new model(s)

service.update(models=[new_model], inference_config=inference_config)
print(service.get_logs())
Using the CLI

You can also update a web service using the ML CLI. The following example demonstrates registering
a new model, and then updating web service to use the new model:
az ml model register -n sklearn_mnist --asset-path outputs/sklearn_mnist_model.pkl --experiment-

name myexperiment --output-metadata-file modelinfo.json
az ml service update -n myservice --model-metadata-file modelinfo.json
TIP
In this example, a JSON document is used to pass the model information from the registration command into
the update command.
To update the service to use a new entry script or environment, create an inference configuration file and
specify it with the ic parameter.
For more information, see the az ml service update reference.
Continuous model deployment

You can continuously deploy models using the Machine Learning extension for Azure DevOps. By
using the Machine Learning extension for Azure DevOps, you can trigger a deployment pipeline when
a new machine learning model is registered in Azure Machine Learning service workspace.
1. Sign up for Azure Pipelines, which makes continuous integration and delivery of your
application to any platform/any cloud possible. Azure Pipelines differs from ML pipelines.
2. Create an Azure DevOps project.
3. Install the Machine Learning extension for Azure Pipelines
4. Use service Connections to set up a service principal connection to your Azure Machine
Learning service workspace to access all your artifacts. Go to project settings, click on service
connections, and select Azure Resource Manager.
5. Define AzureMLWorkspace as the scope level and fill in the subsequent parameters.
6. Next, to continuously deploy your machine learning model using the Azure Pipelines, under
pipelines select release. Add a new artifact, select AzureML Model artifact and the service
connection that was created in the earlier step. Select the model and version to trigger a
deployment.
7. Enable the model trigger on your model artifact. By turning on the trigger, every time the
specified version (i.e the newest version) of that model is register in your workspace, an Azure
DevOps release pipeline is triggered.
For more sample projects and examples, see the following sample repos:
https://github.com/Microsoft/MLOps
https://github.com/Microsoft/MLOpsPython
Package models
In some cases, you may want to create a Docker image without deploying the model. For example,
when you plan on deploying to Azure App Service. Or you may want to download the image and run
on a local Docker install. You may even want to download the files used to build the image, inspect
them, modify them, and build it manually.
Model packaging enables you to do both. It packages up all the assets needed to host a model as a
web service and allows you to download either a fully built Docker image or the files needed to build
one. There are two ways to use model packaging:
Download packaged model: You download a Docker image containing the model and other files
needed to host it as a web service.
Generate dockerfile: You download the dockerfile, model, entry script, and other assets needed to
build a Docker image. You can then inspect the files or make changes before building the image
locally.
Both packages can be used to get a local Docker image.
TIP
Creating a package is similar to deploying a model, as it uses a registered model and inference configuration.
IMPORTANT
Functionality such as downloading a fully built image or building an image locally requires a working Docker
installation on your development environment.
Download a packaged model

The following example demonstrates how to build an image, which is registered in the Azure
Container Registry for your workspace:
package = Model.package(ws, [model], inference_config)

package.wait_for_creation(show_output=True)
After creating a package, you can use package.pull() to pull the image to your local Docker
environment. The output of this command will display the name of the image. For example,
Status: Downloaded newer image for myworkspacef78fd10.azurecr.io/package:20190822181338 . After
downloading, use the docker images command to list the local images:
REPOSITORY TAG IMAGE ID CREATED

SIZE
myworkspacef78fd10.azurecr.io/package 20190822181338 7ff48015d5bd 4 minutes ago
1.43GB
To start a local container using this image, use the following command to start a named container from
the shell or command line. Replace the <imageid> value with the image ID returned from the
docker images command:
docker run -p 6789:5001 --name mycontainer <imageid>
This command starts the latest version of the image named myimage . It maps the local port of 6789 to
the port in the container that the web service is listening on (5001). It also assigns the name
mycontainer to the container, which makes it easier to stop. Once started, you can submit requests to
http://localhost:6789/score .
Generate dockerfile and dependencies

The following example demonstrates how to download the dockerfile, model, and other assets needed
to build the image locally. The generate_dockerfile=True parameter indicates that we want the files, not
a fully built image:
package = Model.package(ws, [model], inference_config, generate_dockerfile=True)

# Download the package
package.save("./imagefiles")
# Get the Azure Container Registry that the model/dockerfile uses
acr=package.get_container_registry()
print("Address:", acr.address)
print("Username:", acr.username)
print("Password:", acr.password)
This code downloads the files needed to build the image to the imagefiles directory. The dockerfile
included in the save files references a base image stored in an Azure Container Registry. When
building the image on your local Docker installation, you must use the address, user name, and
password to authenticate to this registry. Use the following steps to build the image using a local
Docker installation:
1. From a shell or command-line session, use the following command to authenticate Docker with
the Azure Container Registry. Replace <address> , <username> , and <password> with the values
retrieved using package.get_container_registry() :
docker login <address> -u <username> -p <password>
2. To build the image, use the following command. Replace <imagefiles> with the path to the
directory where package.save() saved the files:
docker build --tag myimage <imagefiles>

This command sets the image name to myimage .
To verify that the image has been built, use the docker images command. You should see the myimage
image in the list:
REPOSITORY TAG IMAGE ID CREATED SIZE

<none> <none> 2d5ee0bf3b3b 49 seconds ago 1.43GB
myimage latest 739f22498d64 3 minutes ago 1.43GB
To start a new container based on this image, use the following command:
docker run -p 6789:5001 --name mycontainer myimage:latest
This command starts the latest version of the image named myimage . It maps the local port of 6789 to
the port in the container that the web service is listening on (5001). It also assigns the name
mycontainer to the container, which makes it easier to stop. Once started, you can submit requests to
http://localhost:6789/score .
Example client to test the local container

The following code is an example of a Python client that can be used with the container:
import requests
import json
# URL for the web service

scoring_uri = 'http://localhost:6789/score'
# Two sets of data to score, so we get two results back

data = {"data":
[
[ 1,2,3,4,5,6,7,8,9,10 ],
[ 10,9,8,7,6,5,4,3,2,1 ]
]
}
# Convert to JSON string
input_data = json.dumps(data)
# Set the content type

# Make the request and display the response

resp = requests.post(scoring_uri, input_data, headers=headers)
print(resp.text)
For more example clients in other programming languages, see Consume models deployed as web
services.
Stop the Docker container
To stop the container, use the following command from a different shell or command line:
docker kill mycontainer
Clean up resources
To delete a deployed web service, use service.delete() . To delete a registered model, use
model.delete() .
For more information, see the reference documentation for WebService.delete(), and Model.delete().
Next steps
How to deploy a model using a custom Docker image
Deployment troubleshooting
Consume a ML Model deployed as a web service
Deploy a model to an Azure Kubernetes Service
cluster
Learn how to use the Azure Machine Learning service to deploy a model as a web service on Azure Kubernetes
Service (AKS ). Azure Kubernetes Service is good for high-scale production deployments. Use Azure Kubernetes
service if you need one or more of the following capabilities:
Fast response time.
Autoscaling of the deployed service.
Hardware acceleration options such as GPU and field-programmable gate arrays (FPGA).
IMPORTANT
Cluter scaling is not provided through the Azure Machine Learning SDK. For more information on scaling the nodes in an
AKS cluster, see Scale the node count in an AKS cluster.
When deploying to Azure Kubernetes Service, you deploy to an AKS cluster that is connected to your
workspace. There are two ways to connect an AKS cluster to your workspace:
Create the AKS cluster using the Azure Machine Learning service SDK, the Machine Learning CLI, or the Azure
portal. This process automatically connects the cluster to the workspace.
Attach an existing AKS cluster to your Azure Machine Learning service workspace. A cluster can be attached
using the Azure Machine Learning service SDK, Machine Learning CLI, or the Azure portal.
IMPORTANT
The creation or attachment process is a one time task. Once an AKS cluster is connected to the workspace, you can use it for
deployments. You can detach or delete the AKS cluster if you no longer need it. Once detatched or deleted, you will no
longer be able to deploy to the cluster.
Prerequisites
An Azure Machine Learning service workspace. For more information, see Create an Azure Machine
A machine learning model registered in your workspace. If you don't have a registered model, see How and
where to deploy models.
The Azure CLI extension for Machine Learning service, Azure Machine Learning Python SDK, or the Azure
Machine Learning Visual Studio Code extension.
The Python code snippets in this article assume that the following variables are set:
ws - Set to your workspace.
model - Set to your registered model.
inference_config - Set to the inference configuration for the model.
For more information on setting these variables, see How and where to deploy models.
The CLI snippets in this article assume that you've created an inferenceconfig.json document. For more
information on creating this document, see How and where to deploy models.
Create a new AKS cluster

Time estimate: Approximately 20 minutes.
Creating or attaching an AKS cluster is a one time process for your workspace. You can reuse this cluster for
multiple deployments. If you delete the cluster or the resource group that contains it, you must create a new
cluster the next time you need to deploy. You can have multiple AKS clusters attached to your workspace.
TIP
If you want to secure your AKS cluster using an Azure Virtual Network, you must create the virtual network first. For more
information, see Secure experimentation and inference with Azure Virtual Network.
If you want to create an AKS cluster for development, validation, and testing instead of production, you can
specify the cluster purpose to dev test.
WARNING
If you set cluster_purpose = AksCompute.ClusterPurpose.DEV_TEST , the cluster that is created is not suitable for
production level traffic and may increase inference times. Dev/test clusters also do not guarantee fault tolerance. We
recommend at least 2 virtual CPUs for dev/test clusters.
The following examples demonstrate how to create a new AKS cluster using the SDK and CLI:
Using the SDK
# Use the default configuration (you can also provide parameters to customize this).
# For example, to create a dev/test cluster, use:
# prov_config = AksCompute.provisioning_configuration(cluster_purpose = AksCompute.ClusterPurpose.DEV_TEST)
prov_config = AksCompute.provisioning_configuration()
aks_name = 'myaks'
aks_target = ComputeTarget.create(workspace = ws,
name = aks_name,
provisioning_configuration = prov_config)
# Wait for the create process to complete

aks_target.wait_for_completion(show_output = True)
IMPORTANT
For provisioning_configuration() , if you pick custom values for agent_count and vm_size , and cluster_purpose is
not DEV_TEST , then you need to make sure agent_count multiplied by vm_size is greater than or equal to 12 virtual
CPUs. For example, if you use a vm_size of "Standard_D3_v2", which has 4 virtual CPUs, then you should pick an
agent_count of 3 or greater.
The Azure Machine Learning SDK does not provide support scaling an AKS cluster. To scale the nodes in the cluster, use the
UI for your AKS cluster in the Azure portal. You can only change the node count, not the VM size of the cluster.
For more information on the classes, methods, and parameters used in this example, see the following reference
documents:
AksCompute.ClusterPurpose
AksCompute.provisioning_configuration
ComputeTarget.create
ComputeTarget.wait_for_completion
Using the CLI
az ml computetarget create aks -n myaks
For more information, see the az ml computetarget create ask reference.
Attach an existing AKS cluster

Time estimate: Approximately 5 minutes.
If you already have AKS cluster in your Azure subscription, and it is version 1.12.##, you can use it to deploy your
image.
TIP
The existing AKS cluster can be in a Azure region than your Azure Machine Learning service workspace.
If you want to secure your AKS cluster using an Azure Virtual Network, you must create the virtual network first. For more
information, see Secure experimentation and inference with Azure Virtual Network.
WARNING
When attaching an AKS cluster to a workspace, you can define how you will use the cluster by setting the cluster_purpose
parameter.
If you do not set the cluster_purpose parameter, or set cluster_purpose = AksCompute.ClusterPurpose.FAST_PROD ,
then the cluster must have at least 12 virtual CPUs available.
If you set cluster_purpose = AksCompute.ClusterPurpose.DEV_TEST , then the cluster does not need to have 12 virtual
CPUs. We recommend at least 2 virtual CPUs for dev/test. However a cluster that is configured for dev/test is not suitable
for production level traffic and may increase inference times. Dev/test clusters also do not guarantee fault tolerance.
For more information on creating an AKS cluster using the Azure CLI or portal, see the following articles:
Create an AKS cluster (CLI)
Create an AKS cluster (portal)
The following examples demonstrate how to attach an existing AKS 1.12.## cluster to your workspace:
Using the SDK
# Set the resource group that contains the AKS cluster and the cluster name
resource_group = 'myresourcegroup'
cluster_name = 'myexistingcluster'
# Attach the cluster to your workgroup. If the cluster has less than 12 virtual CPUs, use the following
instead:
# attach_config = AksCompute.attach_configuration(resource_group = resource_group,
# cluster_name = cluster_name,
# cluster_purpose = AksCompute.ClusterPurpose.DEV_TEST)
aks_target = ComputeTarget.attach(ws, 'myaks', attach_config)
documents:
AksCompute.attach_configuration()
AksCompute.ClusterPurpose
AksCompute.attach
Using the CLI
To attach an existing cluster using the CLI, you need to get the resource ID of the existing cluster. To get this value,
use the following command. Replace myexistingcluster with the name of your AKS cluster. Replace
myresourcegroup with the resource group that contains the cluster:
az aks show -n myexistingcluster -g myresourcegroup --query id
This command returns a value similar to the following text:
/subscriptions/{GUID}/resourcegroups/{myresourcegroup}/providers/Microsoft.ContainerService/managedClusters/{m
yexistingcluster}
To attach the existing cluster to your workspace, use the following command. Replace aksresourceid with the
value returned by the previous command. Replace myresourcegroup with the resource group that contains your
workspace. Replace myworkspace with your workspace name.
az ml computetarget attach aks -n myaks -i aksresourceid -g myresourcegroup -w myworkspace
For more information, see the az ml computetarget attach aks reference.
Deploy to AKS
To deploy a model to Azure Kubernetes Service, create a deployment configuration that describes the compute
resources needed. For example, number of cores and memory. You also need an inference configuration, which
describes the environment needed to host the model and web service. For more information on creating the
inference configuration, see How and where to deploy models.
Using the SDK
from azureml.core.webservice import AksWebservice, Webservice
aks_target = AksCompute(ws,"myaks")
# If deploying to a cluster configured for dev/test, ensure that it was created with enough
# cores and memory to handle this deployment configuration. Note that memory is also used by
# things such as dependencies and AML components.
deployment_config = AksWebservice.deploy_configuration(cpu_cores = 1, memory_gb = 1)
service = Model.deploy(ws, "myservice", [model], inference_config, deployment_config, aks_target)
documents:
AksCompute
AksWebservice.deploy_configuration
Model.deploy
Webservice.wait_for_deployment
Using the CLI
To deploy using the CLI, use the following command. Replace myaks with the name of the AKS compute target.
Replace mymodel:1 with the name and version of the registered model. Replace myservice with the name to give
this service:
az ml model deploy -ct myaks -m mymodel:1 -n myservice -ic inferenceconfig.json -dc deploymentconfig.json

AksWebservice.deploy_configuration. The following table describes the mapping between the entities in the JSON
document and the parameters for the method:
computeType NA The compute target. For AKS, the value

must be aks .
autoScaler NA Contains configuration elements for

autoscale. See the autoscaler table.
autoscaleEnabled autoscale_enabled Whether to enable autoscaling for the

web service. If numReplicas = 0 ,
True ; otherwise, False .
minReplicas autoscale_min_replicas The minimum number of containers to

use when autoscaling this web service.
Default, 1 .
maxReplicas autoscale_max_replicas The maximum number of containers to

Default, 10 .
refreshPeriodInSeconds autoscale_refresh_seconds How often the autoscaler attempts to

scale this web service. Default, 1 .
targetUtilization autoscale_target_utilization The target utilization (in percent out of

100) that the autoscaler should attempt
to maintain for this web service.
Default, 70 .
dataCollection NA Contains configuration elements for

data collection.
storageEnabled collect_model_data Whether to enable model data

collection for the web service. Default,
False .
authEnabled auth_enabled Whether or not to enable key

authentication for the web service. Both
tokenAuthEnabled and authEnabled
cannot be True . Default, True .
tokenAuthEnabled token_auth_enabled Whether or not to enable token

authentication for the web service. Both
tokenAuthEnabled and authEnabled
cannot be True . Default, False .
containerResourceRequirements NA Container for the CPU and memory

entities.
cpu cpu_cores The number of CPU cores to allocate for

this web service. Defaults, 0.1
memoryInGB memory_gb The amount of memory (in GB) to

allocate for this web service. Default,
0.5
appInsightsEnabled enable_app_insights Whether to enable Application Insights

logging for the web service. Default,
False .
scoringTimeoutMs scoring_timeout_ms A timeout to enforce for scoring calls to

the web service. Default, 60000 .
maxConcurrentRequestsPerContainer replica_max_concurrent_requests The maximum concurrent requests per

node for this web service. Default, 1 .
maxQueueWaitMs max_request_wait_time The maximum time a request will stay in

thee queue (in milliseconds) before a
503 error is returned. Default, 500 .
numReplicas num_replicas The number of containers to allocate for

this web service. No default value. If this
parameter is not set, the autoscaler is
enabled by default.
keys NA Contains configuration elements for

keys.
primaryKey primary_key A primary auth key to use for this

Webservice
secondaryKey secondary_key A secondary auth key to use for this

Webservice
gpuCores gpu_cores The number of GPU cores to allocate

for this Webservice. Default is 1. Only
supports whole number values.
livenessProbeRequirements NA Contains configuration elements for

liveness probe requirements.
periodSeconds period_seconds How often (in seconds) to perform the

liveness probe. Default to 10 seconds.
Minimum value is 1.
initialDelaySeconds initial_delay_seconds Number of seconds after the container

has started before liveness probes are
initiated. Defaults to 310
timeoutSeconds timeout_seconds Number of seconds after which the

liveness probe times out. Defaults to 2
seconds. Minimum value is 1
successThreshold success_threshold Minimum consecutive successes for the

liveness probe to be considered
successful after having failed. Defaults
to 1. Minimum value is 1.
failureThreshold failure_threshold When a Pod starts and the liveness

probe fails, Kubernetes will try
failureThreshold times before giving up.
Defaults to 3. Minimum value is 1.
namespace namespace The Kubernetes namespace that the

webservice is deployed into. Up to 63
lowercase alphanumeric ('a'-'z', '0'-'9')
and hyphen ('-') characters. The first
and last characters can't be hyphens.
{
"computeType": "aks",
"autoScaler":
{
"autoscaleEnabled": true,
"minReplicas": 1,
"maxReplicas": 3,
"refreshPeriodInSeconds": 1,
"targetUtilization": 70
},
"dataCollection":
{
"storageEnabled": true
},
"authEnabled": true,
"containerResourceRequirements":
{
"cpu": 0.5,
"memoryInGB": 1.0
}
}

Using VS Code
For information on using VS Code, see deploy to AKS via the VS Code extension.
IMPORTANT
Deploying through VS Code requires the AKS cluster to be created or attached to your workspace in advance.
Web service authentication

When deploying to Azure Kubernetes Service, key-based authentication is enabled by default. You can also
enable token-based authentication. Token-based authentication requires clients to use an Azure Active Directory
account to request an authentication token, which is used to make requests to the deployed service.
To disable authentication, set the auth_enabled=False parameter when creating the deployment configuration.
The following example disables authentication using the SDK:
deployment_config = AksWebservice.deploy_configuration(cpu_cores=1, memory_gb=1, auth_enabled=False)
For information on authenticating from a client application, see the Consume an Azure Machine Learning model
deployed as a web service.
If key authentication is enabled, you can use the get_keys method to retrieve a primary and secondary
authentication key:

print(primary)
IMPORTANT
If you need to regenerate a key, use service.regen_key
To enable token authentication, set the token_auth_enabled=True parameter when you are creating or updating a
deployment. The following example enables token authentication using the SDK:
deployment_config = AksWebservice.deploy_configuration(cpu_cores=1, memory_gb=1, token_auth_enabled=True)
If token authentication is enabled, you can use the get_token method to retrieve a JWT token and that token's
expiration time:

print(token)
IMPORTANT
You will need to request a new token after the token's refresh_by time.
Microsoft strongly recommends that you create your Azure Machine Learning workspace in the same region as your Azure
Kubernetes Service cluster. To authenticate with a token, the web service will make a call to the region in which your Azure
Machine Learning workspace is created. If your workspace's region is unavailable, then you will not be able to fetch a token
for your web service even, if your cluster is in a different region than your workspace. This effectively results in Token-based
Authentication being unavailable until your workspace's region is available again. In addition, the greater the distance
between your cluster's region and your workspace's region, the longer it will take to fetch a token.
Update the web service

To update the web service, use the update method. You can update the web service to use a new model, entry
script, or dependencies that can be specified using an inference configuration. For more information, see the
reference for Webservice.update.
IMPORTANT
Using the SDK

The following code demonstrates how to use the SDK to update the model, environment, and entry script for a
web service:

tags={"key": "0.1"},
description="test",
workspace=ws)


Using the CLI

You can also update a web service using the ML CLI. The following example demonstrates registering a new
model, and then updating web service to use the new model:
az ml model register -n sklearn_mnist --asset-path outputs/sklearn_mnist_model.pkl --experiment-name

myexperiment --output-metadata-file modelinfo.json
TIP
In this example, a JSON document is used to pass the model information from the registration command into the update
command.
To update the service to use a new entry script or environment, create an inference configuration file and specify it with the
ic parameter.
Next steps
Secure experimentation and inference in a virtual network
Deploy a deep learning model for inference with
GPU
This article teaches you how to use the Azure Machine Learning service to deploy a GPU -enabled model as a web
service. The information in this article is based on deploying a model on Azure Kubernetes Service (AKS ). The
AKS cluster provides a GPU resource that is used by the model for inference.
Inference, or model scoring, is the phase where the deployed model is used to make predictions. Using GPUs
instead of CPUs offers performance advantages on highly parallelizable computation.
IMPORTANT
GPU inference is only supported on Azure Kubernetes Service.
TIP
Although the code snippets in this article usee a TensorFlow model, you can apply the information to any machine learning
framework that supports GPUs.
NOTE
The information in this article builds on the information in the How to deploy to Azure Kubernetes Service article. Where that
article generally covers deployment to AKS, this article covers GPU specific deployment.
Prerequisites
A Python development environment with the Azure Machine Learning SDK installed. For more
information, see Azure Machine Learning SDK.
A registered model that uses a GPU.
To learn how to register models, see Deploy Models.
To create and register the Tensorflow model used to create this document, see How to Train a
TensorFlow Model.
A general understanding of How and where to deploy models.
Connect to your workspace

To connect to an existing workspace, use the following code:
IMPORTANT
This code snippet expects the workspace configuration to be saved in the current directory or its parent. For more
information on creating a workspace, see Create and manage Azure Machine Learning service workspaces. For more
information on saving the configuration to file, see Create a workspace configuration file.
# Connect to the workspace

Create a Kubernetes cluster with GPUs

Azure Kubernetes Service provides many different GPU options. You can use any of them for model inference. See
the list of N -series VMs for a full breakdown of capabilities and costs.
The following code demonstrates how to create a new AKS cluster for your workspace:
from azureml.core.compute import ComputeTarget, AksCompute

# Choose a name for your cluster

aks_name = "aks-gpu"
# Check to see if the cluster already exists

try:
aks_target = ComputeTarget(workspace=ws, name=aks_name)
# Provision AKS cluster with GPU machine
prov_config = AksCompute.provisioning_configuration(vm_size="Standard_NC6")

aks_target = ComputeTarget.create(
workspace=ws, name=aks_name, provisioning_configuration=prov_config
)
aks_target.wait_for_completion(show_output=True)
IMPORTANT
Azure will bill you as long as the AKS cluster exists. Make sure to delete your AKS cluster when you're done with it.
For more information on using AKS with Azure Machine Learning service, see How to deploy to Azure
Kubernetes Service.
Write the entry script

The entry script receives data submitted to the web service, passes it to the model, and returns the scoring results.
The following script loads the Tensorflow model on startup, and then uses the model to score data.
TIP
The entry script is specific to your model. For example, the script must know the framework to use with your model, data
formats, etc.
import json
import numpy as np
import os
def init():
global X, output, sess
tf.reset_default_graph()
model_root = Model.get_model_path('tf-dnn-mnist')
saver = tf.train.import_meta_graph(
os.path.join(model_root, 'mnist-tf.model.meta'))
X = tf.get_default_graph().get_tensor_by_name("network/X:0")
output = tf.get_default_graph().get_tensor_by_name("network/output/MatMul:0")
sess = tf.Session()
saver.restore(sess, os.path.join(model_root, 'mnist-tf.model'))
def run(raw_data):
data = np.array(json.loads(raw_data)['data'])
# make prediction
out = output.eval(session=sess, feed_dict={X: data})
y_hat = np.argmax(out, axis=1)
return y_hat.tolist()
This file is named score.py . For more information on entry scripts, see How and where to deploy.
Define the conda environment

The conda environment file specifies the dependencies for the service. It includes dependencies required by both
the model and the entry script. The following YAML defines the environment for a Tensorflow model. It specifies
tensorflow-gpu , which will make use of the GPU used in this deployment:
dependencies:
# The python interpreter version.
# Currently Azure ML only supports 3.5.2 and later.
- python=3.6.2
- pip:
- azureml-defaults==1.0.43.*
- numpy
- tensorflow-gpu=1.12
channels:
- conda-forge
For this example, the file is saved as myenv.yml .
Define the deployment configuration

The deployment configuration defines the Azure Kubernetes Service environment used to run the web service:
from azureml.core.webservice import AksWebservice
gpu_aks_config = AksWebservice.deploy_configuration(autoscale_enabled=False,
num_replicas=3,
cpu_cores=2,
memory_gb=4)
For more information, see the reference documentation for AksService.deploy_configuration.
Define the inference configuration

The inference configuration points to the entry script and conda environment file. It also enables GPU support,
which installs CUDA in the docker image created for the web service:
conda_file="myenv.yml",
enable_gpu=True)
For more information, see the reference documentation for InferenceConfig.
Deploy the model

Deploy the model to your AKS cluster and wait for it to create your service.
# Name of the web service that is deployed

aks_service_name = 'aks-dnn-mnist'
# Get the registerd model
model = Model(ws, "tf-dnn-mnist")
# Deploy the model
aks_service = Model.deploy(ws,
models=[model],
inference_config=inference_config,
deployment_config=gpu_aks_config,
deployment_target=aks_target,
name=aks_service_name)
print(aks_service.state)
NOTE
If the InferenceConfig object has enable_gpu=True , then the deployment_target parameter must reference a cluster
that provides a GPU. Otherwise, the deployment will fail.
For more information, see the reference documentation for Model.
Issue a sample query to your service

Send a test query to the deployed model. When you send a jpeg image to the model, it scores the image. The
following code sample downloads test data and then selects a random test image to send to the service.
# Used to test your webservice
import os
import urllib
import gzip
import numpy as np
import struct
import requests
# load compressed MNIST gz files and return numpy arrays

def load_data(filename, label=False):
with gzip.open(filename) as gz:
struct.unpack('I', gz.read(4))
n_items = struct.unpack('>I', gz.read(4))
if not label:
n_rows = struct.unpack('>I', gz.read(4))[0]
n_cols = struct.unpack('>I', gz.read(4))[0]
res = np.frombuffer(gz.read(n_items[0] * n_rows * n_cols), dtype=np.uint8)
res = res.reshape(n_items[0], n_rows * n_cols)
else:
res = np.frombuffer(gz.read(n_items[0]), dtype=np.uint8)
res = res.reshape(n_items[0], 1)
return res
# one-hot encode a 1-D array

def one_hot_encode(array, num_of_classes):
return np.eye(num_of_classes)[array.reshape(-1)]
# Download test data

urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz',
filename='./data/mnist/test-images.gz')
urllib.request.urlretrieve('http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz',
filename='./data/mnist/test-labels.gz')
# Load test data from model training

X_test = load_data('./data/mnist/test-images.gz', False) / 255.0
y_test = load_data('./data/mnist/test-labels.gz', True).reshape(-1)
# send a random row from the test set to score

random_index = np.random.randint(0, len(X_test)-1)
input_data = "{\"data\": [" + str(list(X_test[random_index])) + "]}"
api_key = aks_service.get_keys()[0]
headers = {'Content-Type': 'application/json',
'Authorization': ('Bearer ' + api_key)}
resp = requests.post(aks_service.scoring_uri, input_data, headers=headers)
print("POST to url", aks_service.scoring_uri)

print("label:", y_test[random_index])
For more information on creating a client application, see Create client to consume deployed web service.
Clean up the resources

If you created the AKS cluster specifically for this example, delete your resources after you're done.
IMPORTANT
Azure bills you based on how long the AKS cluster is deployed. Make sure to clean it up after you are done with it.
aks_service.delete()
aks_target.delete()
Next steps
Deploy model on FPGA
Deploy model with ONNX
Train Tensorflow DNN Models
Deploy a machine learning model to Azure App
Service (preview)
Learn how to deploy a model from the Azure Machine Learning service as a web app in Azure App Service.
IMPORTANT
While both Azure Machine Learning service and Azure App Service are generally available, the ability to deploy a model from
the Machine Learning service to App Service is in preview.
With Azure Machine Learning service, you can create Docker images from trained machine learning models. This
image contains a web service that receives data, submits it to the model, and then returns the response. Azure App
Service can be used to deploy the image, and provides the following features:
Advanced authentication for enhanced security. Authentication methods include both Azure Active Directory
and multi-factor auth.
Autoscale without having to redeploy.
SSL support for secure communications between clients and the service.
For more information on features provided by Azure App Service, see the App Service overview.
IMPORTANT
If you need the ability to log the scoring data used with your deployed model, or the results of scoring, you should instead
deploy to Azure Kubernetes Service. For more information, see Collect data on your production models.
Prerequisites
An Azure Machine Learning service workspace. For more information, see the Create a workspace article.
The Azure CLI.
A trained machine learning model registered in your workspace. If you do not have a model, use the Image
classification tutorial: train model to train and register one.
IMPORTANT
The code snippets in this article assume that you have set the following variables:
ws - Your Azure Machine Learning workspace.
model - The registered model that will be deployed.
inference_config - The inference configuration for the model.
For more information on setting these variables, see Deploy models with the Azure Machine Learning service.
Prepare for deployment

Before deploying, you must define what is needed to run the model as a web service. The following list describes
the basic items needed for a deployment:
An entry script. This script accepts requests, scores the request using the model, and returns the results.
IMPORTANT
The entry script is specific to your model; it must understand the format of the incoming request data, the format of
the data expected by your model, and the format of the data returned to clients.
If the request data is in a format that is not usable by your model, the script can transform it into an acceptable
format. It may also transform the response before returning to it to the client.
IMPORTANT
The Azure Machine Learning SDK does not provide a way for the web service access your datastore or data sets. If
you need the deployed model to access data stored outside the deployment, such as in an Azure Storage account,
you must develop a custom code solution using the relevant SDK. For example, the Azure Storage SDK for Python.
Another alternative that may work for your scenario is batch predictions, which does provide access to datastores
when scoring.
For more information on entry scripts, see Deploy models with the Azure Machine Learning service.
Dependencies, such as helper scripts or Python/Conda packages required to run the entry script or model
These entities are encapsulated into an inference configuration. The inference configuration references the entry
script and other dependencies.
IMPORTANT
When creating an inference configuration for use with Azure App Service, you must use an Environment object. The following
example demonstrates creating an environment object and using it with an inference configuration:

# Create an environment and add conda dependencies to it

# Enable Docker based environment
# Build conda dependencies
myenv.python.conda_dependencies = CondaDependencies.create(conda_packages=['scikit-learn'])
For more information on environments, see Create and manage environments for training and deployment.
For more information on inference configuration, see Deploy models with the Azure Machine Learning service.
IMPORTANT
When deploying to Azure App Service, you do not need to create a deployment configuration.
Create the image

To create the Docker image that is deployed to Azure App Service, use Model.package. The following code snippet
demonstrates how to build a new image from the model and inference configuration:
NOTE
The code snippet assumes that model contains a registered model, and that inference_config contains the configuration
for the inference environment. For more information, see Deploy models with the Azure Machine Learning service.
from azureml.core import Model
package = Model.package(ws, [model], inference_config)

# Display the package location/ACR path
print(package.location)
When show_output=True , the output of the Docker build process is shown. Once the process finishes, the image
has been created in the Azure Container Registry for your workspace. Once the image has been built, the location
in your Azure Container Registry is displayed. The location returned is in the format
<acrinstance>.azurecr.io/package:<imagename> . For example, myml08024f78fd10.azurecr.io/package:20190827151241 .
IMPORTANT
Save the location information, as it is used when deploying the image.
Deploy image as a web app

1. Use the following command to get the login credentials for the Azure Container Registry that contains the
image. Replace <acrinstance> with th e value returned previously from package.location :
az acr credential show --name <myacr>
The output of this command is similar to the following JSON document:
{
"passwords": [
{
"name": "password",
"value": "Iv0lRZQ9762LUJrFiffo3P4sWgk4q+nW"
},
{
"name": "password2",
"value": "=pKCxHatX96jeoYBWZLsPR6opszr==mg"
}
],
"username": "myml08024f78fd10"
}
Save the value for username and one of the passwords.

2. If you do not already have a resource group or app service plan to deploy the service, the following
commands demonstrate how to create both:
az group create --name myresourcegroup --location "West Europe"

az appservice plan create --name myplanname --resource-group myresourcegroup --sku B1 --is-linux
In this example, a Basic pricing tier ( --sku B1 ) is used.

IMPORTANT
Images created by the Azure Machine Learning service use Linux, so you must use the --is-linux parameter.
3. To create the web app, use the following command. Replace <app-name> with the name you want to use.
Replace <acrinstance> and <imagename> with the values from returned package.location earlier:
az webapp create --resource-group myresourcegroup --plan myplanname --name <app-name> --deployment-

container-image-name <acrinstance>.azurecr.io/package:<imagename>
This command returns information similar to the following JSON document:
{
"adminSiteName": null,
"appServicePlanName": "myplanname",
"geoRegion": "West Europe",
"hostingEnvironmentProfile": null,
"id": "/subscriptions/0000-
0000/resourceGroups/myResourceGroup/providers/Microsoft.Web/serverfarms/myplanname",
"kind": "linux",
"location": "West Europe",
"maximumNumberOfWorkers": 1,
"name": "myplanname",
< JSON data removed for brevity. >
"targetWorkerSizeId": 0,
"type": "Microsoft.Web/serverfarms",
"workerTierName": null
}
IMPORTANT
At this point, the web app has been created. However, since you haven't provided the credentials to the Azure
Container Registry that contains the image, the web app is not active. In the next step, you provide the
authentication information for the container registry.
4. To provide the web app with the credentials needed to access the container registry, use the following
command. Replace <app-name> with the name you want to use. Replace <acrinstance> and <imagename>
with the values from returned package.location earlier. Replace <username> and <password> with the ACR
login information retrieved earlier:
az webapp config container set --name <app-name> --resource-group myresourcegroup --docker-custom-image-

name <acrinstance>.azurecr.io/package:<imagename> --docker-registry-server-url
https://<acrinstance>.azurecr.io --docker-registry-server-user <username> --docker-registry-server-
password <password>
This command returns information similar to the following JSON document:

[
{
"name": "WEBSITES_ENABLE_APP_SERVICE_STORAGE",
"slotSetting": false,
"value": "false"
},
{
"name": "DOCKER_REGISTRY_SERVER_URL",
"value": "https://myml08024f78fd10.azurecr.io"
},
{
"name": "DOCKER_REGISTRY_SERVER_USERNAME",
"value": "myml08024f78fd10"
},
{
"name": "DOCKER_REGISTRY_SERVER_PASSWORD",
"value": null
},
{
"name": "DOCKER_CUSTOM_IMAGE_NAME",
"value": "DOCKER|myml08024f78fd10.azurecr.io/package:20190827195524"
}
]
At this point, the web app begins loading the image.
IMPORTANT
It may take several minutes before the image has loaded. To monitor progress, use the following command:
az webapp log tail --name <app-name> --resource-group myresourcegroup
Once the image has been loaded and the site is active, the log displays a message that states
Container <container name> for site <app-name> initialized successfully and is ready to serve requests .
Once the image is deployed, you can find the hostname by using the following command:
az webapp show --name <app-name> --resource-group myresourcegroup
This command returns information similar to the following hostname - <app-name>.azurewebsites.net . Use this
value as part of the base url for the service.
Use the Web App

The web service that passes requests to the model is located at {baseurl}/score . For example,
https://<app-name>.azurewebsites.net/score . The following Python code demonstrates how to submit data to the
URL and display the response:
import requests
import json
scoring_uri = "https://mywebapp.azurewebsites.net/score"
[1,2,3,4,5,6,7,8,9,10],
[10,9,8,7,6,5,4,3,2,1]
]})
response = requests.post(scoring_uri, data=test_sample, headers=headers)

Next steps
Learn to configure your Web App in the App Service on Linux documentation.
Learn more about scaling in Get started with Autoscale in Azure.
Use an SSL certificate in your Azure App Service.
Configure your App Service app to use Azure Active Directory sign-in.
Deploy a model to Azure Container Instances
Learn how to use the Azure Machine Learning service to deploy a model as a web service on Azure Container
Instances (ACI). Use Azure Container Instances if one of the following conditions is true:
You need to quickly deploy and validate your model. You do not need to create ACI containers ahead of time.
They are created as part of the deployment process.
You are testing a model that is under development.
For information on quota and region availability for ACI, see Quotas and region availability for Azure Container
Instances article.
Prerequisites
A machine learning model registered in your workspace. If you don't have a registered model, see How and
where to deploy models.
The Azure CLI extension for Machine Learning service, Azure Machine Learning Python SDK, or the Azure
Machine Learning Visual Studio Code extension.
The Python code snippets in this article assume that the following variables are set:
ws - Set to your workspace.
model - Set to your registered model.
inference_config - Set to the inference configuration for the model.
For more information on setting these variables, see How and where to deploy models.
The CLI snippets in this article assume that you've created an inferenceconfig.json document. For more
information on creating this document, see How and where to deploy models.
Deploy to ACI
To deploy a model to Azure Container Instances, create a deployment configuration that describes the compute
resources needed. For example, number of cores and memory. You also need an inference configuration, which
describes the environment needed to host the model and web service. For more information on creating the
inference configuration, see How and where to deploy models.
Using the SDK

deployment_config = AciWebservice.deploy_configuration(cpu_cores = 1, memory_gb = 1)

service = Model.deploy(ws, "aciservice", [model], inference_config, deployment_config)
documents:
AciWebservice.deploy_configuration
Model.deploy
Webservice.wait_for_deployment
Using the CLI
To deploy using the CLI, use the following command. Replace mymodel:1 with the name and version of the
registered model. Replace myservice with the name to give this service:
az ml model deploy -m mymodel:1 -n myservice -ic inferenceconfig.json -dc deploymentconfig.json

AciWebservice.deploy_configuration. The following table describes the mapping between the entities in the JSON
document and the parameters for the method:
computeType NA The compute target. For ACI, the value

must be ACI .

entities.
cpu cpu_cores The number of CPU cores to allocate.

Defaults, 0.1

0.5
location location The Azure region to deploy this

Webservice to. If not specified the
Workspace location will be used. More
details on available regions can be
found here: ACI Regions
authEnabled auth_enabled Whether to enable auth for this

Webservice. Defaults to False
sslEnabled ssl_enabled Whether to enable SSL for this

Webservice. Defaults to False.
appInsightsEnabled enable_app_insights Whether to enable AppInsights for this

sslCertificate ssl_cert_pem_file The cert file needed if SSL is enabled
sslKey ssl_key_pem_file The key file needed if SSL is enabled
cname ssl_cname The cname for if SSL is enabled
dnsNameLabel dns_name_label The dns name label for the scoring

endpoint. If not specified a unique dns
name label will be generated for the
scoring endpoint.
{
"computeType": "aci",
{
"cpu": 0.5,
"memoryInGB": 1.0
},
"sslEnabled": false,
"appInsightsEnabled": false
}
Using VS Code
See deploy your models with VS Code.
IMPORTANT
You don't need to create an ACI container to test in advance. ACI containers are created as needed.
Update the web service

To update the web service, use the update method. You can update the web service to use a new model, entry
script, or dependencies that can be specified using an inference configuration. For more information, see the
reference for Webservice.update.
IMPORTANT
Using the SDK

The following code demonstrates how to use the SDK to update the model, environment, and entry script for a
web service:

tags={"key": "0.1"},
description="test",
workspace=ws)


Using the CLI

You can also update a web service using the ML CLI. The following example demonstrates registering a new
model, and then updating web service to use the new model:
az ml model register -n sklearn_mnist --asset-path outputs/sklearn_mnist_model.pkl --experiment-name

myexperiment --output-metadata-file modelinfo.json
TIP
In this example, a JSON document is used to pass the model information from the registration command into the update
command.
To update the service to use a new entry script or environment, create an inference configuration file and specify it with the
ic parameter.
Next steps
Deploy a model to Notebook VMs
Learn how to use the Azure Machine Learning service to deploy a model as a web service on your Notebook VM.
Use Notebook VMs if one of the following conditions is true:
You need to quickly deploy and validate your model.
You are testing a model that is under development.
TIP
Deploying a model from a Jupyter Notebook on a notebook VM, to a web service on the same VM is a local deployment. In
this case, the 'local' computer is the notebook VM. For more information on deployments, see Deploy models with Azure
Prerequisites
An Azure Machine Learning service workspace with a notebook VM running. For more information, see Setup
environment and workspace.
Deploy to the notebook VMs

An example notebook that demonstrates local deployments is included on your notebook VM. Use the following
steps to load the notebook and deploy the model as a web service on the VM:
1. From the Azure portal, select your Azure Machine Learning notebook VMs.
2. Open the samples-* subdirectory, and then open
how-to-use-azureml/deploy-to-local/register-model-deploy-local.ipynb . Once open, run the notebook.
3. The notebook displays the URL and port that the service is running on. For example,
https://localhost:6789 . You can also run the cell containing
print('Local service port: {}'.format(local_service.port)) to display the port.
4. To test the service from the notebook VM, use the https://localhost:<local_service.port> URL. To test
from a remote client, get the public URL of the service running on the Notebook VM The public URL can be
determined use the following formula;
https://<notebookvm_name>-<local_service_port>.<azure_region_of_notebook>.notebooks.azureml.net/score .
For example, https://mynotebookvm-6789.eastus2.notebooks.azureml.net/score .
Test the service

To submit sample data to the running service, use the following code. Replace the value of service_url with the
URL of from the previous step:
import requests
import json
[1,2,3,4,5,6,7,8,9,10],
[10,9,8,7,6,5,4,3,2,1]
]})
test_sample = bytes(test_sample,encoding = 'utf8')
access_token = "your bearer token"
headers = {'Content-Type':'application/json', 'Authorization': 'Bearer ' + access_token}
service_url = "https://mynotebookvm-6789.eastus2.notebooks.azureml.net/score"
resp = requests.post(service_url, test_sample, headers=headers)
Next steps
Deploy a model using a custom Docker base image
Learn how to use a custom Docker base image when deploying trained models with the Azure Machine Learning
service.
When you deploy a trained model to a web service or IoT Edge device, a package is created which contains a web
server to handle incoming requests.
Azure Machine Learning service provides a default Docker base image so you don't have to worry about creating
one. You can also use Azure Machine Learning service environments to select a specific base image, or use a
custom one that you provide.
A base image is used as the starting point when an image is created for a deployment. It provides the underlying
operating system and components. The deployment process then adds additional components, such as your
model, conda environment, and other assets, to the image before deploying it.
Typically, you create a custom base image when you want to use Docker to manage your dependencies, maintain
tighter control over component versions or save time during deployment. For example, you might want to
standardize on a specific version of Python, Conda, or other component. You might also want to install software
required by your model, where the installation process takes a long time. Installing the software when creating
the base image means that you don't have to install it for each deployment.
IMPORTANT
When you deploy a model, you cannot override core components such as the web server or IoT Edge components. These
components provide a known working environment that is tested and supported by Microsoft.
WARNING
Microsoft may not be able to help troubleshoot problems caused by a custom image. If you encounter problems, you may
be asked to use the default image or one of the images Microsoft provides to see if the problem is specific to your image.
This document is broken into two sections:

Create a custom base image: Provides information to admins and DevOps on creating a custom image and
configuring authentication to an Azure Container Registry using the Azure CLI and Machine Learning CLI.
Deploy a model using a custom base image: Provides information to Data Scientists and DevOps / ML
Engineers on using custom images when deploying a trained model from the Python SDK or ML CLI.
Prerequisites
An Azure Machine Learning service workgroup. For more information, see the Create a workspace article.
The Azure CLI.
The CLI extension for Azure Machine Learning.
An Azure Container Registry or other Docker registry that is accessible on the internet.
The steps in this document assume that you are familiar with creating and using an inference configuration
object as part of model deployment. For more information, see the "prepare to deploy" section of Where to
deploy and how.
Create a custom base image

The information in this section assumes that you are using an Azure Container Registry to store Docker images.
Use the following checklist when planning to create custom images for Azure Machine Learning service:
Will you use the Azure Container Registry created for the Azure Machine Learning service workspace, or a
standalone Azure Container Registry?
When using images stored in the container registry for the workspace, you do not need to authenticate
to the registry. Authentication is handled by the workspace.
WARNING
The Azure Container Rzegistry for your workspace is created the first time you train or deploy a model using
the workspace. If you've created a new workspace, but not trained or created a model, no Azure Container Registry
will exist for the workspace.
For information on retrieving the name of the Azure Container Registry for your workspace, see the Get
container registry name section of this article.
When using images stored in a standalone container registry, you will need to configure a service
principal that has at least read access. You then provide the service principal ID (username) and password
to anyone that uses images from the registry. The exception is if you make the container registry publicly
accessible.
For information on creating a private Azure Container Registry, see Create a private container registry.
For information on using service principals with Azure Container Registry, see Azure Container Registry
authentication with service principals.
Azure Container Registry and image information: Provide the image name to anyone that needs to use it.
For example, an image named myimage , stored in a registry named myregistry , is referenced as
myregistry.azurecr.io/myimage when using the image for model deployment
Image requirements: Azure Machine Learning service only supports Docker images that provide the
following software:
Ubuntu 16.04 or greater.
Conda 4.5.# or greater.
Python 3.5.# or 3.6.#.
Get container registry information
In this section, learn how to get the name of the Azure Container Registry for your Azure Machine Learning
service workspace.
WARNING
The Azure Container Registry for your workspace is created the first time you train or deploy a model using the
workspace. If you've created a new workspace, but not trained or created a model, no Azure Container Registry will exist for
the workspace.
If you've already trained or deployed models using the Azure Machine Learning service, a container registry was
created for your workspace. To find the name of this container registry, use the following steps:
1. Open a new shell or command-prompt and use the following command to authenticate to your Azure
subscription:
az login
Follow the prompts to authenticate to the subscription.

2. Use the following command to list the container registry for the workspace. Replace <myworkspace> with
your Azure Machine Learning service workspace name. Replace <resourcegroup> with the Azure resource
group that contains your workspace:
az ml workspace show -w <myworkspace> -g <resourcegroup> --query containerRegistry
TIP
The information returned is similar to the following text:
/subscriptions/<subscription_id>/resourceGroups/<resource_group>/providers/Microsoft.ContainerRegistry
/registries/<registry_name>
The <registry_name> value is the name of the Azure Container Registry for your workspace.
Build a custom base image
The steps in this section walk-through creating a custom Docker image in your Azure Container Registry.
1. Create a new text file named Dockerfile , and use the following text as the contents:
FROM ubuntu:16.04
ARG CONDA_VERSION=4.5.12
ARG PYTHON_VERSION=3.6
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8

ENV PATH /opt/miniconda/bin:$PATH
RUN apt-get update --fix-missing && \

apt-get install -y wget bzip2 && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-${CONDA_VERSION}-Linux-x86_64.sh -O

~/miniconda.sh && \
/bin/bash ~/miniconda.sh -b -p /opt/miniconda && \
rm ~/miniconda.sh && \
/opt/miniconda/bin/conda clean -tipsy
RUN conda install -y conda=${CONDA_VERSION} python=${PYTHON_VERSION} && \

conda clean -aqy && \
rm -rf /opt/miniconda/pkgs && \
find / -type d -name __pycache__ -prune -exec rm -rf {} \;
2. From a shell or command-prompt, use the following to authenticate to the Azure Container Registry.
Replace the <registry_name> with the name of the container registry you want to store the image in:
az acr login --name <registry_name>
3. To upload the Dockerfile, and build it, use the following command. Replace <registry_name> with the
name of the container registry you want to store the image in:
az acr build --image myimage:v1 --registry <registry_name> --file Dockerfile .
During the build process, information is streamed to back to the command line. If the build is successful,
you receive a message similar to the following text:
Run ID: cda was successful after 2m56s
For more information on building images with an Azure Container Registry, see Build and run a container image
using Azure Container Registry Tasks
For more information on uploading existing images to an Azure Container Registry, see Push your first image to
a private Docker container registry.
Use a custom base image

To use a custom image, you need the following information:
The image name. For example, mcr.microsoft.com/azureml/o16n-sample-user-base/ubuntu-miniconda is the
path to a basic Docker Image provided by Microsoft.
If the image is in a private repository, you need the following information:
The registry address. For example, myregistry.azureecr.io .
A service principal username and password that has read access to the registry.
If you do not have this information, speak to the administrator for the Azure Container Registry that
contains your image.
Publicly available base images
Microsoft provides several docker images on a publicly accessible repository, which can be used with the steps in
this section:
IMAGE DESCRIPTION
mcr.microsoft.com/azureml/o16n-sample-user- Basic image for Azure Machine Learning service

base/ubuntu-miniconda
mcr.microsoft.com/azureml/onnxruntime:v0.4.0 Contains the ONNX runtime.
mcr.microsoft.com/azureml/onnxruntime:v0.4.0- Contains the ONNX runtime and CUDA components.

cuda10.0-cudnn7
mcr.microsoft.com/azureml/onnxruntime:v0.4.0- Contains ONNX runtime and TensorRT.

tensorrt19.03
TIP
Since these images are publicly available, you do not need to provide an address, username or password when using them.
IMPORTANT
Microsoft images that use CUDA or TensorRT must be used on Microsoft Azure Services only.
For more information, see Azure Machine Learning service containers.
TIP
If your model is trained on Azure Machine Learning Compute, using version 1.0.22 or greater of the Azure Machine
Learning SDK, an image is created during training. To discover the name of this image, use
run.properties["AzureML.DerivedImageName"] . The following example demonstrates how to use this image:
# Use an image built during training with SDK 1.0.22 or greater

image_config.base_image = run.properties["AzureML.DerivedImageName"]
Use an image with the Azure Machine Learning SDK

To use an image stored in the Azure Container Registry for your workspace, or a container registry that is
publicly accessible, set the following Environment attributes:
docker.enabled=True
docker.base_image : Set to the registry and path to the image.

# Create the environment
# Enable Docker and reference an image
myenv.docker.base_image = "mcr.microsoft.com/azureml/o16n-sample-user-base/ubuntu-miniconda"
To use an image from a private container registry that is not in your workspace, you must use
docker.base_image_registry to specify the address of the repository and a user name and password:
# Set the container registry information

myenv.docker.base_image_repository.address = "myregistry.azurecr.io"
myenv.docker.base_image_repository.username = "username"
myenv.docker.base_image_repository.password = "password"
After defining the environment, use it with an InferenceConfig object to define the inference environment in
which the model and web service will run.

# Use environment in InferenceConfig
environment=myenv)
At this point, you can continue with deployment. For example, the following code snippet would deploy a web
service locally using the inference configuration and custom image:
from azureml.core.webservice import LocalWebservice, Webservice
service = Model.deploy(ws, "myservice", [model], inference_config, deployment_config)
For more information on deployment, see Deploy models with Azure Machine Learning service.
Use an image with the Machine Learning CLI
IMPORTANT
Currently the Machine Learning CLI can use images from the Azure Container Registry for your workspace or publicly
accessible repositories. It cannot use images from standalone private registries.
When deploying a model using the Machine Learning CLI, you provide an inference configuration file that
references the custom image. The following JSON document demonstrates how to reference an image in a
public container registry:
{
"condaFile": "infenv.yml",
"enableGpu": false,
"baseImage": "mcr.microsoft.com/azureml/o16n-sample-user-base/ubuntu-miniconda",
"baseImageRegistry": "mcr.microsoft.com"
}
This file is used with the az ml model deploy command. The --ic parameter is used to specify the inference
configuration file.
az ml model deploy -n myservice -m mymodel:1 --ic inferenceconfig.json --dc deploymentconfig.json --ct

akscomputetarget
For more information on deploying a model using the ML CLI, see the "model registration, profiling, and
deployment" section of the CLI extension for Azure Machine Learning service article.
Next steps
Learn more about Where to deploy and how.
Learn how to Train and deploy machine learning models using Azure Pipelines.
Use an existing model with Azure Machine Learning
service
Learn how to use an existing machine learning model with the Azure Machine Learning service.
If you have a machine learning model that was trained outside the Azure Machine Learning service, you can still
use the service to deploy the model as a web service or to an IoT Edge device.
TIP
This article provides basic information on registering and deploying an existing model. Once deployed, Azure Machine
Learning service provides monitoring for your model. It also allows you to store input data sent to the deployment, which
can be used for data drift analysis or training new versions of the model.
For more information on the concepts and terms used here, see Manage, deploy, and monitor machine learning models.
For general information on the deployment process, see Deploy models with Azure Machine Learning service.
Prerequisites
An Azure Machine Learning service workspace. For more information, see Create a workspace.
TIP
The Python examples in this article assume that the ws variable is set to your Azure Machine Learning service
workspace.
The CLI examples use a placeholder of myworkspace and myresourcegroup . Replace these with the name of your
workspace and the resource group that contains it.

The Azure CLI and Machine Learning CLI extension.
A trained model. The model must be persisted to one or more files on your development environment.
NOTE
To demonstrate registering a model trained outside Azure Machine Learning service, the example code snippets in
this article use the models created by Paolo Ripamonti's Twitter sentiment analysis project:
https://www.kaggle.com/paoloripamonti/twitter-sentiment-analysis.
Register the model(s)

Registering a model allows you to store, version, and track metadata about models in your workspace. In the
following Python and CLI examples, the models directory contains the model.h5 , model.w2v , encoder.pkl , and
tokenizer.pkl files. This example uploads the files contained in the models directory as a new model registration
named sentiment :
# Tip: When model_path is set to a directory, you can use the child_paths parameter to include
# only some of the files from the directory
model = Model.register(model_path = "./models",
model_name = "sentiment",
description = "Sentiment analysis model trained outside Azure Machine Learning
service",
workspace = ws)
For more information, see the Model.register() reference.
az ml model register -p ./models -n sentiment -w myworkspace -g myresourcegroup
For more information, see the az ml model register reference.

For more information on model registration in general, see Manage, deploy, and monitor machine learning
models.
Define inference configuration

The inference configuration defines the environment used to run the deployed model. The inference configuration
references the following entities, which are used to run the model when it's deployed:
An entry script. This file (named score.py ) loads the model when the deployed service starts. It is also
responsible for receiving data, passing it to the model, and then returning a response.
An Azure Machine Learning service environment. An environment defines the software dependencies needed
to run the model and entry script.
The following example shows how to use the SDK to create an environment and then use it with an inference
configuration:

# Create the environment

# Define the packages needed by the model and scripts

conda_dep.add_conda_package("tensorflow")
conda_dep.add_conda_package("numpy")
conda_dep.add_conda_package("scikit-learn")
conda_dep.add_pip_package("keras")
# Adds dependencies to PythonSection of myenv

environment=myenv)
For more information, see the following articles:

How to use environments.
InferenceConfig reference.
The CLI loads the inference configuration from a YAML file:
{
"condaFile": "myenv.yml"
}
With the CLI, the conda environment is defined in the myenv.yml file referenced by the inference configuration.
The following YAML is the contents of this file:
name: inference_environment
dependencies:
- python=3.6.2
- tensorflow
- numpy
- scikit-learn
- pip:
- azureml-defaults
- keras
For more information on inference configuration, see Deploy models with Azure Machine Learning service.
Entry script
The entry script has only two required functions, init() and run(data) . These functions are used to initialize the
service at startup and run the model using request data passed in by a client. The rest of the script handles loading
and running the model(s).
IMPORTANT
There isn't a generic entry script that works for all models. It is always specific to the model that is used. It must understand
how to load the model, the data format that the model expects, and how to score data using the model.
The following Python code is an example entry script ( score.py ):
import pickle
import json
import time
from keras.models import load_model
from keras.preprocessing.sequence import pad_sequences
from gensim.models.word2vec import Word2Vec
# SENTIMENT
POSITIVE = "POSITIVE"
NEGATIVE = "NEGATIVE"
NEUTRAL = "NEUTRAL"
SENTIMENT_THRESHOLDS = (0.4, 0.7)
SEQUENCE_LENGTH = 300
# Called when the deployed service starts

def init():
global model
global tokenizer
global encoder
global w2v_model
# Get the path where the model(s) registered as the name 'sentiment' can be found.
model_path = Model.get_model_path('sentiment')
# load models
model = load_model(model_path + '/model.h5')
w2v_model = Word2Vec.load(model_path + '/model.w2v')
with open(model_path + '/tokenizer.pkl','rb') as handle:
tokenizer = pickle.load(handle)
with open(model_path + '/encoder.pkl','rb') as handle:

encoder = pickle.load(handle)
# Handle requests to the service

def run(data):
try:
# Pick out the text property of the JSON request.
# This expects a request in the form of {"text": "some text to score for sentiment"}
data = json.loads(data)
prediction = predict(data['text'])
#Return prediction
return prediction
error = str(e)
return error
# Determine sentiment from score

def decode_sentiment(score, include_neutral=True):
if include_neutral:
label = NEUTRAL
if score <= SENTIMENT_THRESHOLDS[0]:
label = NEGATIVE
elif score >= SENTIMENT_THRESHOLDS[1]:
label = POSITIVE
return label
else:
return NEGATIVE if score < 0.5 else POSITIVE
# Predict sentiment using the model

def predict(text, include_neutral=True):
start_at = time.time()
# Tokenize text
x_test = pad_sequences(tokenizer.texts_to_sequences([text]), maxlen=SEQUENCE_LENGTH)
# Predict
score = model.predict([x_test])[0]
# Decode sentiment
label = decode_sentiment(score, include_neutral=include_neutral)
return {"label": label, "score": float(score),

"elapsed_time": time.time()-start_at}
For more information on entry scripts, see Deploy models with Azure Machine Learning service.
Define deployment
The Webservice package contains the classes used for deployment. The class you use determines where the model
is deployed. For example, to deploy as a web service on Azure Kubernetes Service, use
AksWebService.deploy_configuration() to create the deployment configuration.
The following Python code defines a deployment configuration for a local deployment. This configuration deploys
the model as a web service to your local computer.
IMPORTANT
A local deployment requires a working installation of Docker on your local computer:
from azureml.core.webservice import LocalWebservice
deployment_config = LocalWebservice.deploy_configuration()
For more information, see the LocalWebservice.deploy_configuration() reference.

The CLI loads the deployment configuration from a YAML file:
{
"computeType": "LOCAL"
}
Deploying to a different compute target, such as Azure Kubernetes Service in the Azure cloud, is as easy as
changing the deployment configuration. For more information, see How and where to deploy models.
Deploy the model

The following example loads information on the registered model named sentiment , and then deploys it as a
service named sentiment . During deployment, the inference configuration and deployment configuration are
used to create and configure the service environment:
model = Model(ws, name='sentiment')

service = Model.deploy(ws, 'myservice', [model], inference_config, deployment_config)
service.wait_for_deployment(True)
print("scoring URI: " + service.scoring_uri)
For more information, see the Model.deploy() reference.

To deploy the model from the CLI, use the following command. This command deploys version 1 of the registered
model ( sentiment:1 ) using the inference and deployment configuration stored in the inferenceConfig.json and
deploymentConfig.json files:
az ml model deploy -n myservice -m sentiment:1 --ic inferenceConfig.json --dc deploymentConfig.json

For more information on deployment, see How and where to deploy models.
Request-response consumption
After deployment, the scoring URI is displayed. This URI can be used by clients to submit requests to the service.
The following example is a basic Python client that submits data to the service and displays the response:
import requests
import json
scoring_uri = 'scoring uri for your service'

test_data = json.dumps({'text': 'Today is a great day!'})
response = requests.post(scoring_uri, data=test_data, headers=headers)

For more information on how to consume the deployed service, see Create a client.
Next steps
How to create a client for a deployed model
What are field-programmable gate arrays (FPGA)
and how to deploy
This article provides an introduction to field-programmable gate arrays (FPGA), and shows you how to deploy
your models using Azure Machine Learning service to an Azure FPGA.
FPGAs contain an array of programmable logic blocks, and a hierarchy of reconfigurable interconnects. The
interconnects allow these blocks to be configured in various ways after manufacturing. Compared to other chips,
FPGAs provide a combination of programmability and performance.
FPGAs vs. CPU, GPU, and ASIC

The following diagram and table show how FPGAs compare to other processors.
PROCESSOR DESCRIPTION
Application-specific integrated circuits ASICs Custom circuits, such as Google's

TensorFlow Processor Units (TPU),
provide the highest efficiency. They
can't be reconfigured as your needs
change.
Field-programmable gate arrays FPGAs FPGAs, such as those available on

Azure, provide performance close to
ASICs. They are also flexible and
reconfigurable over time, to implement
new logic.
Graphics processing units GPUs A popular choice for AI computations.

GPUs offer parallel processing
capabilities, making it faster at image
rendering than CPUs.
PROCESSOR DESCRIPTION
Central processing units CPUs General-purpose processors, the

performance of which isn't ideal for
graphics and video processing.
FPGAs on Azure are based on Intel's FPGA devices, which data scientists and developers use to accelerate real-
time AI calculations. This FPGA-enabled architecture offers performance, flexibility, and scale, and is available on
Azure.
FPGAs make it possible to achieve low latency for real-time inference (or model scoring) requests. Asynchronous
requests (batching) aren't needed. Batching can cause latency, because more data needs to be processed.
Implementations of neural processing units don't require batching; therefore the latency can be many times lower,
compared to CPU and GPU processors.
Reconfigurable power
You can reconfigure FPGAs for different types of machine learning models. This flexibility makes it easier to
accelerate the applications based on the most optimal numerical precision and memory model being used.
Because FPGAs are reconfigurable, you can stay current with the requirements of rapidly changing AI algorithms.
What's supported on Azure

Microsoft Azure is the world's largest cloud investment in FPGAs. Using this FPGA-enabled hardware
architecture, trained neural networks run quickly and with lower latency. Azure can parallelize pre-trained deep
neural networks (DNN ) across FPGAs to scale out your service. The DNNs can be pre-trained, as a deep
featurizer for transfer learning, or fine-tuned with updated weights.
FPGAs on Azure supports:
Image classification and recognition scenarios
TensorFlow deployment
Intel FPGA hardware
These DNN models are currently available:
ResNet 50
ResNet 152
DenseNet-121
VGG -16
SSD -VGG
FPGAs are available in these Azure regions:
East US
Southeast Asia
West Europe
West US 2
IMPORTANT
To optimize latency and throughput, your client sending data to the FPGA model should be in one of the regions above (the
one you deployed the model to).
The PBS Family of Azure VMs contains Intel Arria 10 FPGAs. It will show as "Standard PBS Family vCPUs"
when you check your Azure quota allocation. The PB6 VM has six vCPUs and one FPGA, and it will automatically
be provisioned by Azure ML as part of deploying a model to an FPGA. It is only used with Azure ML, and it
cannot run arbitrary bitstreams. For example, you will not be able to flash the FPGA with bitstreams to do
encryption, encoding, etc.
Scenarios and applications
Azure FPGAs are integrated with Azure Machine Learning. Microsoft uses FPGAs for DNN evaluation, Bing
search ranking, and software defined networking (SDN ) acceleration to reduce latency, while freeing CPUs for
other tasks.
The following scenarios use FPGAs:
Automated optical inspection system
Land cover mapping
Example: Deploy models on FPGAs

You can deploy a model as a web service on FPGAs with Azure Machine Learning Hardware Accelerated Models.
Using FPGAs provides ultra-low latency inference, even with a single batch size. Inference, or model scoring, is the
phase where the deployed model is used for prediction, most commonly on production data.
Prerequisites
An Azure subscription. If you do not have one, create a free account before you begin. Try the free or paid
version of Azure Machine Learning service today.
FPGA quota. Use the Azure CLI to check whether you have quota:
az vm list-usage --location "eastus" -o table --query "[?localName=='Standard PBS Family vCPUs']"
TIP
The other possible locations are southeastasia , westeurope , and westus2 .
The command returns text similar to the following:
CurrentValue Limit LocalName

-------------- ------- -------------------------
0 6 Standard PBS Family vCPUs
Make sure you have at least 6 vCPUs under CurrentValue.

If you do not have quota, then submit a request at https://aka.ms/accelerateAI.
An Azure Machine Learning service workspace and the Azure Machine Learning SDK for Python installed.
For more information, see Create a workspace.
The Python SDK for hardware-accelerated models:
pip install --upgrade azureml-accel-models
1. Create and containerize models

This document will describe how to create a TensorFlow graph to preprocess the input image, make it a featurizer
using ResNet 50 on an FPGA, and then run the features through a classifier trained on the ImageNet data set.
Follow the instructions to:
Define the TensorFlow model
Convert the model
Deploy the model
Consume the deployed model
Delete deployed services
Use the Azure Machine Learning SDK for Python to create a service definition. A service definition is a file
describing a pipeline of graphs (input, featurizer, and classifier) based on TensorFlow. The deployment command
automatically compresses the definition and graphs into a ZIP file, and uploads the ZIP to Azure Blob storage. The
DNN is already deployed to run on the FPGA.
Load Azure ML workspace
Load your Azure ML workspace.
import os
print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep='\n')
Preprocess image
The input to the web service is a JPEG image. The first step is to decode the JPEG image and preprocess it. The
JPEG images are treated as strings and the result are tensors that will be the input to the ResNet 50 model.
# Input images as a two-dimensional tensor containing an arbitrary number of images represented a strings
import azureml.accel.models.utils as utils
tf.reset_default_graph()
in_images = tf.placeholder(tf.string)
image_tensors = utils.preprocess_array(in_images)
print(image_tensors.shape)
Load featurizer
Initialize the model and download a TensorFlow checkpoint of the quantized version of ResNet50 to be used as a
featurizer. You may replace "QuantizedResnet50" in the code snippet below with by importing other deep neural
networks:
QuantizedResnet152
QuantizedVgg16
Densenet121
from azureml.accel.models import QuantizedResnet50

save_path = os.path.expanduser('~/models')
model_graph = QuantizedResnet50(save_path, is_frozen=True)
feature_tensor = model_graph.import_graph_def(image_tensors)
print(model_graph.version)
print(feature_tensor.name)
print(feature_tensor.shape)
Add classifier
This classifier has been trained on the ImageNet data set. Examples for transfer learning and training your
customized weights are available in the set of sample notebooks.
classifier_output = model_graph.get_default_classifier(feature_tensor)
print(classifier_output)
Save the model

Now that the preprocessor, ResNet 50 featurizer, and the classifier have been loaded, save the graph and
associated variables as a model.
model_name = "resnet50"
model_save_path = os.path.join(save_path, model_name)
print("Saving model in {}".format(model_save_path))
with tf.Session() as sess:

model_graph.restore_weights(sess)
tf.saved_model.simple_save(sess, model_save_path,
inputs={'images': in_images},
outputs={'output_alias': classifier_output})
Save input and output tensors

The input and output tensors that were created during the preprocessing and classifier steps will be needed for
model conversion and inference.
input_tensors = in_images.name
output_tensors = classifier_output.name
print(input_tensors)
print(output_tensors)
IMPORTANT
Save the input and output tensors because you will need them for model conversion and inference requests.
The available models and the corresponding default classifier output tensors are below, which is what you would
use for inference if you used the default classifier.
Resnet50, QuantizedResnet50
output_tensors = "classifier_1/resnet_v1_50/predictions/Softmax:0"
Resnet152, QuantizedResnet152
output_tensors = "classifier/resnet_v1_152/predictions/Softmax:0"
Densenet121, QuantizedDensenet121
output_tensors = "classifier/densenet121/predictions/Softmax:0"
Vgg16, QuantizedVgg16
output_tensors = "classifier/vgg_16/fc8/squeezed:0"
SsdVgg, QuantizedSsdVgg
output_tensors = ['ssd_300_vgg/block4_box/Reshape_1:0', 'ssd_300_vgg/block7_box/Reshape_1:0',

'ssd_300_vgg/block8_box/Reshape_1:0', 'ssd_300_vgg/block9_box/Reshape_1:0',
'ssd_300_vgg/block10_box/Reshape_1:0', 'ssd_300_vgg/block11_box/Reshape_1:0',
'ssd_300_vgg/block4_box/Reshape:0', 'ssd_300_vgg/block7_box/Reshape:0',
'ssd_300_vgg/block8_box/Reshape:0', 'ssd_300_vgg/block9_box/Reshape:0',
'ssd_300_vgg/block10_box/Reshape:0', 'ssd_300_vgg/block11_box/Reshape:0']
Register model
Register the model by using the SDK with the ZIP file in Azure Blob storage. Adding tags and other metadata
about the model helps you keep track of your trained models.
registered_model = Model.register(workspace=ws,
model_path=model_save_path,
model_name=model_name)
print("Successfully registered: ", registered_model.name,

registered_model.description, registered_model.version, sep='\t')
If you've already registered a model and want to load it, you may retrieve it.

model_name = "resnet50"
# By default, the latest version is retrieved. You can specify the version, i.e. version=1
registered_model = Model(ws, name="resnet50")
print(registered_model.name, registered_model.description,
registered_model.version, sep='\t')
Convert model
Convert the TensorFlow graph to the Open Neural Network Exchange format (ONNX). You will need to provide
the names of the input and output tensors, and these names will be used by your client when you consume the
web service.
from azureml.accel import AccelOnnxConverter
convert_request = AccelOnnxConverter.convert_tf_model(
ws, registered_model, input_tensors, output_tensors)
# If it fails, you can run wait_for_completion again with show_output=True.

convert_request.wait_for_completion(show_output=False)
# If the above call succeeded, get the converted model

converted_model = convert_request.result
print("\nSuccessfully converted: ", converted_model.name, converted_model.url, converted_model.version,
converted_model.id, converted_model.created_time, '\n')
Create Docker image

The converted model and all dependencies are added to a Docker image. This Docker image can then be deployed
and instantiated. Supported deployment targets include AKS in the cloud or an edge device such as Azure Data
Box Edge. You can also add tags and descriptions for your registered Docker image.
from azureml.core.image import Image
from azureml.accel import AccelContainerImage
image_config = AccelContainerImage.image_configuration()
# Image name must be lowercase
image_name = "{}-image".format(model_name)
image = Image.create(name=image_name,
models=[converted_model],
image_config=image_config,
workspace=ws)
image.wait_for_creation(show_output=False)
List the images by tag and get the detailed logs for any debugging.
for i in Image.list(workspace=ws):
print('{}(v.{} [{}]) stored at {} with build log {}'.format(
i.name, i.version, i.creation_state, i.image_location, i.image_build_log_uri))
2. Deploy to cloud or edge

Deploy to the cloud
To deploy your model as a high-scale production web service, use Azure Kubernetes Service (AKS ). You can create
a new one using the Azure Machine Learning SDK, CLI, or the Azure portal.
# Specify the Standard_PB6s Azure VM

prov_config = AksCompute.provisioning_configuration(vm_size="Standard_PB6s",
agent_count=1)
aks_name = 'my-aks-cluster'
name=aks_name,
provisioning_configuration=prov_config)
The AKS deployment may take around 15 minutes. Check to see if the deployment succeeded.
aks_target.wait_for_completion(show_output=True)
print(aks_target.provisioning_state)
print(aks_target.provisioning_errors)
Deploy the container to the AKS cluster.

from azureml.core.webservice import Webservice, AksWebservice
# For this deployment, set the web service configuration without enabling auto-scaling or authentication for
testing
aks_config = AksWebservice.deploy_configuration(autoscale_enabled=False,
num_replicas=1,
auth_enabled=False)
aks_service_name = 'my-aks-service'
aks_service = Webservice.deploy_from_image(workspace=ws,
name=aks_service_name,
image=image,
deployment_config=aks_config,
deployment_target=aks_target)
Test the cloud service

The Docker image supports gRPC and the TensorFlow Serving "predict" API. Use the sample client to call into the
Docker image to get predictions from the model. Sample client code is available:
Python
C#
If you want to use TensorFlow Serving, you can download a sample client.
# Using the grpc client in Azure ML Accelerated Models SDK package

from azureml.accel import PredictionClient
address = aks_service.scoring_uri
ssl_enabled = address.startswith("https")
address = address[address.find('/')+2:].strip('/')
port = 443 if ssl_enabled else 80
# Initialize AzureML Accelerated Models client

client = PredictionClient(address=address,
port=port,
use_ssl=ssl_enabled,
service_name=aks_service.name)
Since this classifier was trained on the ImageNet data set, map the classes to human-readable labels.
import requests
classes_entries = requests.get(
"https://raw.githubusercontent.com/Lasagne/Recipes/master/examples/resnet50/imagenet_classes.txt").text.splitl
ines()
# Score image with input and output tensor names

results = client.score_file(path="./snowleopardgaze.jpg",
input_name=input_tensors,
outputs=output_tensors)
# map results [class_id] => [confidence]

results = enumerate(results)
# sort results by confidence
sorted_results = sorted(results, key=lambda x: x[1], reverse=True)
# print top 5 results
for top in sorted_results[:5]:
print(classes_entries[top[0]], 'confidence:', top[1])
Clean-up the service

Delete your web service, image, and model (must be done in this order since there are dependencies).
aks_service.delete()
aks_target.delete()
image.delete()
registered_model.delete()
converted_model.delete()
Deploy to a local edge server

All Azure Data Box Edge devices contain an FPGA for running the model. Only one model can be running on the
FPGA at one time. To run a different model, just deploy a new container. Instructions and sample code can be
found in this Azure Sample.
Secure FPGA web services

To secure your FPGA web services, see the Secure web services document.
Next steps
Check out these notebooks, videos, and blogs:
Several sample notebooks.
Hyperscale hardware: ML at scale on top of Azure + FPGA : Build 2018 (video)
Inside the Microsoft FPGA-based configurable cloud (video)
Project Brainwave for real-time AI: project home page
Troubleshooting Azure Machine Learning service
Azure Kubernetes Service and Azure Container
Instances deployment
Learn how to work around or solve common Docker deployment errors with Azure Container Instances (ACI) and
Azure Kubernetes Service (AKS ) using Azure Machine Learning service.
When deploying a model in Azure Machine Learning service, the system performs a number of tasks. The
deployment tasks are:
1. Register the model in the workspace model registry.
2. Build a Docker image, including:
a. Download the registered model from the registry.
b. Create a dockerfile, with a Python environment based on the dependencies you specify in the
environment yaml file.
c. Add your model files and the scoring script you supply in the dockerfile.
d. Build a new Docker image using the dockerfile.
e. Register the Docker image with the Azure Container Registry associated with the workspace.
IMPORTANT
Depending on your code, image creation happen automatically without your input.
3. Deploy the Docker image to Azure Container Instance (ACI) service or to Azure Kubernetes Service (AKS ).
4. Start up a new container (or containers) in ACI or AKS.
Learn more about this process in the Model Management introduction.
Before you begin

If you run into any issue, the first thing to do is to break down the deployment task (previous described) into
individual steps to isolate the problem.
Breaking the deployment into tasks is helpful if you are using the Webservice.deploy() API, or
Webservice.deploy_from_model() API, as both of these functions perform the aforementioned steps as a single
action. Typically those APIs are convenient, but it helps to break up the steps when troubleshooting by replacing
them with the below API calls.
1. Register the model. Here's some sample code:
# register a model out of a run record

model = best_run.register_model(model_name='my_best_model', model_path='outputs/my_model.pkl')
# or, you can register a file or a folder of files as a model

model = Model.register(model_path='my_model.pkl', model_name='my_best_model', workspace=ws)
2. Build the image. Here's some sample code:

# configure the image
image_config = ContainerImage.image_configuration(runtime="python",
# create the image

image = Image.create(name='myimg', models=[model], image_config=image_config, workspace=ws)
# wait for image creation to finish

image.wait_for_creation(show_output=True)
3. Deploy the image as service. Here's some sample code:
# configure an ACI-based deployment

aci_config = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1)
aci_service = Webservice.deploy_from_image(deployment_config=aci_config,
image=image,
name='mysvc',
workspace=ws)
aci_service.wait_for_deployment(show_output=True)
Once you have broken down the deployment process into individual tasks, we can look at some of the most
common errors.
Image building fails

If the Docker image cannot be built, the image.wait_for_creation() or service.wait_for_deployment() call fails with
some error messages that can offer some clues. You can also find out more details about the errors from the
image build log. Below is some sample code showing how to discover the image build log uri.
# if you already have the image object handy

print(image.image_build_log_uri)
# if you only know the name of the image (note there might be multiple images with the same name but
different version number)
print(ws.images['myimg'].image_build_log_uri)
# list logs for all images in the workspace

for name, img in ws.images.items():
print(img.name, img.version, img.image_build_log_uri)
The image log uri is a SAS URL pointing to a log file stored in your Azure blob storage. Simply copy and paste
the uri into a browser window and you can download and view the log file.
Azure Key Vault access policy and Azure Resource Manager templates
The image build can also fail due to a problem with the access policy on Azure Key Vault. This situation can occur
when you use an Azure Resource Manager template to create the workspace and associated resources (including
Azure Key Vault), multiple times. For example, using the template multiple times with the same parameters as part
of a continuous integration and deployment pipeline.
Most resource creation operations through templates are idempotent, but Key Vault clears the access policies each
time the template is used. Clearing the access policies breaks access to the Key Vault for any existing workspace
that is using it. This condition results in errors when you try to create new images. The following are examples of
the errors that you can receive:
Portal:
Create image "myimage": An internal server error occurred. Please try again. If the problem persists, contact
support.
SDK:
image = ContainerImage.create(name = "myimage", models = [model], image_config = image_config, workspace =

ws)
Creating image
Traceback (most recent call last):
File "C:\Python37\lib\site-packages\azureml\core\image\image.py", line 341, in create
resp.raise_for_status()
File "C:\Python37\lib\site-packages\requests\models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url:
https://eastus.modelmanagement.azureml.net/api/subscriptions/<subscription-id>/resourceGroups/<resource-
group>/providers/Microsoft.MachineLearningServices/workspaces/<workspace-name>/images?api-version=2018-11-19
Traceback (most recent call last):

File "<stdin>", line 1, in <module>
File "C:\Python37\lib\site-packages\azureml\core\image\image.py", line 346, in create
'Content: {}'.format(resp.status_code, resp.headers, resp.content))
azureml.exceptions._azureml_exception.WebserviceException: Received bad response from Model Management
Service:
Response Code: 500
Headers: {'Date': 'Tue, 26 Feb 2019 17:47:53 GMT', 'Content-Type': 'application/json', 'Transfer-Encoding':
'chunked', 'Connection': 'keep-alive', 'api-supported-versions': '2018-03-01-preview, 2018-11-19', 'x-ms-
client-request-id': '3cdcf791f1214b9cbac93076ebfb5167', 'x-ms-client-session-id': '', 'Strict-Transport-
Security': 'max-age=15724800; includeSubDomains; preload'}
Content: b'{"code":"InternalServerError","statusCode":500,"message":"An internal server error occurred.
Please try again. If the problem persists, contact support"}'
CLI:
ERROR: {'Azure-cli-ml Version': None, 'Error': WebserviceException('Received bad response from Model
Management Service:\nResponse Code: 500\nHeaders: {\'Date\': \'Tue, 26 Feb 2019 17:34:05
GMT\', \'Content-Type\': \'application/json\', \'Transfer-Encoding\': \'chunked\', \'Connection\': \'keep-
alive\', \'api-supported-versions\': \'2018-03-01-preview, 2018-11-19\', \'x-ms-client-request-id\':
\'bc89430916164412abe3d82acb1d1109\', \'x-ms-client-session-id\': \'\', \'Strict-Transport-Security\': \'max-
age=15724800; includeSubDomains; preload\'}\nContent:
b\'{"code":"InternalServerError","statusCode":500,"message":"An internal server error occurred. Please try
again. If the problem persists, contact support"}\'',)}
To avoid this problem, we recommend one of the following approaches:

Do not deploy the template more than once for the same parameters. Or delete the existing resources before
using the template to recreate them.
Examine the Key Vault access policies and then use these policies to set the accessPolicies property of the
template.
Check if the Key Vault resource already exists. If it does, do not recreate it through the template. For example,
add a parameter that allows you to disable the creation of the Key Vault resource if it already exists.
Debug locally
If you encounter problems deploying a model to ACI or AKS, try deploying it as a local web service. Using a local
web service makes it easier to troubleshoot problems. The Docker image containing the model is downloaded
and started on your local system.
IMPORTANT
Local web service deployments require a working Docker installation on your local system. Docker must be running before
you deploy a local web service. For information on installing and using Docker, see https://www.docker.com/.
WARNING
Local web service deployments are not supported for production scenarios.
To deploy locally, modify your code to use LocalWebservice.deploy_configuration() to create a deployment

configuration. Then use Model.deploy() to deploy the service. The following example deploys a model (contained
in the model variable) as a local web service:
from azureml.core.model import InferenceConfig, Model

from azureml.core.webservice import LocalWebservice
# Create inference configuration. This creates a docker image that contains the model.
# Create a local deployment, using port 8890 for the web service endpoint
# Deploy the service
service = Model.deploy(
ws, "mymodel", [model], inference_config, deployment_config)
# Wait for the deployment to complete
service.wait_for_deployment(True)
# Display the port that the web service is available on
print(service.port)
At this point, you can work with the service as normal. For example, the following code demonstrates sending
data to the service:
import json
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
[10, 9, 8, 7, 6, 5, 4, 3, 2, 1]
]})
test_sample = bytes(test_sample, encoding='utf8')
prediction = service.run(input_data=test_sample)
print(prediction)
Update the service

During local testing, you may need to update the score.py file to add logging or attempt to resolve any problems
that you've discovered. To reload changes to the score.py file, use reload() . For example, the following code
reloads the script for the service, and then sends data to it. The data is scored using the updated score.py file:
IMPORTANT
The reload method is only available for local deployments. For information on updating a deployment to another
compute target, see the update section of Deploy models.
service.reload()
print(service.run(input_data=test_sample))
NOTE
The script is reloaded from the location specified by the InferenceConfig object used by the service.
To change the model, Conda dependencies, or deployment configuration, use update(). The following example
updates the model used by the service:
service.update([different_model], inference_config, deployment_config)
Delete the service

To delete the service, use delete().
Inspect the Docker log
You can print out detailed Docker engine log messages from the service object. You can view the log for ACI, AKS,
and Local deployments. The following example demonstrates how to print the logs.
# if you already have the service object handy

# if you only know the name of the service (note there might be multiple services with the same name but
different version number)
print(ws.webservices['mysvc'].get_logs())
Service launch fails

After the image is successfully built, the system attempts to start a container using your deployment
configuration. As part of container starting-up process, the init() function in your scoring script is invoked by
the system. If there are uncaught exceptions in the init() function, you might see CrashLoopBackOff error in
the error message.
Use the info in the Inspect the Docker log section to check the logs.
Function fails: get_model_path()

Often, in the init() function in the scoring script, Model.get_model_path() function is called to locate a model
file or a folder of model files in the container. If the model file or folder cannot be found, the function fails. The
easiest way to debug this error is to run the below Python code in the Container shell:

import logging
logging.basicConfig(level=logging.DEBUG)
print(Model.get_model_path(model_name='my-best-model'))
This example prints out the local path (relative to /var/azureml-app ) in the container where your scoring script is
expecting to find the model file or folder. Then you can verify if the file or folder is indeed where it is expected to
be.
Setting the logging level to DEBUG may cause additional information to be logged, which may be useful in
identifying the failure.
Function fails: run(input_data)
If the service is successfully deployed, but it crashes when you post data to the scoring endpoint, you can add
error catching statement in your run(input_data) function so that it returns detailed error message instead. For
example:
def run(input_data):
try:
data = json.loads(input_data)['data']
data = np.array(data)
return json.dumps({"result": result.tolist()})
result = str(e)
# return error message back to the client
return json.dumps({"error": result})
Note: Returning error messages from the run(input_data) call should be done for debugging purpose only. For
security reasons, you should not return error messages this way in a production environment.
HTTP status code 503

Azure Kubernetes Service deployments support autoscaling, which allows replicas to be added to support
additional load. However, the autoscaler is designed to handle gradual changes in load. If you receive large
spikes in requests per second, clients may receive an HTTP status code 503.
There are two things that can help prevent 503 status codes:
Change the utilization level at which autoscaling creates new replicas.
By default, autoscaling target utilization is set to 70%, which means that the service can handle spikes in
requests per second (RPS ) of up to 30%. You can adjust the utilization target by setting the
autoscale_target_utilization to a lower value.
IMPORTANT
This change does not cause replicas to be created faster. Instead, they are created at a lower utilization threshold.
Instead of waiting until the service is 70% utilized, changing the value to 30% causes replicas to be created when
30% utilization occurs.
If the web service is already using the current max replicas and you are still seeing 503 status codes,
increase the autoscale_max_replicas value to increase the maximum number of replicas.
Change the minimum number of replicas. Increasing the minimum replicas provides a larger pool to
handle the incoming spikes.
To increase the minimum number of replicas, set autoscale_min_replicas to a higher value. You can
calculate the required replicas by using the following code, replacing values with values specific to your
project:
from math import ceil
# target requests per second
targetRps = 20
# time to process the request (in seconds)
reqTime = 10
# Maximum requests per container
maxReqPerContainer = 1
# target_utilization. 70% in this example
targetUtilization = .7
concurrentRequests = targetRps * reqTime / targetUtilization
# Number of container replicas

replicas = ceil(concurrentRequests / maxReqPerContainer)
NOTE
If you receive request spikes larger than the new minimum replicas can handle, you may receive 503s again. For
example, as traffic to your service increases, you may need to increase the minimum replicas.
For more information on setting autoscale_target_utilization , autoscale_max_replicas , and

autoscale_min_replicas for, see the AksWebservice module reference.
Advanced debugging
In some cases, you may need to interactively debug the Python code contained in your model deployment. For
example, if the entry script is failing and the reason cannot be determined by additional logging. By using Visual
Studio Code and the Python Tools for Visual Studio (PTVSD ), you can attach to the code running inside the
Docker container.
IMPORTANT
This method of debugging does not work when using Model.deploy() and LocalWebservice.deploy_configuration to
deploy a model locally. Instead, you must create an image using the ContainerImage class.
Local web service deployments require a working Docker installation on your local system. Docker must be running before
you deploy a local web service. For information on installing and using Docker, see https://www.docker.com/.
Configure development environment

1. To install the Python Tools for Visual Studio (PTVSD ) on your local VS Code development environment,
use the following command:
python -m pip install --upgrade ptvsd
For more information on using PTVSD with VS Code, see Remote Debugging.
2. To configure VS Code to communicate with the Docker image, create a new debug configuration:
a. From VS Code, select the Debug menu and then select Open configurations. A file named
launch.json opens.
b. In the launch.json file, find the line that contains "configurations": [ , and insert the following text
after it:
{
"name": "Azure Machine Learning service: Docker Debug",
"type": "python",
"request": "attach",
"port": 5678,
"host": "localhost",
"pathMappings": [
{
"localRoot": "${workspaceFolder}",
"remoteRoot": "/var/azureml-app"
}
]
}
IMPORTANT
If there are already other entries in the configurations section, add a comma (,) after the code that you
inserted.
This section attaches to the Docker container using port 5678.

c. Save the launch.json file.
Create an image that includes PTVSD
1. Modify the conda environment for your deployment so that it includes PTVSD. The following example
demonstrates adding it using the pip_packages parameter:
# Usually a good idea to choose specific version numbers

# so training is made on same packages as scoring
myenv = CondaDependencies.create(conda_packages=['numpy==1.15.4',
'scikit-learn==0.19.1', 'pandas==0.23.4'],
pip_packages = ['azureml-defaults==1.0.17', 'ptvsd'])
with open("myenv.yml","w") as f:
2. To start PTVSD and wait for a connection when the service starts, add the following to the top of your
score.py file:
import ptvsd
# Allows other computers to attach to ptvsd on this IP address and port.
ptvsd.enable_attach(address=('0.0.0.0', 5678), redirect_output = True)
# Wait 30 seconds for a debugger to attach. If none attaches, the script continues as normal.
ptvsd.wait_for_attach(timeout = 30)
print("Debugger attached...")
3. During debugging, you may want to make changes to the files in the image without having to recreate it.
To install a text editor (vim) in the Docker image, create a new text file named Dockerfile.steps and use
the following as the contents of the file:
RUN apt-get update && apt-get -y install vim
A text editor allows you to modify the files inside the docker image to test changes without creating a new
image.
4. To create an image that uses the Dockerfile.steps file, use the docker_file parameter when creating an
image. The following example demonstrates how to do this:
NOTE
This example assumes that ws points to your Azure Machine Learning workspace, and that model is the model
being deployed. The myenv.yml file contains the conda dependencies created in step 1.
from azureml.core.image import Image, ContainerImage

image_config = ContainerImage.image_configuration(runtime= "python",
execution_script="score.py",
conda_file="myenv.yml",
docker_file="Dockerfile.steps")
image = Image.create(name = "myimage",

models = [model],
image_config = image_config,
workspace = ws)
# Print the location of the image in the repository
print(image.image_location)
Once the image has been created, the image location in the registry is displayed. The location is similar to the
following text:
myregistry.azurecr.io/myimage:1
In this text example, the registry name is myregistry and the image is named myimage . The image version is 1 .
Download the image
1. Open a command prompt, terminal, or other shell and use the following Azure CLI command to
authenticate to the Azure subscription that contains your Azure Machine Learning workspace:
az login
2. To authenticate to the Azure Container Registry (ACR ) that contains your image, use the following
command. Replace myregistry with the one returned when you registered the image:
az acr login --name myregistry
3. To download the image to your local Docker, use the following command. Replace myimagepath with the
location returned when you registered the image:
docker pull myimagepath
The image path should be similar to myregistry.azurecr.io/myimage:1 . Where myregistry is your registry,
myimage is your image, and 1 is the image version.
TIP
The authentication from the previous step does not last forever. If you wait long enough between the authentication
command and the pull command, you will receive an authentication failure. If this happens, reauthenticate.
The time it takes to complete the download depends on the speed of your internet connection. A download
status is displayed during the process. Once the download is complete, you can use the docker images
command to verify that it has downloaded.
4. To make it easier to work with the image, use the following command to add a tag. Replace myimagepath
with the location value from step 2.
docker tag myimagepath debug:1
For the rest of the steps, you can refer to the local image as debug:1 instead of the full image path value.
Debug the service
TIP
If you set a timeout for the PTVSD connection in the score.py file, you must connect VS Code to the debug session
before the timeout expires. Start VS Code, open the local copy of score.py , set a breakpoint, and have it ready to go
before using the steps in this section.
For more information on debugging and setting breakpoints, see Debugging.
1. To start a Docker container using the image, use the following command:
docker run --rm --name debug -p 8000:5001 -p 5678:5678 debug:1
2. To attach VS Code to PTVSD inside the container, open VS Code and use the F5 key or select Debug.
When prompted, select the Azure Machine Learning service: Docker Debug configuration. You can
also select the debug icon from the side bar, the Azure Machine Learning service: Docker Debug entry
from the Debug dropdown menu, and then use the green arrow to attach the debugger.
At this point, VS Code connects to PTVSD inside the Docker container and stops at the breakpoint you set
previously. You can now step through the code as it runs, view variables, etc.
For more information on using VS Code to debug Python, see Debug your Python code.
Modify the container files
To make changes to files in the image, you can attach to the running container, and execute a bash shell. From
there, you can use vim to edit files:
1. To connect to the running container and launch a bash shell in the container, use the following command:
docker exec -it debug /bin/bash
2. To find the files used by the service, use the following command from the bash shell in the container:
cd /var/azureml-app
From here, you can use vim to edit the score.py file. For more information on using vim, see Using the
Vim editor.
3. Changes to a container are not normally persisted. To save any changes you make, use the following
command, before you exit the shell started in the step above (that is, in another shell):
docker commit debug debug:2
This command creates a new image named debug:2 that contains your edits.
TIP
You will need to stop the current container and start using the new version before changes take effect.
4. Make sure to keep the changes you make to files in the container in sync with the local files that VS Code
uses. Otherwise, the debugger experience will not work as expected.
Stop the container
To stop the container, use the following command:
docker stop debug
Next steps
Learn more about deployment:
How to deploy and where
Tutorial: Train & deploy models
Consume an Azure Machine Learning model
deployed as a web service
Deploying an Azure Machine Learning model as a web service creates a REST API. You can send data to this
API and receive the prediction returned by the model. In this document, learn how to create clients for the web
service by using C#, Go, Java, and Python.
You create a web service when you deploy an image to Azure Container Instances, Azure Kubernetes Service,
or field-programmable gate arrays (FPGA). You create images from registered models and scoring files. You
retrieve the URI used to access a web service by using the Azure Machine Learning SDK. If authentication is
enabled, you can also use the SDK to get the authentication keys or tokens.
The general workflow for creating a client that uses a machine learning web service is:
1. Use the SDK to get the connection information.
2. Determine the type of request data used by the model.
3. Create an application that calls the web service.
TIP
The examples in this document are manually created without the use of OpenAPI (Swagger) specifications. If you've
enabled an OpenAPI specification for your deployment, you can use tools such as swagger-codegen to create client
libraries for your service.
Connection information
NOTE
Use the Azure Machine Learning SDK to get the web service information. This is a Python SDK. You can use any
language to create a client for the service.
The azureml.core.Webservice class provides the information you need to create a client. The following
Webservice properties are useful for creating a client application:
auth_enabled - If key authentication is enabled, True ; otherwise, False .

token_auth_enabled - If token authentication is enabled, True ; otherwise, False .
scoring_uri - The REST API address.
swagger_uri - The address of the OpenAPI specification. This URI is available if you enabled automatic
schema generation. For more information, see Deploy models with the Azure Machine Learning service.
There are a three ways to retrieve this information for deployed web services:
When you deploy a model, a Webservice object is returned with information about the service:
service = Webservice.deploy_from_model(name='myservice',
deployment_config=myconfig,
models=[model],
image_config=image_config,
workspace=ws)
print(service.swagger_uri)
You can use Webservice.list to retrieve a list of deployed web services for models in your workspace.
You can add filters to narrow the list of information returned. For more information about what can be
filtered on, see the Webservice.list reference documentation.
services = Webservice.list(ws)
print(services[0].scoring_uri)
print(services[0].swagger_uri)
If you know the name of the deployed service, you can create a new instance of Webservice , and
provide the workspace and service name as parameters. The new object contains information about the
deployed service.
service = Webservice(workspace=ws, name='myservice')

print(service.swagger_uri)
Authentication for services

Azure Machine Learning provides two ways to control access to your web services.
AUTHENTICATION METHOD ACI AKS
Key Disabled by default Enabled by default
Token Not Available Disabled by default
When sending a request to a service that is secured with a key or token, use the Authorization header to pass
the key or token. The key or token must be formatted as Bearer <key-or-token> , where <key-or-token> is your
key or token value.
When you enable authentication for a deployment, you automatically create authentication keys.
Authentication is enabled by default when you are deploying to Azure Kubernetes Service.
Authentication is disabled by default when you are deploying to Azure Container Instances.
To control authentication, use the auth_enabled parameter when you are creating or updating a deployment.
If authentication is enabled, you can use the get_keys method to retrieve a primary and secondary
authentication key:

print(primary)
IMPORTANT
If you need to regenerate a key, use service.regen_key .

When you enable token authentication for a web service, a user must provide an Azure Machine Learning
JWT token to the web service to access it.
Token authentication is disabled by default when you are deploying to Azure Kubernetes Service.
Token authentication is not supported when you are deploying to Azure Container Instances.
To control token authentication, use the token_auth_enabled parameter when you are creating or updating a
deployment.
If token authentication is enabled, you can use the get_token method to retrieve a bearer token and that
tokens expiration time:

print(token)
IMPORTANT
You will need to request a new token after the token's refresh_by time.
Request data
The REST API expects the body of the request to be a JSON document with the following structure:
{
"data":
[
<model-specific-data-structure>
]
}
IMPORTANT
The structure of the data needs to match what the scoring script and model in the service expect. The scoring script
might modify the data before passing it to the model.
For example, the model in the Train within notebook example expects an array of 10 numbers. The scoring
script for this example creates a Numpy array from the request, and passes it to the model. The following
example shows the data this service expects:
{
"data":
[
[
0.0199132141783263,
0.0506801187398187,
0.104808689473925,
0.0700725447072635,
-0.0359677812752396,
-0.0266789028311707,
-0.0249926566315915,
-0.00259226199818282,
0.00371173823343597,
0.0403433716478807
]
]
}
The web service can accept multiple sets of data in one request. It returns a JSON document containing an
array of responses.
Binary data
For information on how to enable support for binary data in your service, see Binary data.
Cross-origin resource sharing (CORS )
For information on enabling CORS support in your service, see Cross-origin resource sharing.
Call the service (C#)

This example demonstrates how to use C# to call the web service created from the Train within notebook
example:
using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Net.Http.Headers;
using Newtonsoft.Json;
namespace MLWebServiceClient
{
// The data structure expected by the service
internal class InputData
{
[JsonProperty("data")]
// The service used by this example expects an array containing
// one or more arrays of doubles
internal double[,] data;
}
class Program
{
static void Main(string[] args)
{
// Set the scoring URI and authentication key or token
string scoringUri = "<your web service URI>";
string authKey = "<your key or token>";
// Set the data to be sent to the service.

// In this case, we are sending two sets of data to be scored.
InputData payload = new InputData();
payload.data = new double[,] {
{
0.0199132141783263,
0.0506801187398187,
0.104808689473925,
0.0700725447072635,
-0.0359677812752396,
-0.0266789028311707,
-0.0249926566315915,
-0.00259226199818282,
0.00371173823343597,
0.0403433716478807
},
{
-0.0127796318808497,
-0.044641636506989,
0.0606183944448076,
0.0528581912385822,
0.0479653430750293,
0.0293746718291555,
-0.0176293810234174,
0.0343088588777263,
0.0702112981933102,
0.00720651632920303
}
};
// Create the HTTP client

HttpClient client = new HttpClient();
// Set the auth header. Only needed if the web service requires authentication.
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", authKey);
// Make the request

try {
var request = new HttpRequestMessage(HttpMethod.Post, new Uri(scoringUri));
request.Content = new StringContent(JsonConvert.SerializeObject(payload));
request.Content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
var response = client.SendAsync(request).Result;
// Display the response from the web service
Console.WriteLine(response.Content.ReadAsStringAsync().Result);
}
catch (Exception e)
{
Console.Out.WriteLine(e.Message);
}
}
}
}
The results returned are similar to the following JSON document:
[217.67978776218715, 224.78937091757172]
Call the service (Go)

This example demonstrates how to use Go to call the web service created from the Train within notebook
example:
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
)
// Features for this model are an array of decimal values
type Features []float64
// The web service input can accept multiple sets of values for scoring
type InputData struct {
Data []Features `json:"data",omitempty`
}
// Define some example data

var exampleData = []Features{
[]float64{
0.0199132141783263,
0.0506801187398187,
0.104808689473925,
0.0700725447072635,
-0.0359677812752396,
-0.0266789028311707,
-0.0249926566315915,
-0.00259226199818282,
0.00371173823343597,
0.0403433716478807,
},
[]float64{
-0.0127796318808497,
-0.044641636506989,
0.0606183944448076,
0.0528581912385822,
0.0479653430750293,
0.0293746718291555,
-0.0176293810234174,
0.0343088588777263,
0.0702112981933102,
0.00720651632920303,
},
}
// Set to the URI for your service

var serviceUri string = "<your web service URI>"
// Set to the authentication key or token (if any) for your service
var authKey string = "<your key or token>"
func main() {
// Create the input data from example data
jsonData := InputData{
Data: exampleData,
}
// Create JSON from it and create the body for the HTTP request
jsonValue, _ := json.Marshal(jsonData)
body := bytes.NewBuffer(jsonValue)
// Create the HTTP request

client := &http.Client{}
request, err := http.NewRequest("POST", serviceUri, body)
request.Header.Add("Content-Type", "application/json")
// These next two are only needed if using an authentication key

bearer := fmt.Sprintf("Bearer %v", authKey)
request.Header.Add("Authorization", bearer)
// Send the request to the web service

resp, err := client.Do(request)
if err != nil {
fmt.Println("Failure: ", err)
}
// Display the response received

respBody, _ := ioutil.ReadAll(resp.Body)
fmt.Println(string(respBody))
fmt.Println(string(respBody))
}
[217.67978776218715, 224.78937091757172]
Call the service (Java)

This example demonstrates how to use Java to call the web service created from the Train within notebook
example:
import java.io.IOException;
import org.apache.http.client.fluent.*;
import org.apache.http.entity.ContentType;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
public class App {

// Handle making the request
public static void sendRequest(String data) {
// Replace with the scoring_uri of your service
String uri = "<your web service URI>";
// If using authentication, replace with the auth key or token
String key = "<your key or token>";
try {
// Create the request
Content content = Request.Post(uri)
.addHeader("Content-Type", "application/json")
// Only needed if using authentication
.addHeader("Authorization", "Bearer " + key)
// Set the JSON data as the body
.bodyString(data, ContentType.APPLICATION_JSON)
// Make the request and display the response.
.execute().returnContent();
System.out.println(content);
}
catch (IOException e) {
System.out.println(e);
}
}
public static void main(String[] args) {
// Create the data to send to the service
JSONObject obj = new JSONObject();
// In this case, it's an array of arrays
JSONArray dataItems = new JSONArray();
// Inner array has 10 elements
JSONArray item1 = new JSONArray();
item1.add(0.0199132141783263);
item1.add(0.0506801187398187);
item1.add(0.104808689473925);
item1.add(0.0700725447072635);
item1.add(-0.0359677812752396);
item1.add(-0.0266789028311707);
item1.add(-0.0249926566315915);
item1.add(-0.00259226199818282);
item1.add(0.00371173823343597);
item1.add(0.0403433716478807);
// Add the first set of data to be scored
dataItems.add(item1);
// Create and add the second set
JSONArray item2 = new JSONArray();
item2.add(-0.0127796318808497);
item2.add(-0.044641636506989);
item2.add(0.0606183944448076);
item2.add(0.0606183944448076);
item2.add(0.0528581912385822);
item2.add(0.0479653430750293);
item2.add(0.0293746718291555);
item2.add(-0.0176293810234174);
item2.add(0.0343088588777263);
item2.add(0.0702112981933102);
item2.add(0.00720651632920303);
dataItems.add(item2);
obj.put("data", dataItems);
// Make the request using the JSON document string

sendRequest(obj.toJSONString());
}
}
[217.67978776218715, 224.78937091757172]
Call the service (Python)

This example demonstrates how to use Python to call the web service created from the Train within notebook
example:
import requests
import json
# URL for the web service

scoring_uri = '<your web service URI>'
# If the service is authenticated, set the key or token
key = '<your key or token>'
# Two sets of data to score, so we get two results back

data = {"data":
[
[
0.0199132141783263,
0.0506801187398187,
0.104808689473925,
0.0700725447072635,
-0.0359677812752396,
-0.0266789028311707,
-0.0249926566315915,
-0.00259226199818282,
0.00371173823343597,
0.0403433716478807
],
[
-0.0127796318808497,
-0.044641636506989,
0.0606183944448076,
0.0528581912385822,
0.0479653430750293,
0.0293746718291555,
-0.0176293810234174,
0.0343088588777263,
0.0702112981933102,
0.00720651632920303]
]
}
# Convert to JSON string
input_data = json.dumps(data)
# Set the content type

# If authentication is enabled, set the authorization header
headers['Authorization'] = f'Bearer {key}'
# Make the request and display the response

resp = requests.post(scoring_uri, input_data, headers=headers)
print(resp.text)
[217.67978776218715, 224.78937091757172]
Consume the service from Power BI

Power BI supports consumption of Azure Machine Learning web services to enrich the data in Power BI with
predictions.
To generate a web service that's supported for consumption in Power BI, the schema must support the format
that's required by Power BI. Learn how to create a Power BI-supported schema.
Once the web service is deployed, it's consumable from Power BI dataflows. Learn how to consume an Azure
Machine Learning web service from Power BI.
Next steps
To view a reference architecture for real-time scoring of Python and deep learning models, go to the Azure
architecture center.
Run batch predictions on large data sets with Azure
Machine Learning pipelines
In this article, you learn how to make predictions on large quantities of data asynchronously using the ML
pipelines with Azure Machine Learning service.
Batch prediction (or batch scoring) provides cost-effective inference, with unparalleled throughput for
asynchronous applications. Batch prediction pipelines can scale to perform inference on terabytes of production
data. Batch prediction is optimized for high throughput, fire-and-forget predictions for a large collection of data.
TIP
If your system requires low-latency processing (to process a single document or small set of documents quickly), use real-
time scoring instead of batch prediction.
In the following steps, you create a machine learning pipeline to register a pre-trained computer vision model
(Inception-V3). Then you use the pre-trained model to do batch scoring on images available in your Azure Blob
storage account. These images used for scoring are unlabeled images from the ImageNet dataset.
Prerequisites
If you don’t have an Azure subscription, create a free account before you begin. Try the free or paid
version of Azure Machine Learning service.
Configure your development environment to install the Azure Machine Learning SDK. For more
information, see Configure a development environment for Azure Machine Learning.
Create an Azure Machine Learning workspace that will hold all your pipeline resources. You can use the
following code, or for more options, see Create a workspace configuration file.

ws = Workspace.create(name = '<workspace-name>',
subscription_id = '<subscription-id>',
resource_group = '<resource-group>',
location = '<workspace_region>',
exist_ok = True
)
Set up machine learning resources

The following steps set up the resources you need to run a pipeline:
Access the datastore that already has the pretrained model, input labels, and images to score (this is already
set up for you).
Set up a datastore to store your outputs.
Configure DataReference objects to point to the data in the preceding datastores.
Set up compute machines or clusters where the pipeline steps will run.
Access the datastores
First, access the datastore that has the model, labels, and images.
Use a public blob container, named sampledata, in the pipelinedata account that holds images from the
ImageNet evaluation set. The datastore name for this public container is images_datastore. Register this
datastore with your workspace:
from azureml.core import Datastore
account_name = "pipelinedata"
datastore_name = "images_datastore"
container_name = "sampledata"
batchscore_blob = Datastore.register_azure_blob_container(ws,
datastore_name=datastore_name,
container_name=container_name,
account_name=account_name,
overwrite=True)
Next, set up to use the default datastore for the outputs.

When you create your workspace, Azure Files and Blob storage are attached to the workspace by default. Azure
Files is the default datastore for a workspace, but you can also use Blob storage as a datastore. For more
information, see Azure storage options.
def_data_store = ws.get_default_datastore()
Configure data references

Now, reference the data in your pipeline as inputs to pipeline steps.
A data source in a pipeline is represented by a DataReference object. The DataReference object points to data
that lives in, or is accessible from, a datastore. You need DataReference objects for the directory used for input
images, the directory in which the pretrained model is stored, the directory for labels, and the output directory.
from azureml.data.data_reference import DataReference
input_images = DataReference(datastore=batchscore_blob,
data_reference_name="input_images",
path_on_datastore="batchscoring/images",
mode="download")
model_dir = DataReference(datastore=batchscore_blob,
data_reference_name="input_model",
path_on_datastore="batchscoring/models",
mode="download")
label_dir = DataReference(datastore=batchscore_blob,
data_reference_name="input_labels",
path_on_datastore="batchscoring/labels",
mode="download")
output_dir = PipelineData(name="scores",
datastore=def_data_store,
output_path_on_compute="batchscoring/results")
Set up compute target

In Azure Machine Learning, compute (or compute target) refers to the machines or clusters that perform the
computational steps in your machine learning pipeline. For example, you can create an Azure Machine Learning
compute with the AmlCompute class.
compute_name = "gpucluster"
compute_min_nodes = 0
compute_max_nodes = 4
vm_size = "STANDARD_NC6"
print('Found compute target. just use it. ' + compute_name)
else:
provisioning_config = AmlCompute.provisioning_configuration(
vm_size=vm_size, # NC6 is GPU-enabled
vm_priority='lowpriority', # optional
min_nodes=compute_min_nodes,
max_nodes=compute_max_nodes)

compute_target = ComputeTarget.create(ws,
compute_name,
provisioning_config)
show_output=True,
min_node_count=None,
timeout_in_minutes=20)
Prepare the model

Before you can use the pretrained model, you'll need to download the model and register it with your
workspace.
Download the pretrained model
Download the pretrained computer vision model (InceptionV3) from
http://download.tensorflow.org/models/inception_v3_2016_08_28.tar.gz. Then extract it to the models subfolder.
import os
import tarfile
model_dir = 'models'
if not os.path.isdir(model_dir):
os.mkdir(model_dir)
url = "http://download.tensorflow.org/models/inception_v3_2016_08_28.tar.gz"
response = urllib.request.urlretrieve(url, "model.tar.gz")
tar = tarfile.open("model.tar.gz", "r:gz")
tar.extractall(model_dir)
Register the model

Here's how to register the model:
import shutil
# register downloaded model

model = Model.register(
model_path="models/inception_v3.ckpt",
model_name="inception", # This is the name of the registered model
tags={'pretrained': "inception"},
description="Imagenet trained tensorflow inception",
workspace=ws)
Write your scoring script

WARNING
The following code is only a sample of what is contained in the batch_score.py used by the sample notebook. You’ll need
to create your own scoring script for your scenario.
The batch_score.py script takes input images in dataset_path, pretrained models in model_dir, and outputs
results-label.txt to output_dir.
# Snippets from a sample scoring script
# Refer to the accompanying batch-scoring Notebook
# https://github.com/Azure/MachineLearningNotebooks/blob/master/pipeline/pipeline-batch-scoring.ipynb
# for the implementation script
# Get labels
def get_class_label_dict(label_file):
label = []
proto_as_ascii_lines = tf.gfile.GFile(label_file).readlines()
for l in proto_as_ascii_lines:
label.append(l.rstrip())
return label
class DataIterator:
# Definition of the DataIterator here
def main(_):
# Refer to batch-scoring Notebook for implementation.
label_file_name = os.path.join(args.label_dir, "labels.txt")
label_dict = get_class_label_dict(label_file_name)
classes_num = len(label_dict)
test_feeder = DataIterator(data_dir=args.dataset_path)
total_size = len(test_feeder.labels)
# get model from model registry

model_path = Model.get_model_path(args.model_name)
with tf.Session() as sess:
test_images = test_feeder.input_pipeline(batch_size=args.batch_size)
with slim.arg_scope(inception_v3.inception_v3_arg_scope()):
input_images = tf.placeholder(tf.float32, [args.batch_size, image_size, image_size,
num_channel])
logits, _ = inception_v3.inception_v3(input_images,
num_classes=classes_num,
is_training=False)
probabilities = tf.argmax(logits, 1)
sess.run(tf.global_variables_initializer())
sess.run(tf.local_variables_initializer())
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(sess=sess, coord=coord)
saver = tf.train.Saver()
saver.restore(sess, model_path)
out_filename = os.path.join(args.output_dir, "result-labels.txt")
# copy the file to artifacts

shutil.copy(out_filename, "./outputs/")
Build and run the batch scoring pipeline

Prepare the run environment
Specify the conda dependencies for your script. You'll need this object later, when you create the pipeline step.
from azureml.core.runconfig import DEFAULT_GPU_IMAGE
cd = CondaDependencies.create(
pip_packages=["tensorflow-gpu==1.10.0", "azureml-defaults"])
# Runconfig
amlcompute_run_config = RunConfiguration(conda_dependencies=cd)
amlcompute_run_config.environment.docker.enabled = True
amlcompute_run_config.environment.docker.gpu_support = True
amlcompute_run_config.environment.docker.base_image = DEFAULT_GPU_IMAGE
amlcompute_run_config.environment.spark.precache_packages = False
Specify the parameter for your pipeline

Create a pipeline parameter by using a PipelineParameter object with a default value.
from azureml.pipeline.core.graph import PipelineParameter

batch_size_param = PipelineParameter(
name="param_batch_size",
default_value=20)
Create the pipeline step

Create the pipeline step by using the script, environment configuration, and parameters. Specify the compute
target you already attached to your workspace as the target of execution of the script. Use PythonScriptStep to
create the pipeline step.
from azureml.pipeline.steps import PythonScriptStep

inception_model_name = "inception_v3.ckpt"
batch_score_step = PythonScriptStep(
name="batch_scoring",
script_name="batch_score.py",
arguments=["--dataset_path", input_images,
"--model_name", "inception",
"--label_dir", label_dir,
"--output_dir", output_dir,
"--batch_size", batch_size_param],
inputs=[input_images, label_dir],
outputs=[output_dir],
runconfig=amlcompute_run_config,
source_directory=scripts_folder
Run the pipeline

Now run the pipeline, and examine the output it produced. The output has a score corresponding to each input
image.
import pandas as pd
from azureml.pipeline.core import Pipeline
# Run the pipeline

pipeline = Pipeline(workspace=ws, steps=[batch_score_step])
pipeline_run = Experiment(ws, 'batch_scoring').submit(
pipeline, pipeline_params={"param_batch_size": 20})
# Wait for the run to finish (this might take several minutes)
pipeline_run.wait_for_completion(show_output=True)
# Download and review the output

step_run = list(pipeline_run.get_children())[0]
step_run.download_file("./outputs/result-labels.txt")
df = pd.read_csv("result-labels.txt", delimiter=":", header=None)

df.columns = ["Filename", "Prediction"]
df.head()
Publish the pipeline

After you're satisfied with the outcome of the run, publish the pipeline so you can run it with different input
values later. When you publish a pipeline, you get a REST endpoint. This endpoint accepts invoking of the
pipeline with the set of parameters you have already incorporated by using PipelineParameter.
published_pipeline = pipeline_run.publish_pipeline(
name="Inception_v3_scoring",
description="Batch scoring using Inception v3 model",
version="1.0")
Rerun the pipeline by using the REST endpoint

To rerun the pipeline, you'll need an Azure Active Directory authentication header token, as described in
AzureCliAuthentication class.
from azureml.pipeline.core.run import PipelineRun

from azureml.pipeline.core import PublishedPipeline
rest_endpoint = published_pipeline.endpoint
# specify batch size when running the pipeline
response = requests.post(rest_endpoint,
headers=aad_token,
json={"ExperimentName": "batch_scoring",
"ParameterAssignments": {"param_batch_size": 50}})
# Monitor the run

published_pipeline_run = PipelineRun(ws.experiments["batch_scoring"], run_id)
RunDetails(published_pipeline_run).show()
Next steps
To see this working end-to-end, try the batch scoring notebook in GitHub, or go to the Azure architecture center
to see a sample solution architecture.
In this article, you can learn how to collect input model data from the Azure Machine Learning services you've
deployed into Azure Kubernetes Cluster (AKS ) into an Azure Blob storage.
Once enabled, this data you collect helps you:
Monitor data drifts as production data enters your model
Make better decisions on when to retrain or optimize your model
Retrain your model with the data collected
What is collected and where does it go?

The following data can be collected:
Model input data from web services deployed in Azure Kubernetes Cluster (AKS ) (Voice, images, and
video are not collected)
Model predictions using production input data.
NOTE
Pre-aggregation or pre-calculations on this data are not part of the service at this time.
The output gets saved in an Azure Blob. Since the data gets added into an Azure Blob, you can then choose your
favorite tool to run the analysis.
The path to the output data in the blob follows this syntax:
/modeldata/<subscriptionid>/<resourcegroup>/<workspace>/<webservice>/<model>/<version>/<identifier>/<year>/<
month>/<day>/data.csv
# example: /modeldata/1a2b3c4d-5e6f-7g8h-9i10-j11k12l13m14/myresourcegrp/myWorkspace/aks-w-
collv9/best_model/10/inputs/2018/12/31/data.csv
Prerequisites
An Azure Machine Learning service workspace, a local directory containing your scripts, and the Azure
Machine Learning SDK for Python installed. Learn how to get these prerequisites using the How to
configure a development environment document.
A trained machine learning model to be deployed to Azure Kubernetes Service (AKS ). If you don't have
one, see the train image classification model tutorial.
An Azure Kubernetes Service cluster. For information on how to create and deploy to one, see the How to
deploy and where document.
Set up your environment and install the Monitoring SDK.
Enable data collection
Data collection can be enabled regardless of the model being deployed through Azure Machine Learning Service
or other tools.
To enable it, you need to:
1. Open the scoring file.
2. Add the following code at the top of the file:
from azureml.monitoring import ModelDataCollector
3. Declare your data collection variables in your init() function:
global inputs_dc, prediction_dc

inputs_dc = ModelDataCollector("best_model", identifier="inputs", feature_names=["feat1", "feat2",
"feat3". "feat4", "feat5", "feat6"])
prediction_dc = ModelDataCollector("best_model", identifier="predictions", feature_names=
["prediction1", "prediction2"])
CorrelationId is an optional parameter, you do not need to set it up if your model doesn’t require it.
Having a correlationId in place does help you for easier mapping with other data. (Examples include:
LoanNumber, CustomerId, etc.)
Identifier is later used for building the folder structure in your Blob, it can be used to divide “raw” data
versus “processed”.
4. Add the following lines of code to the run(input_df) function:
data = np.array(data)
inputs_dc.collect(data) #this call is saving our input data into Azure Blob
prediction_dc.collect(result) #this call is saving our input data into Azure Blob
5. Data collection is not automatically set to true when you deploy a service in AKS, so you must update
your configuration file such as:
aks_config = AksWebservice.deploy_configuration(collect_model_data=True)
AppInsights for service monitoring can also be turned on by changing this configuration:
aks_config = AksWebservice.deploy_configuration(collect_model_data=True, enable_app_insights=True)
6. To create a new image and deploy the service, see the How to deploy and where document.
If you already have a service with the dependencies installed in your environment file and scoring file, enable
data collection by:
1. Go to Azure Portal.
2. Open your workspace.
3. Go to Deployments -> Select service -> Edit.
4. In Advanced Settings, deselect Enable Model data collection.
In this window, you can also choose to "Enable Appinsights diagnostics" to track the health of your
service.
5. Select Update to apply the change.
Disable data collection

You can stop collecting data any time. Use Python code or the Azure portal to disable data collection.
Option 1 - Disable in the Azure portal:
1. Sign in to Azure portal.
3. Go to Deployments -> Select service -> Edit.
4. In Advanced Settings, deselect Enable Model data collection.
5. Select Update to apply the change.

Option 2 - Use Python to disable data collection:
## replace <service_name> with the name of the web service

<service_name>.update(collect_model_data=False)
Validate your data and analyze it

You can choose any tool of your preference to analyze the data collected into your Azure Blob.
To quickly access the data from your blob:
1. Sign in to Azure portal.
3. Click on Storage.
4. Follow the path to the output data in the blob with this syntax:
/modeldata/<subscriptionid>/<resourcegroup>/<workspace>/<webservice>/<model>/<version>/<identifier>/<year>/<
month>/<day>/data.csv
# example: /modeldata/1a2b3c4d-5e6f-7g8h-9i10-j11k12l13m14/myresourcegrp/myWorkspace/aks-w-
collv9/best_model/10/inputs/2018/12/31/data.csv
Analyzing model data through Power BI

1. Download and Open PowerBi Desktop
2. Select Get Data and click on Azure Blob Storage.
3. Add your storage account name and enter your storage key. You can find this information in your blob's
Settings >> Access keys.
4. Select the container modeldata and click on Edit.
5. In the query editor, click under “Name” column and add your Storage account 1. Model path into the filter.
Note: if you want to only look into files from a specific year or month, just expand the filter path. For
example, just look into March data:
/modeldata/subscriptionid>/resourcegroupname>/workspacename>/webservicename>/modelname>/
modelversion>/identifier>/year>/3
6. Filter the data that is relevant to you based on Name. If you stored predictions and inputs you'll need to
do create a query per each.
7. Click on the double arrow aside the Content column to combine the files.
8. Click OK and the data will preload.

9. You can now click Close and Apply .
10. If you added inputs and predictions your tables will automatically correlate by RequestId.
11. Start building your custom reports on your model data.
Analyzing model data using Databricks
1. Create a Databricks workspace.
2. Go to your Databricks workspace.
3. In your databricks workspace select Upload Data.
4. Create New Table and select Other Data Sources -> Azure Blob Storage -> Create Table in Notebook.
5. Update the location of your data. Here is an example:

file_location = "wasbs://mycontainer@storageaccountname.blob.core.windows.net/modeldata/1a2b3c4d-5e6f-
7g8h-9i10-j11k12l13m14/myresourcegrp/myWorkspace/aks-w-collv9/best_model/10/inputs/2018/*/*/data.csv"
file_type = "csv"
6. Follow the steps on the template in order to view and analyze your data.
Example notebook
The how -to-use-azureml/deployment/enable-data-collection-for-models-in-aks/enable-data-collection-for-
models-in-aks.ipynb notebook demonstrates concepts in this article.
Detect data drift (preview) on models deployed to
Azure Kubernetes Service (AKS)
In this article, you learn how to monitor for data drift between the training dataset and inference data of a
deployed model. In the context of machine learning, trained machine learning models may experience degraded
prediction performance because of drift. With the Azure Machine Learning service, you can monitor data drift and
the service can send an email alert to you when drift is detected.
What is data drift?

Data drift happens when data served to a model in production is different from the data used to train the model. It
is one of the top reasons where model accuracy degrades over time, thus monitoring data drift helps detect model
performance issues.
What can I monitor?

With Azure Machine Learning service, you can monitor the inputs to a model deployed on AKS and compare this
data to the training dataset for the model. At regular intervals, the inference data is snapshot and profiled, then
computed against the baseline dataset to produce a data drift analysis that:
Measures the magnitude of data drift, called the drift coefficient.
Measures the data drift contribution by feature, informing which features caused data drift.
Measures distance metrics. Currently Wasserstein and Energy Distance are computed.
Measures distributions of features. Currently kernel density estimation and histograms.
Send alerts to data drift by email.
NOTE
This service is in (Preview) and limited in configuration options. Please see our API Documentation and Release Notes for
details and updates.
How data drift is monitored in Azure Machine Learning service

Using Azure Machine Learning service, data drift is monitored through datasets or deployments. To monitor for
data drift, a baseline dataset - usually the training dataset for a model - is specified. A second dataset - usually
model input data gathered from a deployment - is tested against the baseline dataset. Both datasets are profiled
and input to the data drift monitoring service. A machine learning model is trained to detect differences between
the two datasets. The model's performance is converted to the drift coefficient, which measures the magnitude of
drift between the two datasets. Using model interpretability, the features that contribute to the drift coefficient are
computed. From the dataset profile, statistical information about each feature is tracked.
Prerequisites
An Azure subscription. If you don’t have one, create a free account before you begin. Try the free or paid
The Azure Machine Learning SDK for Python installed. Use the instructions at Azure Machine Learning
SDK to do the following:
Create a Miniconda environment
Install the Azure Machine Learning SDK for Python
A workspace configuration file.
Install the data drift SDK using the following command:
pip install azureml-contrib-datadrift
Create a dataset from your model's training data.

Specify the training dataset when registering the model. The following example demonstrates using the
datasets parameter to specify the training dataset:
model = Model.register(model_path=model_file,
model_name=model_name,
workspace=ws,
datasets=datasets)
print(model_name, image_name, service_name, model)
Enable model data collection to collect data from the AKS deployment of the model and confirm data is
being collected in the modeldata blob container.
Configure data drift

To configure data drift for your experiment, import dependencies as seen in the following Python example.
This example demonstrates configuring the DataDriftDetector object:
# Import Azure ML packages

from azureml.core import Experiment, Run, RunDetails
from azureml.contrib.datadrift import DataDriftDetector, AlertConfiguration
# if email address is specified, setup AlertConfiguration

alert_config = AlertConfiguration('your_email@contoso.com')
# create a new DatadriftDetector object

datadrift = DataDriftDetector.create(ws, model.name, model.version, services, frequency="Day",
alert_config=alert_config)
print('Details of Datadrift Object:\n{}'.format(datadrift))
Submit a DataDriftDetector run

With the DataDriftDetector object configured, you can submit a data drift run on a given date for the model. As
part of the run, enable DataDriftDetector alerts by setting the drift_threshold parameter. If the
datadrift_coefficient is above the given drift_threshold , an email is sent.
# adhoc run today
target_date = datetime.today()
# create a new compute - creates datadrift-server

run = datadrift.run(target_date, services, feature_list=feature_list, create_compute_target=True)
# or specify existing compute cluster

run = datadrift.run(target_date, services, feature_list=feature_list, compute_target='cpu-cluster')
# show details of the data drift run

exp = Experiment(ws, datadrift._id)
dd_run = Run(experiment=exp, run_id=run)
RunDetails(dd_run).show()
Visualize drift metrics

After you submit your DataDriftDetector run, you are able to see the drift metrics that are saved in each run
iteration for a data drift task:
METRIC DESCRIPTION
wasserstein_distance Statistical distance defined for one-dimensional numerical

distribution.
energy_distance Statistical distance defined for one-dimensional numerical

distribution.
datadrift_coefficient Calculated similarly as Matthew's correlation coefficient, but

this output is a real number ranging from 0 to 1. In the
context of drift, 0 indicates no drift and 1 indicates maximum
drift.
datadrift_contribution Feature importance of features contributing to drift.
There are multiple ways to view drift metrics:

Use the RunDetails Jupyter widget.
Use the get_metrics() function on any datadrift run object.
View the metrics in the Azure portal on your model.
The following Python example demonstrates how to plot relevant data drift metrics. You can use the returned
metrics to build custom visualizations:
# start and end are datetime objects

drift_metrics = datadrift.get_output(start_time=start, end_time=end)
# Show all data drift result figures, one per serivice.

# If setting with_details is False (by default), only the data drift magnitude will be shown; if it's True,
all details will be shown.
drift_figures = datadrift.show(with_details=True)
Schedule data drift scans
When you enable data drift detection, a DataDriftDetector is run at the specified, scheduled frequency. If the
datadrift_coefficient reaches the given drift_threshold , an email is sent with each scheduled run.
datadrift.enable_schedule()
datadrift.disable_schedule()
The configuration of the data drift detector can be seen on the model details page in the Azure portal.
View results in Azure portal
To view results in your workspace in Azure portal, navigate to the model page. On the details tab of the model, the
data drift configuration is shown. A 'Data Drift (Preview )' tab is now available visualizing the data drift metrics.
Receiving drift alerts
By setting the drift coefficient alerting threshold and providing an email address, an Azure Monitor email alert is
automatically sent whenever the drift coefficient is above the threshold.
In order for you to set up custom alerts and actions, all data drift metrics are stored in the Application Insights
resource that was created along with the Azure Machine Learning service workspace. You can follow the link in the
email alert to the Application Insights query.
Retrain your model after drift
When data drift negatively impacts the performance of your deployed model, it is time to retrain the model. To do
so, proceed with the following steps.
Investigate the collected data and prepare data to train the new model.
Split it into train/test data.
Train the model again using the new data.
Evaluate performance of the newly generated model.
Deploy new model if performance is better than the production model.
Next steps
For a full example of using data drift, see the Azure ML data drift notebook. This Jupyter Notebook
demonstrates using an Azure Open Dataset to train a model to predict the weather, deploy it to AKS, and
monitor for data drift.
We would greatly appreciate your questions, comments, or suggestions as data drift moves toward general
availability. Use the product feedback button below!
Monitor your Azure Machine Learning models with
Application Insights
In this article, you learn how to set up Azure Application Insights for your Azure Machine Learning service.
Application Insights gives you the opportunity to monitor:
Request rates, response times, and failure rates.
Dependency rates, response times, and failure rates.
Exceptions.
Learn more about Application Insights.
Prerequisites
An Azure Machine Learning workspace, a local directory that contains your scripts, and the Azure
Machine Learning SDK for Python installed. To learn how to get these prerequisites, see How to configure
a development environment.
A trained machine learning model to be deployed to Azure Kubernetes Service (AKS ) or Azure Container
Instance (ACI). If you don't have one, see the Train image classification model tutorial.
Use SDK to configure

Update a deployed service
1. Identify the service in your workspace. The value for ws is the name of your workspace.

aks_service= Webservice(ws, "my-service-name")
2. Update your service and enable Application Insights.
aks_service.update(enable_app_insights=True)
Log custom traces in your service

If you want to log custom traces, follow the standard deployment process for AKS or ACI in the How to deploy
and where document. Then use the following steps:
1. Update the scoring file by adding print statements.
print ("model initialized" + time.strftime("%H:%M:%S"))
2. Update the service configuration.

config = Webservice.deploy_configuration(enable_app_insights=True)
3. Build an image and deploy it on AKS or ACI.

Disable tracking in Python
To disable Application Insights, use the following code:
## replace <service_name> with the name of the web service

<service_name>.update(enable_app_insights=False)
Use portal to configure

You can enable and disable Application Insights in the Azure portal.
2. On the Deployments tab, select the service where you want to enable Application Insights.
3. Select Edit.
4. In Advanced Settings, select the Enable AppInsights diagnostics check box.
5. Select Update at the bottom of the screen to apply the changes.

Disable
2. Select Deployments, select the service, and select Edit.
3. In Advanced Settings, clear the Enable AppInsights diagnostics check box.
4. Select Update at the bottom of the screen to apply the changes.
Evaluate data
Your service's data is stored in your Application Insights account, within the same resource group as your Azure
Machine Learning service. To view it:
1. Go to your Machine Learning service workspace in the Azure portal and click on Application Insights link.
2. Select the Overview tab to see a basic set of metrics for your service.
3. To look into your custom traces, select Analytics.
4. In the schema section, select Traces. Then select Run to run your query. Data should appear in a table
format and should map to your custom calls in your scoring file.
To learn more about how to use Application Insights, see What is Application Insights?.
Example notebook
The enable-app-insights-in-production-service.ipynb notebook demonstrates concepts in this article.
Next steps
You can also collect data on your models in production. Read the article Collect data for models in production.
Also read Azure Monitor for containers.
Create and run machine learning pipelines with
Azure Machine Learning SDK
In this article, you learn how to create, publish, run, and track a machine learning pipeline by using the Azure
Machine Learning SDK. Use ML pipelines to create a workflow that stitches together various ML phases, and
then publish that pipeline into your Azure Machine Learning workspace to access later or share with others. ML
pipelines are ideal for batch scoring scenarios, using various computes, reusing steps instead of rerunning them,
as well as sharing ML workflows with others.
While you can use a different kind of pipeline called an Azure Pipeline for CI/CD automation of ML tasks, that
type of pipeline is never stored inside your workspace. Compare these different pipelines.
Each phase of an ML pipeline, such as data preparation and model training, can include one or more steps.
The ML pipelines you create are visible to the members of your Azure Machine Learning service workspace.
ML pipelines use remote compute targets for computation and the storage of the intermediate and final data
associated with that pipeline. They can read and write data to and from supported Azure Storage locations.
Prerequisites
Create an Azure Machine Learning workspace to hold all your pipeline resources.
Configure your development environment to install the Azure Machine Learning SDK, or use a Notebook
VM with the SDK already installed.
Start by attaching your workspace:
import azureml.core
Set up machine learning resources

Create the resources required to run an ML pipeline:
Set up a datastore used to access the data needed in the pipeline steps.
Configure a DataReference object to point to data that lives in, or is accessible in, a datastore.
Set up the compute targets on which your pipeline steps will run.
Set up a datastore
A datastore stores the data for the pipeline to access. Each workspace has a default datastore. You can register
additional datastores.
When you create your workspace, Azure Files and Azure Blob storage are attached to the workspace. A default
datastore is registered to connect to the Azure Blob storage. To learn more, see Deciding when to use Azure Files,
Azure Blobs, or Azure Disks.
# Default datastore
def_data_store = ws.get_default_datastore()
# Get the blob storage associated with the workspace

def_blob_store = Datastore(ws, "workspaceblobstore")
# Get file storage associated with the workspace

def_file_store = Datastore(ws, "workspacefilestore")
Upload data files or directories to the datastore for them to be accessible from your pipelines. This example uses
the Blob storage as the datastore:
def_blob_store.upload_files(
["./data/20news.pkl"],
target_path="20newsgroups",
overwrite=True)
A pipeline consists of one or more steps. A step is a unit run on a compute target. Steps might consume data
sources and produce “intermediate” data. A step can create data such as a model, a directory with model and
dependent files, or temporary data. This data is then available for other steps later in the pipeline.
Configure data reference
You just created a data source that can be referenced in a pipeline as an input to a step. A data source in a pipeline
is represented by a DataReference object. The DataReference object points to data that lives in or is accessible
from a datastore.
from azureml.data.data_reference import DataReference
blob_input_data = DataReference(
datastore=def_blob_store,
data_reference_name="test_data",
path_on_datastore="20newsgroups/20news.pkl")
Intermediate data (or output of a step) is represented by a PipelineData object. output_data1 is produced as the
output of a step, and used as the input of one or more future steps. PipelineData introduces a data dependency
between steps, and creates an implicit execution order in the pipeline.
from azureml.pipeline.core import PipelineData
output_data1 = PipelineData(
"output_data1",
datastore=def_blob_store,
output_name="output_data1")
Set up compute target

In Azure Machine Learning, the term computes__ (or compute target) refers to the machines or clusters that
perform the computational steps in your machine learning pipeline. See compute targets for model training for a
full list of compute targets and how to create and attach them to your workspace. The process for creating and or
attaching a compute target is the same regardless of whether you are training a model or running a pipeline step.
After you create and attach your compute target, use the ComputeTarget object in your pipeline step.
IMPORTANT
Performing management operations on compute targets is not supported from inside remote jobs. Since machine learning
pipelines are submitted as a remote job, do not use management operations on compute targets from inside the pipeline.
Below are examples of creating and attaching compute targets for:

Azure Databricks
Azure Data Lake Analytics
Azure Machine Learning compute
You can create an Azure Machine Learning compute for running your steps.
compute_name = "aml-compute"
vm_size = "STANDARD_NC6"
print('Found compute target: ' + compute_name)
else:
provisioning_config = AmlCompute.provisioning_configuration(vm_size=vm_size, # STANDARD_NC6 is GPU-
enabled
min_nodes=0,
max_nodes=4)
# create the compute target
ws, compute_name, provisioning_config)
# Can poll for a minimum number of nodes and for a specific timeout.
# If no min node count is provided it will use the scale settings for the cluster
# For a more detailed view of current cluster status, use the 'status' property
print(compute_target.status.serialize())
Azure Databricks
Azure Databricks is an Apache Spark-based environment in the Azure cloud. It can be used as a compute target
with an Azure Machine Learning pipeline.
Create an Azure Databricks workspace before using it. To create a workspace resource, see the Run a Spark job on
Azure Databricks document.
To attach Azure Databricks as a compute target, provide the following information:
Databricks compute name: The name you want to assign to this compute resource.
Databricks workspace name: The name of the Azure Databricks workspace.
Databricks access token: The access token used to authenticate to Azure Databricks. To generate an access
token, see the Authentication document.
The following code demonstrates how to attach Azure Databricks as a compute target with the Azure Machine
Learning SDK:
import os
from azureml.core.compute import ComputeTarget, DatabricksCompute
databricks_compute_name = os.environ.get(
"AML_DATABRICKS_COMPUTE_NAME", "<databricks_compute_name>")
databricks_workspace_name = os.environ.get(
"AML_DATABRICKS_WORKSPACE", "<databricks_workspace_name>")
databricks_resource_group = os.environ.get(
"AML_DATABRICKS_RESOURCE_GROUP", "<databricks_resource_group>")
databricks_access_token = os.environ.get(
"AML_DATABRICKS_ACCESS_TOKEN", "<databricks_access_token>")
try:
databricks_compute = ComputeTarget(
workspace=ws, name=databricks_compute_name)
print('Compute target already exists')
print('compute not found')
print('databricks_compute_name {}'.format(databricks_compute_name))
print('databricks_workspace_name {}'.format(databricks_workspace_name))
print('databricks_access_token {}'.format(databricks_access_token))
# Create attach config

attach_config = DatabricksCompute.attach_configuration(resource_group=databricks_resource_group,
workspace_name=databricks_workspace_name,
access_token=databricks_access_token)
databricks_compute = ComputeTarget.attach(
ws,
databricks_compute_name,
attach_config
)
databricks_compute.wait_for_completion(True)
For a more detailed example, see an example notebook on GitHub.

Azure Data Lake Analytics
Azure Data Lake Analytics is a big data analytics platform in the Azure cloud. It can be used as a compute target
with an Azure Machine Learning pipeline.
Create an Azure Data Lake Analytics account before using it. To create this resource, see the Get started with
Azure Data Lake Analytics document.
To attach Data Lake Analytics as a compute target, you must use the Azure Machine Learning SDK and provide
the following information:
Compute name: The name you want to assign to this compute resource.
Resource Group: The resource group that contains the Data Lake Analytics account.
Account name: The Data Lake Analytics account name.
The following code demonstrates how to attach Data Lake Analytics as a compute target:
import os
from azureml.core.compute import ComputeTarget, AdlaCompute
adla_compute_name = os.environ.get(
"AML_ADLA_COMPUTE_NAME", "<adla_compute_name>")
adla_resource_group = os.environ.get(
"AML_ADLA_RESOURCE_GROUP", "<adla_resource_group>")
adla_account_name = os.environ.get(
"AML_ADLA_ACCOUNT_NAME", "<adla_account_name>")
try:
adla_compute = ComputeTarget(workspace=ws, name=adla_compute_name)
print('Compute target already exists')
print('compute not found')
print('adla_compute_name {}'.format(adla_compute_name))
print('adla_resource_id {}'.format(adla_resource_group))
print('adla_account_name {}'.format(adla_account_name))
# create attach config
attach_config = AdlaCompute.attach_configuration(resource_group=adla_resource_group,
account_name=adla_account_name)
# Attach ADLA
adla_compute = ComputeTarget.attach(
ws,
adla_compute_name,
attach_config
)
adla_compute.wait_for_completion(True)
For a more detailed example, see an example notebook on GitHub.
TIP
Azure Machine Learning pipelines can only work with data stored in the default data store of the Data Lake Analytics
account. If the data you need to work with is in a non-default store, you can use a DataTransferStep to copy the data
before training.
Construct your pipeline steps

Once you create and attach a compute target to your workspace, you are ready to define a pipeline step. There are
many built-in steps available via the Azure Machine Learning SDK. The most basic of these steps is a
PythonScriptStep, which runs a Python script in a specified compute target:
from azureml.pipeline.steps import PythonScriptStep
trainStep = PythonScriptStep(
script_name="train.py",
arguments=["--input", blob_input_data, "--output", processed_data1],
inputs=[blob_input_data],
outputs=[processed_data1],
source_directory=project_folder
)
Reuse of previous results ( allow_reuse ) is key when using pipelines in a collaborative environment since
eliminating unnecessary reruns offers agility. Reuse is the default behavior when the script_name, inputs, and the
parameters of a step remain the same. When the output of the step is reused, the job is not submitted to the
compute, instead, the results from the previous run are immediately available to the next step's run. If
allow_reuse is set to false, a new run will always be generated for this step during pipeline execution.
After you define your steps, you build the pipeline by using some or all of those steps.
NOTE
No file or data is uploaded to the Azure Machine Learning service when you define the steps or build the pipeline.
# list of steps to run

compareModels = [trainStep, extractStep, compareStep]
from azureml.pipeline.core import Pipeline
# Build the pipeline

pipeline1 = Pipeline(workspace=ws, steps=[compareModels])
The following example uses the Azure Databricks compute target created earlier:
from azureml.pipeline.steps import DatabricksStep
dbStep = DatabricksStep(
name="databricksmodule",
inputs=[step_1_input],
outputs=[step_1_output],
num_workers=1,
notebook_path=notebook_path,
notebook_params={'myparam': 'testparam'},
run_name='demo run name',
compute_target=databricks_compute,
allow_reuse=False
)
# List of steps to run
steps = [dbStep]
# Build the pipeline

pipeline1 = Pipeline(workspace=ws, steps=steps)
For more information, see the azure-pipeline-steps package and Pipeline class reference.
Submit the pipeline

When you submit the pipeline, Azure Machine Learning service checks the dependencies for each step and
uploads a snapshot of the source directory you specified. If no source directory is specified, the current local
directory is uploaded. The snapshot is also stored as part of the experiment in your workspace.
IMPORTANT
To prevent files from being included in the snapshot, create a .gitignore or .amlignore file in the directory and add the files
to it. The .amlignore file uses the same syntax and patterns as the .gitignore file. If both files exist, the .amlignore file
takes precedence.
For more information, see Snapshots.
# Submit the pipeline to be run

pipeline_run1 = Experiment(ws, 'Compare_Models_Exp').submit(pipeline1)
pipeline_run1.wait_for_completion()
When you first run a pipeline, Azure Machine Learning:

Downloads the project snapshot to the compute target from the Blob storage associated with the workspace.
Builds a Docker image corresponding to each step in the pipeline.
Downloads the Docker image for each step to the compute target from the container registry.
Mounts the datastore if a DataReference object is specified in a step. If mount is not supported, the data is
instead copied to the compute target.
Runs the step in the compute target specified in the step definition.
Creates artifacts, such as logs, stdout and stderr, metrics, and output specified by the step. These artifacts are
then uploaded and kept in the user’s default datastore.
For more information, see the Experiment class reference.

the history.
Publish a pipeline
You can publish a pipeline to run it with different inputs later. For the REST endpoint of an already published
pipeline to accept parameters, you must parameterize the pipeline before publishing.
1. To create a pipeline parameter, use a PipelineParameter object with a default value.
from azureml.pipeline.core.graph import PipelineParameter
pipeline_param = PipelineParameter(
name="pipeline_arg",
default_value=10)
2. Add this PipelineParameter object as a parameter to any of the steps in the pipeline as follows:
compareStep = PythonScriptStep(
script_name="compare.py",
arguments=["--comp_data1", comp_data1, "--comp_data2", comp_data2, "--output_data", out_data3, "--
param1", pipeline_param],
inputs=[ comp_data1, comp_data2],
outputs=[out_data3],
target=compute_target,
source_directory=project_folder)
3. Publish this pipeline that will accept a parameter when invoked.
published_pipeline1 = pipeline_run1.publish_pipeline(
name="My_Published_Pipeline",
description="My Published Pipeline Description",
version="1.0")
Run a published pipeline

All published pipelines have a REST endpoint. This endpoint invokes the run of the pipeline from external systems,
such as non-Python clients. This endpoint enables "managed repeatability" in batch scoring and retraining
scenarios.
To invoke the run of the preceding pipeline, you need an Azure Active Directory authentication header token, as
described in AzureCliAuthentication class reference or get more details in the Authentication in Azure Machine
Learning notebook.
from azureml.pipeline.core import PublishedPipeline

import requests
response = requests.post(published_pipeline1.endpoint,
headers=aad_token,
json={"ExperimentName": "My_Pipeline",
"ParameterAssignments": {"pipeline_arg": 20}})
View results of a published pipeline

See the list of all your published pipelines and their run details:
1. Sign in to the Azure portal.
2. View your workspace to find the list of pipelines.
3. Select a specific pipeline to see the run results.

Disable a published pipeline
To hide a pipeline from your list of published pipelines, you disable it:
# Get the pipeline by using its ID in the Azure portal
p = PublishedPipeline.get(ws, id="068f4885-7088-424b-8ce2-eeb9ba5381a6")
p.disable()
You can enable it again with p.enable() . For more information, see PublishedPipeline class reference.
Caching & reuse

In order to optimize and customize the behavior of your pipelines, you can do a few things around caching and
reuse. For example, you can choose to:
Turn off the default reuse of the step run output by setting allow_reuse=False during step definition.
Reuse is key when using pipelines in a collaborative environment since eliminating unnecessary runs offers
agility. However, you can opt out of reuse.
Extend hashing beyond the script, to also include an absolute path or relative paths to the source_directory
to other files and directories using the hash_paths=['<file or directory']
Force output regeneration for all steps in a run with
pipeline_run = exp.submit(pipeline, regenerate_outputs=False)
By default, allow_reuse for steps is enabled and only the main script file is hashed. So, if the script for a given step
remains the same ( script_name , inputs, and the parameters), the output of a previous step run is reused, the job is
not submitted to the compute, and the results from the previous run are immediately available to the next step
instead.
step = PythonScriptStep(name="Hello World",

script_name="hello_world.py",
compute_target=aml_compute,
source_directory=source_directory,
allow_reuse=False,
hash_paths=['hello_world.ipynb'])
Next steps
Use these Jupyter notebooks on GitHub to explore machine learning pipelines further.
See the SDK reference help for the azureml-pipelines-core package and the azureml-pipelines-steps package.
Manage and request quotas for Azure resources
As with other Azure services, there are limits on certain resources associated with the Azure Machine Learning
service. These limits range from a cap on the number of workspaces you can create to limits on the actual
underlying compute that gets used for model training or inference/scoring.
This article gives more details on the pre-configured limits on various Azure resources for your subscription and
also contains handy links to request quota enhancements for each type of resource. These limits are put in place to
prevent budget over-runs due to fraud, and to honor Azure capacity constraints.
Keep these quotas in mind as you design and scale up your Azure Machine Learning service resources for
production workloads. For example, if your cluster doesn't reach the target number of nodes you specified, then
you might have reached an Azure Machine Learning Compute cores limit for your subscription. If you want to
raise the limit or quota above the Default Limit, open an online customer support request at no charge. The limits
can't be raised above the Maximum Limit value shown in the following tables due to Azure Capacity constraints. If
there is no Maximum Limit column, then the resource doesn't have adjustable limits.
Special considerations
A quota is a credit limit, not a capacity guarantee. If you have large-scale capacity needs, contact Azure
support.
Your quota is shared across all the services in your subscriptions including Azure Machine Learning service.
The only exception is Azure Machine Learning compute which has a separate quota from the core compute
quota. Be sure to calculate the quota usage across all services when evaluating your capacity needs.
Default limits vary by offer Category Type, such as Free Trial, Pay-As-You-Go, and series, such as Dv2, F, G,
and so on.
Default resource quotas

Here is a breakdown of the quota limits by various resource types within your Azure subscription.
IMPORTANT
Limits are subject to change. The latest can always be found at the service-level quota document for all of Azure.
Virtual machines
There is a limit on the number of virtual machines you can provision on an Azure subscription across your services
or in a standalone manner. This limit is at the region level both on the total cores and also on a per family basis.
It is important to emphasize that virtual machine cores have a regional total limit and a regional per size series
(Dv2, F, etc.) limit that are separately enforced. For example, consider a subscription with a US East total VM core
limit of 30, an A series core limit of 30, and a D series core limit of 30. This subscription would be allowed to
deploy 30 A1 VMs, or 30 D1 VMs, or a combination of the two not to exceed a total of 30 cores (for example, 10
A1 VMs and 20 D1 VMs).
RESOURCE DEFAULT LIMIT MAXIMUM LIMIT
VMs per subscription 25,0001 per region. 25,000 per region.
VM total cores per subscription 201 per region. Contact support.
VM per series, such as Dv2 and F, cores 201 per region. Contact support.
per subscription
Coadministrators per subscription Unlimited. Unlimited.
Storage accounts per region per 250 250

subscription
Resource groups per subscription 980 980
Availability sets per subscription 2,000 per region. 2,000 per region.
Azure Resource Manager API request 4,194,304 bytes. 4,194,304 bytes.

size
Tags per subscription2 Unlimited. Unlimited.
Unique tag calculations per 10,000 10,000

subscription2
Cloud services per subscription N/A3 N/A3
Affinity groups per subscription N/A3 N/A3
Subscription-level deployments per 8004 800

location
1Default limits vary by offer

category type, such as Free Trial and Pay-As-You-Go, and by series, such as Dv2, F,
and G. For example, the default for Enterprise Agreement subscriptions is 350.
2You can apply an unlimited number of tags per subscription. The number of tags per resource or resource group
is limited to 50. Resource Manager returns a list of unique tag name and values in the subscription only when the
number of tags is 10,000 or less. You still can find a resource by tag when the number exceeds 10,000.
3These features are no longer required with Azure resource groups and Resource Manager.
4If you reach the limit of 800deployments, delete deployments from the history that are no longer needed. To
delete subscription level deployments, use Remove-AzDeployment or az deployment delete.
NOTE
Virtual machine cores have a regional total limit. They also have a limit for regional per-size series, such as Dv2 and F. These
limits are separately enforced. For example, consider a subscription with a US East total VM core limit of 30, an A series core
limit of 30, and a D series core limit of 30. This subscription can deploy 30 A1 VMs, or 30 D1 VMs, or a combination of the
two not to exceed a total of 30 cores. An example of a combination is 10 A1 VMs and 20 D1 VMs.
For a more detailed and up-to-date list of quota limits, check the Azure-wide quota article here.
For Azure Machine Learning Compute, there is a default quota limit on both the number of cores and number of
unique compute resources allowed per region in a subscription. This quota is separate from the VM core quota
above and the core limits are not shared currently between the two resource types.
Available resources:
Dedicated cores per region have a default limit of 24 - 300 depending on your subscription offer type. The
number of dedicated cores per subscription can be increased. Contact Azure support to discuss increase
options.
Low -priority cores per region have a default limit of 24 - 300 depending on your subscription offer type.
The number of low -priority cores per subscription can be increased. Contact Azure support to discuss
increase options.
Clusters per region have a default limit of 100 and a maximum limit of 200. Contact Azure support if you
want to request an increase beyond this limit.
There are other strict limits which cannot be exceeded once hit.
RESOURCE MAXIMUM LIMIT
Maximum workspaces per resource group 800
Maximum nodes in a single Azure Machine Learning Compute 100 nodes

(AmlCompute) resource
Maximum GPU MPI processes per node 1-4
Maximum GPU workers per node 1-4
Maximum job lifetime 90 days1
Maximum job lifetime on a Low Priority Node 1 day2
Maximum parameter servers per node 1
1 The maximum lifetime refers to the time that a run start and when it finishes. Completed runs persist indefinitely;
data for runs not completed within the maximum lifetime is not accessible. 2 Jobs on a Low Priority node could be
pre-empted any time there is a capacity constraint. It is recommended to implement checkpointing in your job.
Azure Machine Learning Pipelines
For Azure Machine Learning Pipelines, there is a quota limit on the number of steps in a pipeline and on the
number of schedule-based runs of published pipelines per region in a subscription.
Maximum number of steps allowed in a pipeline is 30,000
Maximum number of schedule-based runs of published pipelines per subscription per month is 100,000
NOTE
If you want to increase this limit, contact Microsoft Support.
Container instances
There is also a limit on the number of container instances that you can spin up in a given time period (scoped
hourly) or across your entire subscription.
RESOURCE DEFAULT LIMIT
Container groups per subscription 1001
Number of containers per container group 60
Number of volumes per container group 20
Ports per IP 5
Container instance log size - running instance 4 MB
Container instance log size - stopped instance 16 KB or 1,000 lines
Container creates per hour 3001
Container creates per 5 minutes 1001
Container deletes per hour 3001
Container deletes per 5 minutes 1001
1To request a limit increase, create an Azure Support request.
For a more detailed and up-to-date list of quota limits, check the Azure-wide quota article here.
Storage
There is a limit on the number of storage accounts per region as well in a given subscription. The default limit is
200 and includes both Standard and Premium Storage accounts. If you require more than 200 storage accounts in
a given region, make a request through Azure Support. The Azure Storage team will review your business case and
may approve up to 250 storage accounts for a given region.
Find your quotas

Viewing your quota for various resources, such as Virtual Machines, Storage, Network, is easy through the Azure
portal.
1. On the left pane, select All services and then select Subscriptions under the General category.
2. From the list of subscriptions, select the subscription whose quota you are looking for.
There is a caveat, specifically for viewing the Azure Machine Learning Compute quota. As mentioned
above, that quota is separate from the compute quota on your subscription.
3. On the left pane, select Machine Learning service and then select any workspace from the list shown
4. On the next blade, under the Support + troubleshooting section select Usage + quotas to view your
current quota limits and usage.
5. Select a subscription to view the quota limits. Remember to filter to the region you are interested in.
Request quota increases

If you want to raise the limit or quota above the default limit, open an online customer support request at no
charge.
The limits can't be raised above the maximum limit value shown in the tables. If there is no maximum limit, then
the resource doesn't have adjustable limits. This article covers the quota increase process in more detail.
When requesting a quota increase, you need to select the service you are requesting to raise the quota against,
which could be services such as Machine Learning service quota, Container instances or Storage quota. In addition
for Azure Machine Learning Compute, you can simply click on the Request Quota button while viewing the quota
following the steps above.
NOTE
Free Trial subscriptions are not eligible for limit or quota increases. If you have a Free Trial subscription, you can upgrade to a
Pay-As-You-Go subscription. For more information, see Upgrade Azure Free Trial to Pay-As-You-Go and Free Trial
subscription FAQ.
Export or delete your Machine Learning service
workspace data
In Azure Machine Learning, you can export or delete your workspace data with the authenticated REST API. This
article tells you how.
NOTE
For information about viewing or deleting personal data, see Azure Data Subject Requests for the GDPR. For more
information about GDPR, see the GDPR section of the Service Trust portal.
NOTE
This article provides steps for how to delete personal data from the device or service and can be used to support your
obligations under the GDPR. If you’re looking for general info about GDPR, see the GDPR section of the Service Trust portal.
Control your workspace data

In-product data stored by Azure Machine Learning Services is available for export and deletion through the Azure
portal, CLI, SDK, and authenticated REST APIs. Telemetry data can be accessed through the Azure Privacy portal.
In Azure Machine Learning Services, personal data consists of user information in run history documents and
telemetry records of some user interactions with the service.
Delete workspace data with the REST API

In order to delete data, the following API calls can be made with the HTTP DELETE verb. These are authorized by
having an Authorization: Bearer <arm-token> header in the request, where <arm-token> is the AAD access token
for the https://management.core.windows.net/ endpoint.
To learn how to get this token and call Azure endpoints, see Azure REST API documentation.
In the examples following, replace the text in {} with the instance names that determine the associated resource.
Delete an entire workspace
Use this call to delete an entire workspace.
WARNING
All information will be deleted and the workspace will no longer be usable.
https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Micros
oft.MachineLearningServices/workspaces/{workspaceName}?api-version=2018-03-01-preview
Delete models
Use this call to get a list of models and their IDs:
https://{location}.modelmanagement.azureml.net/api/subscriptions/{subscriptionId}/resourceGroups/{resourceGroup
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/models?api-version=2018-03-01-preview
Individual models can be deleted with:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/models/{id}?api-version=2018-03-01-
preview
Delete assets
Use this call to get a list of assets and their IDs:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/assets?api-version=2018-03-01-preview
Individual assets can be deleted with:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/assets/{id}?api-version=2018-03-01-
preview
Delete images
Use this call to get a list of images and their IDs:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/images?api-version=2018-03-01-preview
Individual images can be deleted with:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/images/{id}?api-version=2018-03-01-
preview
Delete services
Use this call to get a list of services and their IDs:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/services?api-version=2018-03-01-
preview
Individual services can be deleted with:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/services/{id}?api-version=2018-03-01-
preview
Export service data with the REST API

In order to export data, the following API calls can be made with the HTTP GET verb. These are authorized by
having an Authorization: Bearer <arm-token> header in the request, where <arm-token> is the AAD access token
for the endpoint https://management.core.windows.net/
To learn how to get this token and call Azure endpoints, see Azure REST API documentation.
In the examples following, replace the text in {} with the instance names that determine the associated resource.
Export Workspace information
Use this call to get a list of all workspaces:
oft.MachineLearningServices/workspaces?api-version=2018-03-01-preview
Information about an individual workspace can be obtained by:
oft.MachineLearningServices/workspaces/{workspaceName}?api-version=2018-03-01-preview
Export Compute Information

All compute targets attached to a workspace can be obtained by:
https://management.azure.com/subscriptions/{subscriptionId}/resourceGroup/{resourceGroupName}/providers/Microso
ft.MachineLearningServices/workspaces/{workspaceName}/computes?api-version=2018-03-01-preview
Information about a single compute target can be obtained by:
https://management.azure.com/subscriptions/{subscriptionId}/resourceGroup/{resourceGroupName}/providers/Microso
ft.MachineLearningServices/workspaces/{workspaceName}/computes/{computeName}?api-version=2018-03-01-preview
Export run history data

Use this call to get a list of all experiments and their information:
https://{location}.experiments.azureml.net/history/v1.0/subscriptions/{subscriptionId}/resourceGroups/{resource
GroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/experiments
All the runs for a particular experiment can be obtained by:
GroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/experiments/{experimentName}/
runs
Run history items can be obtained by:
runs/{runId}
All run metrics for an experiment can be obtained by:
metrics
A single run metric can be obtained by:
metrics/{metricId}
Export artifacts
Use this call to get a list of artifacts and their paths:
https://{location}.experiments.azureml.net/artifact/v1.0/subscriptions/{subscriptionId}/resourceGroups/{resourc
eGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/artifacts/origins/Experiment
Run/containers/{runId}
Export notifications
Use this call to get a list of stored tasks:
https://{location}.experiments.azureml.net/notification/v1.0/subscriptions/{subscriptionId}/resourceGroups/{res
ourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/tasks
Notifications for a single task can be obtained by:
https://{location}.experiments.azureml.net/notification/v1.0/subscriptions/{subscriptionId}/resourceGroups/{res
ourceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}tasks/{taskId}
Export data stores

Use this call to get a list of data stores:
https://{location}.experiments.azureml.net/datastore/v1.0/subscriptions/{subscriptionId}/resourceGroups/{resour
ceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/datastores
Individual data stores can be obtained by:
https://{location}.experiments.azureml.net/datastore/v1.0/subscriptions/{subscriptionId}/resourceGroups/{resour
ceGroupName}/providers/Microsoft.MachineLearningServices/workspaces/{workspaceName}/datastores/{name}
Export models
Use this call to get a list of models and their IDs:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/models?api-version=2018-03-01-preview
Individual models can be obtained by:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/models/{id}?api-version=2018-03-01-
preview
Export assets
Use this call to get a list of assets and their IDs:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/assets?api-version=2018-03-01-preview
Individual assets can be obtained by:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/assets/{id}?api-version=2018-03-01-
preview
Export images
Use this call to get a list of images and their IDs:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/images?api-version=2018-03-01-preview
Individual images can be obtained by:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/images/{id}?api-version=2018-03-01-
preview
Export services
Use this call to get a list of services and their IDs:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/services?api-version=2018-03-01-
preview
Individual services can be obtained by:
Name}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}/services/{id}?api-version=2018-03-01-
preview
Export Pipeline Experiments

Individual experiments can be obtained by:
https://{location}.aether.ms/api/v1.0/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/provide
rs/Microsoft.MachineLearningServices/workspaces/{workspaceName}/Experiments/{experimentId}
Export Pipeline Graphs

Individual graphs can be obtained by:
rs/Microsoft.MachineLearningServices/workspaces/{workspaceName}/Graphs/{graphId}
Export Pipeline Modules

Modules can be obtained by:
rs/Microsoft.MachineLearningServices/workspaces/{workspaceName}/Modules/{id}
Export Pipeline Templates

Templates can be obtained by:
rs/Microsoft.MachineLearningServices/workspaces/{workspaceName}/Templates/{templateId}
Export Pipeline Data Sources

Data Sources can be obtained by:
rs/Microsoft.MachineLearningServices/workspaces/{workspaceName}/DataSources/{id}
Delete visual interface assets

In the visual interface where you created your experiment, delete individual assets:
1. On the left, select the type of asset you want to delete.
2. In the list, select the individual assets to delete.

3. On the bottom, select Delete.
Export visual interface data

In the visual interface where you created your experiment, export data you have added:
1. On the left, select Data.
2. On the top, select My Datasets or Samples to locate the data you want to export.
3. In the list, select the individual datasets to export.
4. On the bottom, select Download.
Use the CLI extension for Azure Machine Learning
service
The Azure Machine Learning CLI is an extension to the Azure CLI, a cross-platform command-line interface for
the Azure platform. This extension provides commands for working with the Azure Machine Learning service.
It allows you to automate your machine learning activities. The following list provides some example actions
that you can do with the CLI extension:
Run experiments to create machine learning models
Register machine learning models for customer usage
Package, deploy, and track the lifecycle of your machine learning models
The CLI is not a replacement for the Azure Machine Learning SDK. It is a complementary tool that is optimized
to handle highly parameterized tasks which suit themselves well to automation.
Prerequisites
To use the CLI, you must have an Azure subscription. If you don’t have an Azure subscription, create a
The Azure CLI.
Full reference docs

Find the full reference docs for the azure-cli-ml extension of Azure CLI.
Install the extension

To install the Machine Learning CLI extension, use the following command:
TIP
Example files you can use with the commands below can be found here.
When prompted, select y to install the extension.

To verify that the extension has been installed, use the following command to display a list of ML -specific
subcommands:
az ml -h
Update the extension

To update the Machine Learning CLI extension, use the following command:
az extension update -n azure-cli-ml
Remove the extension

To remove the CLI extension, use the following command:
az extension remove -n azure-cli-ml
Resource management
The following commands demonstrate how to use the CLI to manage resources used by Azure Machine
Learning.
If you do not already have one, create a resource group:
az group create -n myresourcegroup -l westus2
Create an Azure Machine Learning service workspace:
az ml workspace create -w myworkspace -g myresourcegroup
For more information, see az ml workspace create.

Attach a workspace configuration to a folder to enable CLI contextual awareness.
az ml folder attach -w myworkspace -g myresourcegroup
This command creates a .azureml subdirectory that contains example runconfig and conda
environment files. It also contains a config.json file that is used to communicate with your Azure
Machine Learning workspace.
For more information, see az ml folder attach.
Attach an Azure blob container as a Datastore.
az ml datastore attach-blob -n datastorename -a accountname -c containername
For more information, see az ml datastore attach-blob.

Upload files to a Datastore.
az ml datastore upload -n datastorename -p sourcepath
For more information, see az ml datastore upload.

Attach an AKS cluster as a Compute Target.
az ml computetarget attach aks -n myaks -i myaksresourceid -g myresourcegroup -w myworkspace
For more information, see az ml computetarget attach aks

Create a new AMLcompute target.
az ml computetarget create amlcompute -n cpu --min-nodes 1 --max-nodes 1 -s STANDARD_D3_V2
For more information, see az ml computetarget create amlcompute.
Run experiments
Start a run of your experiment. When using this command, specify the name of the runconfig file (the
text before *.runconfig if you are looking at your file system) against the -c parameter.
az ml run submit-script -c sklearn -e testexperiment train.py
TIP
The az ml folder attach command creates a .azureml subdirectory, which contains two example runconfig
files.
If you have a Python script that creates a run configuration object programmatically, you can use
RunConfig.save() to save it as a runconfig file.
For more example runconfig files, see https://github.com/MicrosoftDocs/pipelines-azureml/tree/master/.azureml.
For more information, see az ml run submit-script.

View a list of experiments:
az ml experiment list
For more information, see az ml experiment list.
Environment management
The following commands demonstrate how to create, register, and list Azure Machine Learning service
environments for your workspace:
Create scaffolding files for an environment:
az ml environment scaffold -n myenv -d myenvdirectory
For more information, see az ml environment scaffold.

Register an environment:
az ml environment register -d myenvdirectory
For more information, see az ml environment register.

List registered environments:
az ml environment list
For more information, see az ml environment list.
Download a registered environment:
az ml environment download -n myenv -d downloaddirectory
For more information, see az ml environment download.
Model registration, profiling, deployment

The following commands demonstrate how to register a trained model, and then deploy it as a production
service:
Register a model with Azure Machine Learning:
az ml model register -n mymodel -p sklearn_regression_model.pkl
For more information, see az ml model register.

OPTIONAL Profile your model to get optimal CPU and memory values for deployment.
az ml model profile -n myprofile -m mymodel:1 --ic inferenceconfig.json -d "{\"data\":

[[1,2,3,4,5,6,7,8,9,10],[10,9,8,7,6,5,4,3,2,1]]}" -t myprofileresult.json
For more information, see az ml model profile.

Deploy your model to AKS
az ml model deploy -n myservice -m mymodel:1 --ic inferenceconfig.json --dc deploymentconfig.json --

ct akscomputetarget
For more information on the inference configuration file schema, see Inference configuration schema.
For more information on the deployment configuration file schema, see Deployment configuration
schema.
For more information, see az ml model deploy.
Inference configuration schema

The entries in the inferenceconfig.json document map to the parameters for the InferenceConfig class. The
following table describes the mapping between entities in the JSON document and the parameters for the
method:
entryScript entry_script Path to local file that contains the

code to run for the image.
runtime runtime Which runtime to use for the image.

Current supported runtimes are
'spark-py' and 'python'.
condaFile conda_file Optional. Path to local file containing a

conda environment definition to use
for the image.
extraDockerFileSteps extra_docker_file_steps Optional. Path to local file containing

additional Docker steps to run when
setting up image.
sourceDirectory source_directory Optional. Path to folders that contains

all files to create the image.
enableGpu enable_gpu Optional. Whether to enable GPU

support in the image. The GPU image
must be used on Microsoft Azure
Service. For example, Azure Container
Instances, Azure Machine Learning
Compute, Azure Virtual Machines, and
Azure Kubernetes Service. Defaults to
False.
baseImage base_image Optional. A custom image to be used

as base image. If no base image is
given, then the base image will be
used based off of given runtime
parameter.
baseImageRegistry base_image_registry Optional. Image registry that contains

the base image.
cudaVersion cuda_version Optional. Version of CUDA to install

for images that need GPU support.
The GPU image must be used on
Microsoft Azure Services. For example,
Azure Container Instances, Azure
Machine Learning Compute, Azure
Virtual Machines, and Azure
Kubernetes Service. Supported
versions are 9.0, 9.1, and 10.0. If
'enable_gpu' is set, defaults to '9.1'.
description description A description for this image.
The following JSON is an example inference configuration for use with the CLI:
{
"condaFile": "myenv.yml",
"enableGpu": false,
"baseImage": null,
"baseImageRegistry": null
}
Deployment configuration schema

Local deployment configuration schema
LocalWebservice.deploy_configuration. The following table describes the mapping between the entities in the
JSON document and the parameters for the method:
computeType NA The compute target. For local, the

value must be local .
port port The local port on which to expose the

service's HTTP endpoint.
{
"computeType": "local",
"port": 32267
}
Azure Container Instance deployment configuration schema

AciWebservice.deploy_configuration. The following table describes the mapping between the entities in the
computeType NA The compute target. For ACI, the value

must be ACI .

entities.
cpu cpu_cores The number of CPU cores to allocate.

Defaults, 0.1

0.5
location location The Azure region to deploy this

Webservice to. If not specified the
Workspace location will be used. More
details on available regions can be
found here: ACI Regions
authEnabled auth_enabled Whether to enable auth for this

sslEnabled ssl_enabled Whether to enable SSL for this

Webservice. Defaults to False.
appInsightsEnabled enable_app_insights Whether to enable AppInsights for

this Webservice. Defaults to False
sslCertificate ssl_cert_pem_file The cert file needed if SSL is enabled
sslKey ssl_key_pem_file The key file needed if SSL is enabled
cname ssl_cname The cname for if SSL is enabled
dnsNameLabel dns_name_label The dns name label for the scoring

endpoint. If not specified a unique dns
name label will be generated for the
scoring endpoint.
{
"computeType": "aci",
{
"cpu": 0.5,
"memoryInGB": 1.0
},
"sslEnabled": false,
"appInsightsEnabled": false
}
Azure Kubernetes Service deployment configuration schema

AksWebservice.deploy_configuration. The following table describes the mapping between the entities in the
computeType NA The compute target. For AKS, the

value must be aks .
autoScaler NA Contains configuration elements for

autoscale. See the autoscaler table.
autoscaleEnabled autoscale_enabled Whether to enable autoscaling for the

web service. If numReplicas = 0 ,
True ; otherwise, False .
minReplicas autoscale_min_replicas The minimum number of containers to

Default, 1 .
maxReplicas autoscale_max_replicas The maximum number of containers

to use when autoscaling this web
service. Default, 10 .
refreshPeriodInSeconds autoscale_refresh_seconds How often the autoscaler attempts to

scale this web service. Default, 1 .
targetUtilization autoscale_target_utilization The target utilization (in percent out of

100) that the autoscaler should
attempt to maintain for this web
service. Default, 70 .
dataCollection NA Contains configuration elements for

data collection.
storageEnabled collect_model_data Whether to enable model data

collection for the web service. Default,
False .
authEnabled auth_enabled Whether or not to enable key

authentication for the web service.
Both tokenAuthEnabled and
authEnabled cannot be True .
Default, True .
tokenAuthEnabled token_auth_enabled Whether or not to enable token

authentication for the web service.
Both tokenAuthEnabled and
authEnabled cannot be True .
Default, False .

entities.
cpu cpu_cores The number of CPU cores to allocate

for this web service. Defaults, 0.1

0.5
appInsightsEnabled enable_app_insights Whether to enable Application

Insights logging for the web service.
Default, False .
scoringTimeoutMs scoring_timeout_ms A timeout to enforce for scoring calls

to the web service. Default, 60000 .
maxConcurrentRequestsPerContainer replica_max_concurrent_requests The maximum concurrent requests per

node for this web service. Default, 1 .
maxQueueWaitMs max_request_wait_time The maximum time a request will stay

in thee queue (in milliseconds) before
a 503 error is returned. Default, 500 .
numReplicas num_replicas The number of containers to allocate

for this web service. No default value.
If this parameter is not set, the
autoscaler is enabled by default.
keys NA Contains configuration elements for

keys.
primaryKey primary_key A primary auth key to use for this

Webservice
secondaryKey secondary_key A secondary auth key to use for this

Webservice
gpuCores gpu_cores The number of GPU cores to allocate

for this Webservice. Default is 1. Only
supports whole number values.
livenessProbeRequirements NA Contains configuration elements for

liveness probe requirements.
periodSeconds period_seconds How often (in seconds) to perform the

liveness probe. Default to 10 seconds.
Minimum value is 1.
initialDelaySeconds initial_delay_seconds Number of seconds after the container

has started before liveness probes are
initiated. Defaults to 310
timeoutSeconds timeout_seconds Number of seconds after which the

liveness probe times out. Defaults to 2
seconds. Minimum value is 1
successThreshold success_threshold Minimum consecutive successes for

the liveness probe to be considered
successful after having failed. Defaults
to 1. Minimum value is 1.
failureThreshold failure_threshold When a Pod starts and the liveness

probe fails, Kubernetes will try
failureThreshold times before giving
up. Defaults to 3. Minimum value is 1.
namespace namespace The Kubernetes namespace that the

webservice is deployed into. Up to 63
lowercase alphanumeric ('a'-'z', '0'-'9')
and hyphen ('-') characters. The first
and last characters can't be hyphens.
{
"computeType": "aks",
"autoScaler":
{
"autoscaleEnabled": true,
"minReplicas": 1,
"maxReplicas": 3,
"refreshPeriodInSeconds": 1,
"targetUtilization": 70
},
"dataCollection":
{
"storageEnabled": true
},
{
"cpu": 0.5,
"memoryInGB": 1.0
}
}
Next steps
Command reference for the Machine Learning CLI extension.
Train and deploy machine learning models using Azure Pipelines
Algorithm & module reference overview
This reference content provides the technical background on each of the machine learning algorithms and
modules available in the visual interface (preview ) of Azure Machine Learning service.
Each module represents a set of code that can run independently and perform a machine learning task, given
the required inputs. A module might contain a particular algorithm, or perform a task that is important in
machine learning, such as missing value replacement, or statistical analysis.
TIP
In any experiment in the visual interface, you can get information about a specific module. Select the module, then
select the more help link in the Quick Help pane.
Modules
Modules are organized by functionality:
FUNCTIONALITY DESCRIPTION MODULE
Data format conversions Convert data among various file Convert to CSV
formats used in machine learning,
Data input and output Move data from cloud sources into Import Data
your experiment. Write your results Export Data
or intermediate data to Azure Enter Data Manually
Storage, a SQL database, or Hive,
while running an experiment, or use
cloud storage to exchange data
between experiments.
Data transformation Operations on data that are unique Select Columns in Dataset
to machine learning, such as Edit Metadata
normalizing or binning data, feature Clean Missing Data
selection, and dimensionality Feature Hashing
reduction. Extract N Gram Features from Text
Add Columns
Add Rows
Remove Duplicate Rows
Preprocess Text
Join Data
Split Data
Normalize Data
Partition and Sample
Python and R modules Write code and embed it in a module Execute Python Script
to integrate Python and R with your Create Python Model
experiment. Execute R Script
Machine learning algorithms:

FUNCTIONALITY DESCRIPTION MODULE
Classification Predict a class. Choose from binary Multiclass Decision Forest

(two-class) or multiclass algorithms. Multiclass Boosted Decision Tree
Multiclass Logistic Regression
Multiclass Neural Network
Two-Class Logistic Regression
Two-Class Averaged Perceptron
Two-Class Boosted Decision Tree
Two-Class Decision Forest
Two-Class Neural Network
Two‑Class Support Vector Machine
Clustering Group data together. K-Means Clustering
Regression Predict a value. Linear Regression

Neural Network Regression
Decision Forest Regression
Boosted Decision Tree Regression
Build and evaluate models:
Train Run data through the algorithm. Train Model

Evaluate Model Measure the accuracy of the trained Evaluate Model

model.
Score Get predictions from the model Apply Transformation

you've just trained. Assign Data to Clusters
Score Model
Error messages
Learn about the error messages and exception codes you might encounter using modules in the visual
interface of Azure Machine Learning service.
Convert to CSV module
This article describes a module of the visual interface (preview ) for Azure Machine Learning service.
Use this module to convert a dataset into a CSV format that can be downloaded, exported, or shared with R or
Python script modules.
More about the CSV format
The CSV format, which stands for "comma-separated values", is a file format used by many external machine
learning tools. CSV is a common interchange format when working with open-source languages such as R or
Python.
Even if you do most of your work in Azure Machine Learning, there are times when you might find it handy to
convert your dataset to CSV to use in external tools. For example:
Download the CSV file to open it with Excel, or import it into a relational database.
Save the CSV file to cloud storage and connect to it from Power BI to create visualizations.
Use the CSV format to prepare data for use in R and Python. Just right-click the output of the module to
generate the code needed to access the data directly from Python or a Jupyter notebook.
When you convert a dataset to CSV, the file is saved in your Azure ML workspace. You can use an Azure storage
utility to open and use the file directly, or you can right-click the module output and download the CSV file to your
computer, or use it in R or Python code.
How to configure Convert to CSV

1. Add the Convert to CSV module to your experiment. You can find this module in the Data Format
Conversions group in the interface.
2. Connect it to any module that outputs a dataset.
Results
Double-click the output of Convert to CSV, and select one of these options.
Result Dataset -> Download: Immediately opens a copy of the data in CSV format that you can save to a
local folder. If you do not specify a folder, a default file name is applied and the CSV file is saved in the local
Downloads library.
Result Dataset -> Save as Dataset: Saves the CSV file back to the Azure ML workspace as a separate
dataset.
Generate Data Access Code: Azure ML generates two sets of code for you to access the data, either by
using Python or by using R. To access the data, copy the code snippet into your application. (Generate Data
Access Code will come soon.)
Next steps
See the set of modules available to Azure Machine Learning service.
Import Data module
Use this module to load data into a machine learning experiment from existing cloud data services.
The module now features a wizard to help you choose a storage option and select from among existing
subscriptions and accounts to quickly configure all options. Need to edit an existing data connection? No
problem; the wizard loads all previous configuration details so that you don't have to start again from scratch.
After you define the data you want and connect to the source, Import Data infers the data type of each column
based on the values it contains, and loads the data into your Azure Machine Learning workspace. The output of
Import Data is a dataset that can be used with any experiment.
If your source data changes, you can refresh the dataset and add new data by rerunning Import Data. However, if
you don't want to re-read from the source each time you run the experiment, select the Use cached results
option to TRUE. When this option is selected, the module checks whether the experiment has run previously
using the same source and same input options. If a previous run is found, the data in the cache is used, instead of
reloading the data from the source.
Data sources
The Import Data module supports the following data sources. Click the links for detailed instructions and
examples of using each data source.
If you are not sure how or where you should store your data, see this guide to common data scenarios in the data
science process: Scenarios for advanced analytics in Azure Machine Learning.
DATA SOURCE USE WITH
Web URL via HTTP Get data that is hosted on a web URL that uses HTTP and
that has been provided in the CSV, TSV, ARFF, or SvmLight
formats
Import from Azure Blob Storage Get data that is stored in the Azure blob service
How to use Import Data

1. Add the Import Data module to your experiment. You can find this module in the Data Input and
Output category in the interface.
2. Click Launch Data Import Wizard to configure the data source using a wizard.
The wizard gets the account name and credentials, and help you configure other options. If you are editing
an existing configuration, it loads the current values first.
3. If you do not want to use the wizard, click Data source, and choose the type of cloud-based storage you
are reading from.
Additional settings depend on the type of storage you choose, and whether the storage is secured or not.
You might need to provide the account name, file type, or credentials. Some sources do not require
authentication; for others, you might need to know the account name, a key, or container name.
4. Select the Use cached results option if you want to cache the dataset for reuse on successive runs.
Assuming there have been no other changes to module parameters, the experiment loads the data only the
first time the module is run, and thereafter uses a cached version of the dataset.
Deselect this option if you need to reload the data each time you run the experiment.
When Import Data loads the data into the interface, it infers the data type of each column based on the
values it contains, either numerical or categorical.
If a header is present, the header is used to name the columns of the output dataset.
If there are no existing column headers in the data, new column names are generated using the
format col1, col2,… , coln*.
Results
When import completes, click the output dataset and select Visualize to see if the data was imported
successfully.
If you want to save the data for re-use, rather than importing a new set of data each time the experiment is run,
right-click the output and select Save as Dataset. Choose a name for the dataset. The saved dataset preserves
the data at the time of saving, and data is not updated when the experiment is rerun, even if the dataset in the
experiment changes. This can be handy for taking snapshots of data.
After importing the data, it might need some additional preparations for modeling and analysis:
Use Edit Metadata to change column names, to handle a column as a different data type, or to indicate that
some columns are labels or features.
Use Select Columns in Dataset to select a subset of columns to transform or use in modeling. The
transformed or removed columns can easily be rejoined to the original dataset by using the Add Columns
module.
Use Partition and Sample to divide the dataset, perform sampling, or get the top n rows.
Next steps
Export Data module
Use this module to save results, intermediate data, and working data from your experiments into cloud storage
destinations outside Azure Machine Learning.
This module supports exporting or saving your data to the following cloud data services:
Export to Azure Blob Storage: Saves data to the Blob service in Azure. Data in the Blob service can be
shared publicly or saved in secured application data stores.
How to configure Export Data

1. Add the Export Data module to your experiment in the interface. You can find this module in the Input
and Output category.
2. Connect Export Data to the module that contains the data you want to export.
3. Double-click Export Data to open the Properties pane.
4. For Data destination, select the type of cloud storage where you'll save your data. If you make any
changes to this option, all other properties are reset. So be sure to choose this option first!
5. Provide an account name and authentication method required to access the specified storage account.
Export to Azure Blob Storage is the only option in private preview. Below shows how to set the module.
a. The Azure blob service is for storing large amounts of data, including binary data. There are two
types of blob storage: public blobs, and blobs that require login credentials.
b. For Authentication type, choose Public (SAS ) if you know that the storage supports access via a
SAS URL.
A SAS URL is a special type of URL that can be generated by using an Azure storage utility, and is
available for only a limited time. It contains all the information that is needed for authentication and
download.
For URI, type or paste the full URI that defines the account and the public blob.
For file format, CSV and TSV are supported.
c. For private accounts, choose Account, and provide the account name and the account key, so that
the experiment can write to the storage account.
Account name: Type or paste the name of the account where you want to save the data. For
example, if the full URL of the storage account is http://myshared.blob.core.windows.net , you
would type myshared .
Account key: Paste the storage access key that is associated with the account.
Path to container, directory, or blob: Type the name of the blob where the exported data
will be stored. For example, to save the results of your experiment to a new blob named
results01.csv in the container predictions in an account named mymldata, the full URL for
the blob would be http://mymldata.blob.core.windows.net/predictions/results01.csv .
Therefore, in the field Path to container, directory, or blob, you would specify the
container and blob name as follows: predictions/results01.csv
If you specify the name of a blob that does not already exist, Azure creates the blob for you.
When writing to an existing blob, you can specify that current contents of the blob be
overwritten by setting the property, Azure blob storage write mode. By default, this
property is set to Error, meaning that an error is raised whenever an existing blob file of the
same name is found.
d. For File format for blob file, select the format in which data should be stored.
CSV: Comma-separated values (CSV ) are the default storage format. To export column
headings together with the data, select the option, Write blob header row. For more
information about the comma- delimited format used in Azure Machine Learning, see
Convert to CSV.
TSV: Tab-separated values (TSV ) format is compatible with many machine learning tools. To
export column headings together with the data, select the option, Write blob header row.
e. Use cached results: Select this option if you want to avoid rewriting the results to the blob file each
time you run the experiment. If there are no other changes to module parameters, the experiment
writes the results only the first time the module is run, or when there are changes to the data.
f. Run the experiment.
Next steps
Enter Data Manually module
Use this module to create a small dataset by typing values. The dataset can have multiple columns.
This module can be helpful in scenarios such as these:
Generating a small set of values for testing
Creating a short list of labels
Typing a list of column names to insert in a dataset
Enter Data Manually

1. Add the Enter Data Manually module to your experiment. You can find this module in the Data Input and
Output category in Azure Machine Learning.
2. For DataFormat, select one of the following options. These options determine how the data that you
provide should be parsed. The requirements for each format differ greatly, so be sure to read the related
topics.
ARFF. The attribute-relation file format, used by Weka.
CSV. Comma-separated values format. For more information, see Convert to CSV.
SVMLight. A format used by Vowpal Wabbit and other machine learning frameworks.
TSV. Tab-separated values format.
If you choose a format and do not provide data that meets the format specifications, a run-time error
occurs.
3. Click inside the Data text box to start entering data. The following formats require special attention:
CSV: To create multiple columns, paste in comma-separated text, or type multiple columns using
commas between fields.
If you select the HasHeader option, you can use the first row of values as the column heading.
If you deselect this option, the columns names, Col1, Col2, and so forth, are used. You can add or
change columns names later using Edit Metadata.
TSV: To create multiple columns, paste in tab-separated text, or type multiple columns using tabs
between fields.
If you select the HasHeader option, you can use the first row of values as the column heading.
If you deselect this option, the columns names, Col1, Col2, and so forth, are used. You can add or
change columns names later using Edit Metadata.
ARFF: Paste in an existing ARFF format file. If you are typing values directly, be sure to add the
optional header and required attribute fields at the beginning of the data.
For example, the following header and attribute rows could be added to a simple list. The column
heading would be SampleText .
% Title: SampleText.ARFF
% Source: Enter Data module
@ATTRIBUTE SampleText STRING
@DATA
\<type first data row here>
SVMLight: Type or paste in values using the SVMLight format.

For example, the following sample represents the first couple lines of the Blood Donation dataset, in
SVMight format:
# features are [Recency], [Frequency], [Monetary], [Time]

1 1:2 2:50 3:12500 4:98
1 1:0 2:13 3:3250 4:28
When you run the Enter Data Manually module, these lines are converted to a dataset of columns
and index values as follows:
COL1 COL2 COL3 COL4 LABELS
0.00016 0.004 0.999961 0.00784 1
0 0.004 0.999955 0.008615 1
4. Press ENTER after each row, to start a new line.

Be sure to press ENTER after the final row.
If you press ENTER multiple times to add multiple empty trailing rows, the final empty row is removed
trimmed, but other empty rows are treated as missing values.
If you create rows with missing values, you can always filter them out later.
5. Right-click the module and select Run selected to parse the data and load it into your workspace as a
dataset.
To view the dataset, click the output port and select Visualize.
Next steps
Select Columns in Dataset module
Use this module to choose a subset of columns to use in downstream operations. The module does not physically
remove the columns from the source dataset; instead, it creates a subset of columns, much like a database view or
projection.
This module is useful when you need to limit the columns available for a downstream operation, or if you want to
reduce the size of the dataset by removing unneeded columns.
The columns in the dataset are output in the same order as in the original data, even if you specify them in a
different order.
How to use
This module has no parameters. You use the column selector to choose the columns to include or exclude.
Choose columns by name
There are multiple options in the module for choosing columns by name:
Filter and search
Click the BY NAME option.
If you have connected a dataset that is already populated, a list of available columns should appear. If no
columns appear, you might need to run upstream modules to view the column list.
To filter the list, type in the search box. For example, if you type the letter w in the search box, the list is
filtered to show the column names that contain the letter w .
Select columns and click the right arrow button to move the selected columns to the list in the right-hand
pane.
To select a continuous range of column names, press Shift + Click.
To add individual columns to the selection, press Ctrl + Click.
Click the checkmark button to save and close.
Use names in combination with other rules
Click the WITH RULES option.
Choose a rule, such as showing columns of a specific data type.
Then, click individual columns of that type by name, to add them to the selection list.
Type or paste a comma-separated list of column names
If your dataset is wide, it might be easier to use indexes or generated lists of names, rather than selecting
columns individually. Assuming you have prepared the list in advance:
1. Click the WITH RULES option.
2. Select No columns, select Include, and then click inside the text box with the red exclamation mark.
3. Paste in or type a comma-separated list of previously validated column names. You cannot save the
module if any column has an invalid name, so be sure to check the names beforehand.
You can also use this method to specify a list of columns using their index values.
Choose by type
If you use the WITH RULES option, you can apply multiple conditions on the column selections. For example,
you might need to get only feature columns of a numeric data type.
The BEGIN WITH option determines your starting point and is important for understanding the results.
If you select the ALL COLUMNS option, all columns are added to the list. Then, you must use the Exclude
option to remove columns that meet certain conditions.
For example, you might start with all columns and then remove columns by name, or by type.
If you select the NO COLUMNS option, the list of columns starts out empty. You then specify conditions to
add columns to the list.
If you apply multiple rules, each condition is additive. For example, say you start with no columns, and
then add a rule to get all numeric columns. In the Automobile price dataset, that results in 16 columns.
Then, you click the + sign to add a new condition, and select Include all features. The resulting dataset
includes all the numeric columns, plus all the feature columns, including some string feature columns.
Choose by column index
The column index refers to the order of the column within the original dataset.
Columns are numbered sequentially starting at 1.
To get a range of columns, use a hyphen.
Open-ended specifications such as 1- or -3 are not allowed.
Duplicate index values (or column names) are not allowed, and might result in an error.
For example, assuming your dataset has at least eight columns, you could paste in any of the following examples
to return multiple non-contiguous columns:
8,1-4,6
1,3-8
1,3-6,4
the final example does not result in an error; however, it returns a single instance of column 4 .
Change order of columns
The option Allow duplicates and preserve column order in selection starts with an empty list, and adds
columns that you specify by name or by index. Unlike other options, which always return columns in their "natural
order", this option outputs the columns in the order that you name or list them.
For example, in a dataset with the columns Col1, Col2, Col3, and Col4, you could reverse the order of the columns
and leave out column 2, by specifying either of the following lists:
Col4, Col3, Col1
4,3,1
Next steps
Edit Metadata module
This article describes a module of the visual interface (preview ) for the Azure Machine Learning service.
Use the Edit Data module to change metadata that's associated with columns in a dataset. The value and data
type of the dataset will change after use of the Edit Metadata module.
Typical metadata changes might include:
Treating Boolean or numeric columns as categorical values.
Indicating which column contains the class label or contains the values you want to categorize or predict.
Marking columns as features.
Changing date/time values to numeric values or vice versa.
Renaming columns.
Use Edit Metadata anytime you need to modify the definition of a column, typically to meet requirements for a
downstream module. For example, some modules work only with specific data types or require flags on the
columns, such as IsFeature or IsCategorical .
After you perform the required operation, you can reset the metadata to its original state.
Configure Edit Metadata

1. In Azure Machine Learning, add the Edit Metadata module to your experiment and connect the dataset
you want to update. You can find the dataset under Data Transformation in the Manipulate category.
2. Select Launch the column selector and choose the column or set of columns to work with. You can
choose columns individually by name or index, or you can choose a group of columns by type.
3. Select the Data type option if you need to assign a different data type to the selected columns. You
might need to change the data type for certain operations. For example, if your source dataset has
numbers handled as text, you must change them to a numeric data type before using math operations.
The supported data types are String, Integer, Double, Boolean, and DateTime.
If you select multiple columns, you must apply the metadata changes to all selected columns. For
example, let's say you choose two or three numeric columns. You can change them all to a string
data type and rename them in one operation. However, you can't change one column to a string
data type and another column from a float to an integer.
If you don't specify a new data type, the column metadata is unchanged.
The column type and values will change after you perform the Edit Metadata operation. You can
recover the original data type at any time by using Edit Metadata to reset the column data type.
NOTE
If you change any type of number to the DateTime type, leave the DateTime Format field blank. Currently it
isn't possible to specify the target data format.
4. Select the Categorical option to specify that the values in the selected columns should be treated as
categories.
For example, you might have a column that contains the numbers 0, 1, and 2, but know that the numbers
actually mean "Smoker," "Non-smoker," and "Unknown." In that case, by flagging the column as
categorical you ensure that the values are used only to group data and not in numeric calculations.
5. Use the Fields option if you want to change the way that Azure Machine Learning uses the data in a
model.
Feature: Use this option to flag a column as a feature in modules that operate only on feature
columns. By default, all columns are initially treated as features.
Label: Use this option to mark the label, which is also known as the predictable attribute or target
variable. Many modules require that exactly one label column is present in the dataset.
In many cases, Azure Machine Learning can infer that a column contains a class label. By setting
this metadata, you can ensure that the column is identified correctly. Setting this option does not
change data values. It changes only the way that some machine-learning algorithms handle the
data.
TIP
Do you have data that doesn't fit into these categories? For example, your dataset might contain values such as
unique identifiers that aren't useful as variables. Sometimes such IDs can cause problems when used in a model.
Fortunately, Azure Machine Learning keeps all of your data, so that you don't have to delete such columns from
the dataset. When you need to perform operations on some special set of columns, just remove all other columns
temporarily by using the Select Columns in Dataset module. Later you can merge the columns back into the
dataset by using the Add Columns module.
6. Use the following options to clear previous selections and restore metadata to the default values.
Clear feature: Use this option to remove the feature flag.
All columns are initially treated as features. For modules that perform mathematical operations,
you might need to use this option in order to prevent numeric columns from being treated as
variables.
Clear label: Use this option to remove the label metadata from the specified column.
Clear score: Use this option to remove the score metadata from the specified column.
You currently can't explicitly mark a column as a score in Azure Machine Learning. However, some
operations result in a column being flagged as a score internally. Also, a custom R module might
output score values.
7. For New column names, enter the new name of the selected column or columns.
Column names can use only characters that are supported by UTF -8 encoding. Empty strings,
nulls, or names that consist entirely of spaces aren't allowed.
To rename multiple columns, enter the names as a comma-separated list in order of the column
indexes.
All selected columns must be renamed. You can't omit or skip columns.
Next steps
See the set of modules available to the Azure Machine Learning service.
Clean Missing Data module
Use this module to remove, replace, or infer missing values.
Data scientists often check data for missing values and then perform various operations to fix the data or insert
new values. The goal of such cleaning operations is to prevent problems caused by missing data that can arise
when training a model.
This module supports multiple type of operations for "cleaning" missing values, including:
Replacing missing values with a placeholder, mean, or other value
Completely removing rows and columns that have missing values
Inferring values based on statistical methods
Using this module does not change your source dataset. Instead, it creates a new dataset in your workspace that
you can use in the subsequent workflow. You can also save the new, cleaned dataset for reuse.
This module also outputs a definition of the transformation used to clean the missing values. You can re-use this
transformation on other datasets that have the same schema, by using the Apply Transformation module.
How to use Clean Missing Data

This module lets you define a cleaning operation. You can also save the cleaning operation so that you can apply it
later to new data. See the following links for a description of how to create and save a cleaning process:
To replace missing values
To apply a cleaning transformation to new data
IMPORTANT
The cleaning method that you use for handling missing values can dramatically affect your results. We recommend that you
experiment with different methods. Consider both the justification for use of a particular method, and the quality of the
results.
Replace missing values

Each time that you apply the Clean Missing Data module to a set of data, the same cleaning operation is applied to
all columns that you select. Therefore, if you need to clean different columns using different methods, use separate
instances of the module.
1. Add the Clean Missing Data module to your experiment, and connect the dataset that has missing values.
2. For Columns to be cleaned, choose the columns that contain the missing values you want to change. You
can choose multiple columns, but you must use the same replacement method in all selected columns.
Therefore, typically you need to clean string columns and numeric columns separately.
For example, to check for missing values in all numeric columns:
a. Open the Column Selector, and select WITH RULES.
b. For BEGIN WITH, select NO COLUMNS.
You can also start with ALL COLUMNS and then exclude columns. Initially, rules are not shown if
you first click ALL COLUMNS, but you can click NO COLUMNS and then click ALL COLUMNS
again to start with all columns and then filter out (exclude) columns based on the name, data type, or
columns index.
c. For Include, select Column type from the dropdown list, and then select Numeric, or a more
specific numeric type.
Any cleaning or replacement method that you choose must be applicable to all columns in the selection. If
the data in any column is incompatible with the specified operation, the module returns an error and stops
the experiment.
3. For Minimum missing value ratio, specify the minimum number of missing values required for the
operation to be performed.
You use this option in combination with Maximum missing value ratio to define the conditions under
which a cleaning operation is performed on the dataset. If there are too many or too few rows that are
missing values, the operation cannot be performed.
The number you enter represents the ratio of missing values to all values in the column. By default, the
Minimum missing value ratio property is set to 0. This means that missing values are cleaned even if
there is only one missing value.
WARNING
This condition must be met by each and every column in order for the specified operation to apply. For example,
assume you selected three columns and then set the minimum ratio of missing values to .2 (20%), but only one
column actually has 20% missing values. In this case, the cleanup operation would apply only to the column with
over 20% missing values. Therefore, the other columns would be unchanged.
If you have any doubt about whether missing values were changed, select the option, Generate missing value
indicator column. A column is appended to the dataset to indicate whether or not each column met the specified
criteria for the minimum and maximum ranges.
4. For Maximum missing value ratio, specify the maximum number of missing values that can be present
for the operation to be performed.
For example, you might want to perform missing value substitution only if 30% or fewer of the rows
contain missing values, but leave the values as-is if more than 30% of rows have missing values.
You define the number as the ratio of missing values to all values in the column. By default, the Maximum
missing value ratio is set to 1. This means that missing values are cleaned even if 100% of the values in
the column are missing.
5. For Cleaning Mode, select one of the following options for replacing or removing missing values:
Custom substitution value: Use this option to specify a placeholder value (such as a 0 or NA) that
applies to all missing values. The value that you specify as a replacement must be compatible with
the data type of the column.
Replace with mean: Calculates the column mean and uses the mean as the replacement value for
each missing value in the column.
Applies only to columns that have Integer, Double, or Boolean data types.
Replace with median: Calculates the column median value, and uses the median value as the
replacement for any missing value in the column.
Applies only to columns that have Integer or Double data types.
Replace with mode: Calculates the mode for the column, and uses the mode as the replacement
value for every missing value in the column.
Applies to columns that have Integer, Double, Boolean, or Categorical data types.
Remove entire row: Completely removes any row in the dataset that has one or more missing
values. This is useful if the missing value can be considered randomly missing.
Remove entire column: Completely removes any column in the dataset that has one or more
missing values.
6. The option Replacement value is available if you have selected the option, Custom substitution value.
Type a new value to use as the replacement value for all missing values in the column.
Note that you can use this option only in columns that have the Integer, Double, Boolean, or Date data
types. For date columns, the replacement value can also be entered as the number of 100-nanosecond ticks
since 1/1/0001 12:00 A.M.
7. Generate missing value indicator column: Select this option if you want to output some indication of
whether the values in the column met the criteria for missing value cleaning. This option is particularly
useful when you are setting up a new cleaning operation and want to make sure it works as designed.
Results
The module returns two outputs:
Cleaned dataset: A dataset comprised of the selected columns, with missing values handled as specified,
along with an indicator column, if you selected that option.
Columns not selected for cleaning are also "passed through".
Cleaning transformation: A data transformation used for cleaning, that can be saved in your workspace
and applied to new data later.
Apply a saved cleaning operation to new data
If you need to repeat cleaning operations often, we recommend that you save your recipe for data cleansing as a
transform, to reuse with the same dataset. Saving a cleaning transformation is particularly useful if you must
frequently re-import and then clean data that has the same schema.
1. Add the Apply Transformation module to your experiment.
2. Add the dataset you want to clean, and connect the dataset to the right-hand input port.
3. Expand the Transforms group in the left-hand pane of the interface. Locate the saved transformation and
drag it into the experiment.
4. Connect the saved transformation to the left input port of Apply Transformation.
When you apply a saved transformation, you cannot select the columns to which the transformation are
applied. That is because the transformation has been already defined and applies automatically to the
columns specified in the original operation.
However, suppose you created a transformation on a subset of numeric columns. You can apply this
transformation to a dataset of mixed column types without raising an error, because the missing values are
changed only in the matching numeric columns.
Next steps
Feature Hashing
Use the Feature Hashing module to transform a stream of English text into a set of integer features. You can then
pass this hashed feature set to a machine learning algorithm to train a text analytics model.
The feature hashing functionality provided in this module is based on the nimbusml framework. For more
information, see NgramHash class.
What is feature hashing

Feature hashing works by converting unique tokens into integers. It operates on the exact strings that you provide
as input and does not perform any linguistic analysis or preprocessing.
For example, take a set of simple sentences like these, followed by a sentiment score. Assume that you want to use
this text to build a model.
USERTEX T SENTIMENT
I loved this book 3
I hated this book 1
This book was great 3
I love books 2
Internally, the Feature Hashing module creates a dictionary of n-grams. For example, the list of bigrams for this
dataset would be something like this:
TERM (BIGRAMS) FREQUENCY
This book 3
I loved 1
I hated 1
I love 1
You can control the size of the n-grams by using the N -grams property. If you choose bigrams, unigrams are also
computed. The dictionary would also include single terms like these:
TERM (UNIGRAMS) FREQUENCY
book 3
I 3
TERM (UNIGRAMS) FREQUENCY
books 1
was 1
After the dictionary has been built, the Feature Hashing module converts the dictionary terms into hash values. It
then computes whether a feature was used in each case. For each row of text data, the module outputs a set of
columns, one column for each hashed feature.
For example, after hashing, the feature columns might look something like this:
RATING HASHING FEATURE 1 HASHING FEATURE 2 HASHING FEATURE 3
4 1 1 0
5 0 0 0
If the value in the column is 0, the row didn't contain the hashed feature.
If the value is 1, the row did contain the feature.
Feature hashing lets you represent text documents of variable-length as numeric feature vectors of equal-length to
achieve dimensionality reduction. If you tried to use the text column for training as is, it would be treated as a
categorical feature column with many distinct values.
Numeric outputs also make it possible to use common machine learning methods including classification,
clustering, and information retrieval. Because lookup operations can use integer hashes rather than string
comparisons, getting the feature weights is also much faster.
Configure Feature Hashing

1. Add the Feature Hashing module to your experiment in visual interface.
2. Connect the dataset that contains the text you want to analyze.
TIP
Because feature hashing does not perform lexical operations such as stemming or truncation, you can sometimes get
better results by doing text preprocessing before applying feature hashing.
3. Set Target columns to the text columns that you want to convert to hashed features.
The columns must be the string data type.
Choosing multiple text columns can have a significant impact on feature dimensionality. For example,
the number of columns for a 10-bit hash goes from 1024 for a single column to 2048 for two
columns.
4. Use Hashing bitsize to specify the number of bits to use when creating the hash table.
The default bit size is 10. For many problems, this value is adequate. You may need more space to avoid
collisions depending on the size of the training text n-grams vocabulary.
5. For N -grams, type a number that defines the maximum length of the n-grams to add to the training
dictionary. An n-gram is a sequence of n words, treated as a unique unit.
For example, if you enter 3, unigrams, bigrams, and trigrams will be created.
Results
After processing is complete, the module outputs a transformed dataset in which the original text column has been
converted to multiple columns, each representing a feature in the text. Depending on how significant the dictionary
is, the resulting dataset can be large:
COLUMN NAME 1 COLUMN TYPE 2
USERTEXT Original data column
SENTIMENT Original data column
USERTEXT - Hashing feature 1 Hashed feature column
USERTEXT - Hashing feature n Hashed feature column
After you create the transformed dataset, you can use it as the input to the Train Model module.
Best practices
The following best practices can help you get the most out of the Feature Hashing module:
Add a Preprocess Text module before using Feature Hashing to preprocess the input text.
Add a Select Columns module after the Feature Hashing module to remove the text columns from the
output data set. You don't need the text columns after the hashing features have been generated.
Consider using these text preprocessing options, to simplify results and improve accuracy:
Word breaking
Stop word removal
Case normalization
Removal of punctuation and special characters
Stemming
The optimal set of preprocessing methods to apply in any individual solution depends on domain, vocabulary, and
business need. Experiment with your data to see which text processing methods are most effective.
Next steps
Extract N-Gram Features from Text
Use the Extract N -Gram Features from Text module to featurize unstructured text data.
This module supports the following scenarios for using an n-gram dictionary:
Create a new n-gram dictionary from a column of free text.
Use an existing set of text features to featurize a free text column.
Score or publish a model that uses n-grams.
Configure Extract N-Gram Features from Text

Create a new n-gram dictionary
1. Add the Extract N -Gram Features from Text module to your experiment and connect the dataset that has
the text you want to process.
2. Use Text column to choose a column of type string that contains the text you want to extract. Since results
are verbose, you can only process a single column at a time.
3. Set Vocabulary mode to Create to indicate that you're creating a new list of n-gram features.
4. Set N -Grams size to indicate the maximum size of the n-grams to extract and store.
For example, if you enter 3, unigrams, bigrams, and trigrams will be created.
5. Weighting function specifies how to build the document feature vector and how to extract vocabulary
from documents.
Binary Weight: Assigns a binary presence value to the extracted n-grams. The value for each n-
gram is 1 when it exists in the given document, and 0 otherwise.
TF Weight: Assigns a term-frequency score (TF) to the extracted n-grams. The value for each n-
gram is its occurrence frequency in the given document.
IDF Weight: Assigns an inverse document frequency score (IDF) to the extracted n-grams. The
value for each n-gram is the log of corpus size divided by its occurrence frequency in the whole
corpus. IDF = log of corpus_size / document_frequency
TF-IDF Weight: Assigns a term frequency/inverse document frequency score (TF/IDF) to the
extracted n-grams. The value for each n-gram is its TF score multiplied by its IDF score.
6. Set Minimum word length to the minimum number of letters that can be used in any single word in an n-
gram.
7. Use Maximum word length to set the maximum number of letters that can be used in any single word in
an n-gram.
By default, up to 25 characters per word or token are allowed.
8. Use Minimum n-gram document absolute frequency, to set the minimum occurrences required for any
n-gram to be included in the n-gram dictionary.
For example, if you use the default value of 5, any n-gram must appear at least five times in the corpus to be
included in the n-gram dictionary.
9. Set Maximum n-gram document ratio to the maximum ratio of the number of rows that contain a
particular n-gram, over the number of rows in the overall corpus.
For example, a ratio of 1 would indicate that, even if a specific n-gram is present in every row, the n-gram
can be added to the n-gram dictionary. More typically, a word that occurs in every row would be considered
a noise word and would be removed. To filter out domain-dependent noise words, try reducing this ratio.
IMPORTANT
The rate of occurrence of particular words is not uniform, but varies from document to document. For example, if you
are analyzing customer comments about a specific product, the product name might be very high frequency and
close to a noise word, but be a significant term in other contexts.
10. Select the option Normalize n-gram feature vectors to normalize the feature vectors. If enabled, each n-
gram feature vector is divided by its L2 norm.
Use an existing n-gram dictionary
1. Add the Extract N -Gram Features from Text module to your experiment and connect the dataset that has
the text you want to process to the Dataset port.
2. Use Text column to select the text column that contains the text you want to featurize. By default, the
module selects all columns of type string. For best results, process a single column at a time.
3. Add the saved dataset containing a previously generated n-gram dictionary, and connect it to the Input
vocabulary port. You can also connect the Result vocabulary output of an upstream instance of the
Extract N -Gram Features from Text module.
4. For Vocabulary mode, select the following update option from the drop-down list:
ReadOnly: Represents the input corpus for the input vocabulary. Rather than computing term
frequencies from the new text dataset (on the left input), the n-gram weights from the input vocabulary
are applied as is.
TIP
Use this option when scoring a text classifier.
5. For all other options, see the property descriptions in the previous section.
Score or publish a model that uses n-grams
1. Copy the Extract N -Gram Features from Text module from the training dataflow to the scoring dataflow.
2. Connect the Result Vocabulary output from the training dataflow to the Input Vocabulary on the scoring
dataflow.
3. In the scoring workflow, modify the Extract N -Gram Features from Text module and make these
changes, leaving all else the same:
Set the Vocabulary mode parameter to ReadOnly.
4. To publish the experiment, save the Result Vocabulary as dataset.
5. Then, connect the saved dataset to the Extract N -Gram Features from Text module in your scoring graph.
Results
The Extract N -Gram Features from Text module creates two types of outputs:
Results dataset: A summary of the analyzed text combined with the n-grams that were extracted. Columns
that you didn't select in the Text column option are passed through to the output. For each column of text
that you analyze, the module generates these columns:
Matrix of n-gram occurrences: The module generates a column for each n-gram found in the total
corpus and adds a score in each column to indicate the weight of the n-gram for that row.
Result vocabulary: The vocabulary contains the actual n-gram dictionary, together with the term
frequency scores that are generated as part of the analysis. You can save the dataset for reuse with a
different set of inputs, or for a later update. You can also reuse the vocabulary for modeling and scoring.
Result vocabulary
The vocabulary contains the n-gram dictionary with the term frequency scores that are generated as part of the
analysis. The scores DF and IDF are generated regardless of other options.
ID: An identifier generated for each unique n-gram.
NGram: The n-gram. Spaces or other word separators are replaced by the underscore character.
DF: The term frequency score for the n-gram in the original corpus.
IDF: The inverse document frequency score for the n-gram in the original corpus.
You can manually update this dataset; however, you may introduce errors. For example:
An error is raised if the module finds duplicate rows with the same key in the input vocabulary. Be sure that no
two rows in the vocabulary have the same word.
The input schema of the vocabulary datasets must match exactly, including column names and column types.
The ID column and DF score column must be of the integer type.
The IDF column must be of type FLOAT.
NOTE
Do not connect the data output to the Train Model module directly. Free text columns should be removed before feeding
into Train Model, otherwise free text columns will be treated as categorical features.
Next steps
Add Columns module
Use this module to concatenate two datasets. You combine all columns from the two datasets that you specify as
inputs to create a single dataset. If you need to concatenate more than two datasets, use several instances of Add
Columns.
How to configure Add Columns

1. Add the Add Columns module to your experiment.
2. Connect the two datasets that you want to concatenate. If you want to combine more than two datasets,
you can chain together several combinations of Add Columns.
It is possible to combine two columns that have a different number of rows. The output dataset is
padded with missing values for each row in the smaller source column.
You cannot choose individual columns to add. All the columns from each dataset are concatenated
when you use Add Columns. Therefore, if you want to add only a subset of the columns, use Select
Columns in Dataset to create a dataset with the columns you want.
Results
After the experiment has run:
To see the first rows of the new dataset, right-click the output of Add Columns and select Visualize.
The number of columns in the new dataset equals the sum of the columns of both input datasets.
If there are two columns with the same name in the input datasets, a numeric suffix is added to the name of the
column. For example, if there are two instances of a column named TargetOutcome, the left column would be
renamed TargetOutcome_1 and the right column would be renamed TargetOutcome_2.
Next steps
Add Rows module
Use this module to concatenate two datasets. In concatenation, the rows of the second dataset are added to the
end of the first dataset.
Concatenation of rows is useful in scenarios such as these:
You have generated a series of evaluation statistics, and you want to combine them into one table for easier
reporting.
You have been working with different datasets, and you want to combine the datasets to create a final
dataset.
How to use Add Rows

To concatenate rows from two datasets, the rows must have exactly the same schema. This means, the same
number of columns, and the same type of data in the columns.
1. Drag the Add Rows module into your experiment, You can find it under Data Transformation, in the
Manipulate category.
2. Connect the datasets to the two input ports. The dataset that you want to append should be connected to
the second (right) port.
3. Run the experiment. The number of rows in the output dataset should equal the sum of the rows of both
input datasets.
If you add the same dataset to both inputs of the Add Rows module, the dataset is duplicated.
Next steps
Remove Duplicate Rows module
Use this module to remove potential duplicates from a dataset.
For example, assume your data looks like the following, and represents multiple records for patients.
PATIENTID INITIALS GENDER AGE ADMITTED
1 F.M. M 53 Jan
2 F.A.M. M 53 Jan
3 F.A.M. M 24 Jan
3 F.M. M 24 Feb
4 F.M. M 23 Feb
F.M. M 23
5 F.A.M. M 53
6 F.A.M. M NaN
7 F.A.M. M NaN
Clearly, this example has multiple columns with potentially duplicate data. Whether they are actually duplicates
depends on your knowledge of the data.
For example, you might know that many patients have the same name. You wouldn't eliminate duplicates
using any name columns, only the ID column. That way, only the rows with duplicate ID values are filtered
out, regardless of whether the patients have the same name or not.
Alternatively, you might decide to allow duplicates in the ID field, and use some other combination of files
to find unique records, such as first name, last name, age, and gender.
To set the criteria for whether a row is duplicate or not, you specify a single column or a set of columns to use as
keys. Two rows are considered duplicates only when the values in all key columns are equal. If any row has
missing value for keys, they will not be considered duplicate rows. For example, if Gender and Age are set as Keys
in above table, row 6 and 7 are not duplicate rows given they have missing value in Age.
When you run the module, it creates a candidate dataset, and returns a set of rows that have no duplicates across
the set of columns you specified.
IMPORTANT
The source dataset is not altered; this module creates a new dataset that is filtered to exclude duplicates, based on the criteria
you specify.
How to use Remove Duplicate Rows
1. Add the module to your experiment. You can find the Remove Duplicate Rows module under Data
Transformation, Manipulation.
2. Connect the dataset that you want to check for duplicate rows.
3. In the Properties pane, under Key column selection filter expression, click Launch column selector, to
choose columns to use in identifying duplicates.
In this context, Key does not mean a unique identifier. All columns that you select using the Column
Selector are designated as key columns. All unselected columns are considered non-key columns. The
combination of columns that you select as keys determines the uniqueness of the records. (Think of it as a
SQL statement that uses multiple equalities joins.)
Examples:
"I want to ensure that IDs are unique": Choose only the ID column.
"I want to ensure that the combination of first name, last name, and ID is unique": Select all three
columns.
4. Use the Retain first duplicate row checkbox to indicate which row to return when duplicates are found:
If selected, the first row is returned and others discarded.
If you uncheck this option, the last duplicate row is kept in the results, and others are discarded.
6. To review the results, right-click the module, select Results dataset, and click Visualize.
TIP
If the results are difficult to understand, or if you want to exclude some columns from consideration, you can remove
columns by using the Select Columns in Dataset module.
Next steps
Preprocess Text
Use the Preprocess Text module to clean and simplify text. It supports these common text processing operations:
Removal of stop-words
Using regular expressions to search for and replace specific target strings
Lemmatization, which converts multiple related words to a single canonical form
Case normalization
Removal of certain classes of characters, such as numbers, special characters, and sequences of repeated
characters such as "aaaa"
Identification and removal of emails and URLs
The Preprocess Text module currently only supports English.
Configure Text Preprocessing

1. Add the Preprocess Text module to your experiment in Azure Machine Learning Service. You can find this
module under Text Analytics.
2. Connect a dataset that has at least one column containing text.
3. Select the language from the Language dropdown list.
4. Text column to clean: Select the column that you want to preprocess.
5. Remove stop words: Select this option if you want to apply a predefined stopword list to the text column.
Stopword lists are language-dependent and customizable.
6. Lemmatization: Select this option if you want words to be represented in their canonical form. This option
is useful for reducing the number of unique occurrences of otherwise similar text tokens.
The lemmatization process is highly language-dependent..
7. Detect sentences: Select this option if you want the module to insert a sentence boundary mark when
performing analysis.
This module uses a series of three pipe characters ||| to represent the sentence terminator.
8. Perform optional find-and-replace operations using regular expressions.
Custom regular expression: Define the text you're searching for.
Custom replacement string: Define a single replacement value.
9. Normalize case to lowercase: Select this option if you want to convert ASCII uppercase characters to
their lowercase forms.
If characters aren't normalized, the same word in uppercase and lowercase letters is considered two
different words.
10. You can also remove the following types of characters or character sequences from the processed output
text:
Remove numbers: Select this option to remove all numeric characters for the specified language.
Identification numbers are domain-dependent and language dependent. If numeric characters are an
integral part of a known word, the number might not be removed.
Remove special characters: Use this option to remove any non-alphanumeric special characters.
Remove duplicate characters: Select this option to remove extra characters in any sequences that
repeat for more than twice. For example, a sequence like "aaaaa" would be reduced to "aa".
Remove email addresses: Select this option to remove any sequence of the format
<string>@<string> .
Remove URLs: Select this option to remove any sequence that includes the following URL prefixes:
http , https , ftp , www
11. Expand verb contractions: This option applies only to languages that use verb contractions; currently,
English only.
For example, by selecting this option, you could replace the phrase "wouldn't stay there" with "would not
stay there".
12. Normalize backslashes to slashes: Select this option to map all instances of \\ to / .
13. Split tokens on special characters: Select this option if you want to break words on characters such as & ,
- , and so forth. This option can also reduce the special characters when it repeats more than twice.
For example, the string MS---WORD would be separated into three tokens, MS , - , and WORD .
Next steps
Join Data
This article describes how to use the Join Data module in the Azure Machine Learning service visual interface to
merge two datasets using a database-style join operation.
How to configure Join Data

To perform a join on two datasets, they should be related by a key column. Composite keys using multiple columns
are also supported.
1. Add the datasets you want to combine, and then drag the Join Data module into your experiment.
You can find the module in the Data Transformation category, under Manipulation.
2. Connect the datasets to the Join Data module.
3. Select Launch column selector to choose key column(s). Remember to choose columns for both the left
and right inputs.
For a single key:
Select a single key column for both inputs.
For a composite key:
Select all the key columns from left input and right input in the same order. The Join Data module will join
the tables when all key columns match. Check the option Allow duplicates and preserve column order
in selection if the column order isn't the same as the original table.
4. Select the Match case option if you want to preserve case sensitivity on a text column join.
5. Use the Join type dropdown list to specify how the datasets should be combined.
Inner Join: An inner join is the most common join operation. It returns the combined rows only
when the values of the key columns match.
Left Outer Join: A left outer join returns joined rows for all rows from the left table. When a row in
the left table has no matching rows in the right table, the returned row contains missing values for all
columns that come from the right table. You can also specify a replacement value for missing values.
Full Outer Join: A full outer join returns all rows from the left table (table1) and from the right table
(table2).
For each of the rows in either table that have no matching rows in the other, the result includes a row
containing missing values.
Left Semi-Join: A left semi-join returns only the values from the left table when the values of the
key columns match.
6. For the option Keep right key columns in joined table:
Select this option to view the keys from both input tables.
Deselect to only return the key columns from the left input.
7. Run the experiment, or select the Join Data module and selected Run Selected to perform the join.
8. To view the results, right-click the Join Data > Results dataset > Visualize.
Next steps
Split Data module
Use this module to divide a dataset into two distinct sets.
This module is particularly useful when you need to separate data into training and testing sets. You can customize
the way that data is divided as well. Some options support randomization of data; others are tailored for a certain
data type or model type.
How to configure
TIP
Before choosing the splitting mode, read all options to determine the type of split you need. If you change the splitting
mode, all other options could be reset.
1. Add the Split Data module to your experiment in the interface. You can find this module under Data
Transformation, in the Sample and Split category.
2. Splitting mode: Choose one of the following modes, depending on the type of data you have, and how
you want to divide it. Each splitting mode has different options. Click the following topics for detailed
instructions and examples.
Split Rows: Use this option if you just want to divide the data into two parts. You can specify the
percentage of data to put in each split, but by default, the data is divided 50-50.
You can also randomize the selection of rows in each group, and use stratified sampling. In stratified
sampling, you must select a single column of data for which you want values to be apportioned
equally among the two result datasets.
Regular Expression Split Choose this option when you want to divide your dataset by testing a
single column for a value.
For example, if you are analyzing sentiment, you could check for the presence of a particular product
name in a text field, and then divide the dataset into rows with the target product name, and those
without.
Relative Expression Split: Use this option whenever you want to apply a condition to a number
column. The number could be a date/time field, a column containing age or dollar amounts, or even
a percentage. For example, you might want to divide your data set depending on the cost of the
items, group people by age ranges, or separate data by a calendar date.
Split Rows
1. Add the Split Data module to your experiment in the interface, and connect the dataset you want to split.
2. For Splitting mode, choose Split rows.
3. Fraction of rows in the first output dataset. Use this option to determine how many rows go into the
first (left-hand) output. All other rows will go to the second (right-hand) output.
The ratio represents the percentage of rows sent to the first output dataset, so you must type a decimal
number between 0 and 1.
For example, if you type 0.75 as the value, the dataset would be split by using a 75:25 ratio, with 75% of the
rows sent to the first output dataset, and 25% sent to the second output dataset.
4. Select the Randomized split option if you want to randomize selection of data into the two groups. This is
the preferred option when creating training and test datasets.
5. Random Seed: Type a non-negative integer value to initialize the pseudorandom sequence of instances to
be used. This default seed is used in all modules that generate random numbers.
Specifying a seed makes the results generally reproducible. If you need to repeat the results of a split
operation, you should specify a seed for the random number generator. Otherwise the random seed is set
by default to 0, which means the initial seed value is obtained from the system clock. As a result, the
distribution of data might be slightly different each time you perform a split.
6. Stratified split: Set this option to True to ensure that the two output datasets contain a representative
sample of the values in the strata column or stratification key column.
With stratified sampling, the data is divided such that each output dataset gets roughly the same
percentage of each target value. For example, you might want to ensure that your training and testing sets
are roughly balanced with regard to the outcome, or with regard ot some other column such as gender.
Regular expression split

1. Add the Split Data module to your experiment, and connect it as input to the dataset you want to split.
2. For Splitting mode, select Regular expression split.
3. In the Regular expression box, type a valid regular expression.
The regular expression should follow Python regular expression syntax.
Based on the regular expression you provide, the dataset is divided into two sets of rows: rows with values
that match the expression and all remaining rows.
Relative expression split.

1. Add the Split Data module to your experiment, and connect it as input to the dataset you want to split.
2. For Splitting mode, select relative expression split.
3. In the Relational expression text box, type an expression that performs a comparison operation, on a
single column:
Numeric column:
The column contains numbers of any numeric data type, including date/time data types.
The expression can reference a maximum of one column name.
Use the ampersand character (&) for the AND operation and use the pipe character (|) for the OR
operation.
The following operators are supported: < , > , <= , >= , == , !=
You cannot group operations by using ( and ) .

String column:
The following operators are supported: == , !=

The expression divides the dataset into two sets of rows: rows with values that meet the condition, and all
remaining rows.
Next steps
Normalize Data module
Use this module to transform a dataset through normalization.
Normalization is a technique often applied as part of data preparation for machine learning. The goal of
normalization is to change the values of numeric columns in the dataset to use a common scale, without distorting
differences in the ranges of values or losing information. Normalization is also required for some algorithms to
model the data correctly.
For example, assume your input dataset contains one column with values ranging from 0 to 1, and another column
with values ranging from 10,000 to 100,000. The great difference in the scale of the numbers could cause
problems when you attempt to combine the values as features during modeling.
Normalization avoids these problems by creating new values that maintain the general distribution and ratios in
the source data, while keeping values within a scale applied across all numeric columns used in the model.
This module offers several options for transforming numeric data:
You can change all values to a 0-1 scale, or transform the values by representing them as percentile ranks
rather than absolute values.
You can apply normalization to a single column, or to multiple columns in the same dataset.
If you need to repeat the experiment, or apply the same normalization steps to other data, you can save the
steps as a normalization transform, and apply it to other datasets that have the same schema.
WARNING
Some algorithms require that data be normalized before training a model. Other algorithms perform their own data scaling
or normalization. Therefore, when you choose a machine learning algorithm to use in building a predictive model, be sure to
review the data requirements of the algorithm before applying normalization to the training data.
Configure Normalize Data

You can apply only one normalization method at a time using this module. Therefore, the same normalization
method is applied to all columns that you select. To use different normalization methods, use a second instance of
Normalize Data.
1. Add the Normalize Data module to your experiment. You can find the module In Azure Machine Learning,
under Data Transformation, in the Scale and Reduce category.
2. Connect a dataset that contains at least one column of all numbers.
3. Use the Column Selector to choose the numeric columns to normalize. If you don't choose individual
columns, by default all numeric type columns in the input are included, and the same normalization process
is applied to all selected columns.
This can lead to strange results if you include numeric columns that shouldn't be normalized! Always check
the columns carefully.
If no numeric columns are detected, check the column metadata to verify that the data type of the column is
a supported numeric type.
TIP
To ensure that columns of a specific type are provided as input, try using the Select Columns in Dataset module
before Normalize Data.
4. Use 0 for constant columns when checked: Select this option when any numeric column contains a
single unchanging value. This ensures that such columns are not used in normalization operations.
5. From the Transformation method dropdown list, choose a single mathematical function to apply to all
selected columns.
Zscore: Converts all values to a z-score.
The values in the column are transformed using the following formula:
Mean and standard deviation are computed for each column separately. Population standard
deviation is used.
MinMax: The min-max normalizer linearly rescales every feature to the [0,1] interval.
Rescaling to the [0,1] interval is done by shifting the values of each feature so that the minimal value
is 0, and then dividing by the new maximal value (which is the difference between the original
maximal and minimal values).
Logistic: The values in the column are transformed using the following formula:
LogNormal: This option converts all values to a lognormal scale.

Here μ and σ are the parameters of the distribution, computed empirically from the data as
maximum likelihood estimates, for each column separately.
TanH: All values are converted to a hyperbolic tangent.
6. Run the experiment, or double-click the Normalize Data module and select Run Selected.
Results
The Normalize Data module generates two outputs:
To view the transformed values, right-click the module, select Transformed dataset, and click Visualize.
By default, values are transformed in place. If you want to compare the transformed values to the original
values, use the Add Columns module to recombine the datasets and view the columns side by side.
To save the transformation so that you can apply the same normalization method to another similar dataset,
right-click the module, select Transformation function, and click Save as Transform.
You can then load the saved transformations from the Transforms group of the left navigation pane and
apply it to a dataset with the same schema by using ./Apply Transformation.
Next steps
Partition and Sample module
Use this module to perform sampling on a dataset or to create partitions from your dataset.
Sampling is an important tool in machine learning because it lets you reduce the size of a dataset while
maintaining the same ratio of values. This module supports several related tasks that are important in machine
learning:
Dividing your data into multiple subsections of the same size.
You might use the partitions for cross-validation, or to assign cases to random groups.
Separating data into groups and then working with data from a specific group.
After randomly assigning cases to different groups, you might need to modify the features that are
associated with only one group.
Sampling.
You can extract a percentage of the data, apply random sampling, or choose a column to use for balancing
the dataset and perform stratified sampling on its values.
Creating a smaller dataset for testing.
If you have a lot of data, you might want to use only the first n rows while setting up the experiment, and
then switch to using the full dataset when you build your model. You can also use sampling to create s
smaller dataset for use in development.
Configure Partition and Sample

This module supports multiple methods for dividing your data into partitions or for sampling. Choose the method
first, and then set additional options required by the method.
Head
Sampling
Assign to folds
Pick fold
Get TOP N rows from a dataset
Use this mode to get only the first n rows. This option is useful if you want to test an experiment on a small
number of rows, and don't need the data to be balanced or sampled in any way.
1. Add the Partition and Sample module to your experiment in the interface, and connect the dataset.
2. Partition or sample mode: Set this option to Head.
3. Number of rows to select: Type the number of rows to return.
The number of rows you specify must be a non-negative integer. If the number of selected rows is larger
than the number of rows in the dataset, the entire dataset is returned.
The module outputs a single dataset containing only the specified number of rows. The rows are always read from
the top of the dataset.
Create a sample of data
This option supports simple random sampling or stratified random sampling. This is useful if you want to create a
smaller representative sample dataset for testing.
1. Add the Partition and Sample module to your experiment, and connect the dataset.
2. Partition or sample mode: Set this to Sampling.
3. Rate of sampling: Type a value between 0 and 1. this value specifies the percentage of rows from the
source dataset that should be included in the output dataset.
For example, if you want only half of the original dataset, type 0.5 to indicate that the sampling rate
should be 50%.
The rows of the input dataset are shuffled and selectively put into the output dataset, according to the
specified ratio.
4. Random seed for sampling: Optionally, type an integer to use as a seed value.
This option is important if you want the rows to be divided the same way every time. The default value is 0,
meaning that a starting seed is generated based on the system clock. This can lead to slightly different
results each time you run the experiment.
5. Stratified split for sampling: Select this option if it is important that the rows in the dataset should be
divided evenly by some key column before sampling.
For Stratification key column for sampling, select a single strata column to use when dividing the
dataset. The rows in the dataset are then divided as follows:
a. All input rows are grouped (stratified) by the values in the specified strata column.
b. Rows are shuffled within each group.
c. Each group is selectively added to the output dataset to meet the specified ratio.
With this option, the module outputs a single dataset that contains a representative sampling of the data.
The remaining, unsampled portion of the dataset is not output.
Split data into partitions

Use this option when you want to divide the dataset into subsets of the data. This option is also useful when you
want to create a custom number of folds for cross-validation, or to split rows into several groups.
1. Add the Partition and Sample module to your experiment, and connect the dataset.
2. For Partition or sample mode, select Assign to Folds.
3. Use replacement in the partitioning: Select this option if you want the sampled row to be put back into
the pool of rows for potential reuse. As a result, the same row might be assigned to several folds.
If you do not use replacement (the default option), the sampled row is not put back into the pool of rows for
potential reuse. As a result, each row can be assigned to only one fold.
4. Randomized split: Select this option if you want rows to be randomly assigned to folds.
If you do not select this option, rows are assigned to folds using the round-robin method.
5. Random seed: Optionally, type an integer to use as the seed value. This option is important if you want the
rows to be divided the same way every time. Otherwise, the default value of 0 means that a random
starting seed will be used.
6. Specify the partitioner method: Indicate how you want data to be apportioned to each partition, using
these options:
Partition evenly: Use this option to place an equal number of rows in each partition. To specify the
number of output partitions, type a whole number in the Specify number of folds to split evenly
into text box.
Partition with customized proportions: Use this option to specify the size of each partition as a
comma-separated list.
For example, if you want to create three partitions, with the first partition containing 50% of the data,
and the remaining two partitions each containing 25% of the data, click the List of proportions
separated by comma text box, and type these numbers: .5, .25, .25
The sum of all partition sizes must add up to exactly 1.
If you enter numbers that add up to less than 1, an extra partition is created to hold the
remaining rows. For example, if you type the values .2 and .3, a third partition is created that
holds the remaining 50 percent of all rows.
If you enter numbers that add up to more than 1, an error is raised when you run the
experiment.
7. Stratified split: Select this option if you want the rows to be stratified when split, and then choose the
strata column.
With this option, the module outputs multiple datasets, partitioned using the rules you specified.
Use data from a predefined partition
This option is used when you have divided a dataset into multiple partitions and now want to load each partition
in turn for further analysis or processing.
1. Add the Partition and Sample module to the experiment.
2. Connect it to the output of a previous instance of Partition and Sample. That instance must have used the
Assign to Folds option to generate some number of partitions.
3. Partition or sample mode: Select Pick Fold.
4. Specify which fold to be sampled from: Select a partition to use by typing its index. Partition indices are
1-based. For example, if you divided the dataset into three parts, the partitions would have the indices 1, 2,
and 3.
If you type an invalid index value, a design-time error is raised: "Error 0018: Dataset contains invalid data."
In addition to grouping the dataset by folds, you can separate the dataset into two groups: a target fold, and
everything else. To do this, type the index of a single fold, and then select the option, Pick complement of
the selected fold, to get everything but the data in the specified fold.
5. If you are working with multiple partitions, you must add additional instances of the Partition and
Sample module to handle each partition.
For example, let's say previously partitioned patients into four folds using age. To work with each individual
fold, you need four copies of the Partition and Sample module, and in each, you select a different fold, as
shown below. It's not correct to use the Assign to Folds output directly.

With this option, the module outputs a single dataset containing only the rows assigned to that fold.
NOTE
You cannot view the fold designations directly; they are present only in the metadata.
Next steps
Score Model module
Use this module to generate predictions using a trained classification or regression model.
How to use
1. Add the Score Model module to your experiment.
2. Attach a trained model and a dataset containing new input data.
The data should be in a format compatible with the type of trained model you are using. The schema of
the input dataset should also generally match the schema of the data used to train the model.
Results
After you have generated a set of scores using Score Model:
To generate a set of metrics used for evaluating the model’s accuracy (performance). you can connect the
scored dataset to Evaluate Model,
Right-click the module and select Visualize to see a sample of the results.
Save the results to a dataset.
The score, or predicted value, can be in many different formats, depending on the model and your input data:
For classification models, Score Model outputs a predicted value for the class, as well as the probability of the
predicted value.
For regression models, Score Model generates just the predicted numeric value.
For image classification models, the score might be the class of object in the image, or a Boolean indicating
whether a particular feature was found.
Publish scores as a web service

A common use of scoring is to return the output as part of a predictive web service. For more information, see
this tutorial on how to create a web service based on an experiment in Azure Machine Learning:
Next steps
Module: Assign Data to Clusters
This article describes how to use the Assign Data to Clusters module in the Azure Machine Learning visual
interface. The module generates predictions through a clustering model that was trained with the K -means
clustering algorithm.
The Assign Data to Clusters module returns a dataset that contains the probable assignments for each new data
point.
How to use Assign Data to Clusters

1. In the Azure Machine Learning visual interface, locate a previously trained clustering model. You can create
and train a clustering model by using either of the following methods:
Configure the K-means clustering algorithm by using the K-Means Clustering module, and train the
model by using a dataset and the Train Clustering Model module (this article).
You can also add an existing trained clustering model from the Saved Models group in your
workspace.
2. Attach the trained model to the left input port of Assign Data to Clusters.
3. Attach a new dataset as input.
In this dataset, labels are optional. Generally, clustering is an unsupervised learning method. You are not
expected to know the categories in advance. However, the input columns must be the same as the columns
that were used in training the clustering model, or an error occurs.
TIP
To reduce the number of columns that are written to the interface from the cluster predictions, use Select columns in
the dataset, and select a subset of the columns.
4. Leave the Check for Append or Uncheck for Result Only check box selected if you want the results to
contain the full input dataset, including a column that displays the results (cluster assignments).
If you clear this check box, only the results are returned. This option might be useful when you create
predictions as part of a web service.
Results
To view the values in the dataset, right-click the module, select Result datasets, and then select Visualize.
Train Model module
Use this module to train a classification or regression model. Training takes place after you have defined a
model and set its parameters, and requires tagged data. You can also use Train Model to retrain an existing
model with new data.
How the training process works

In Azure Machine Learning, creating and using a machine learning model is typically a three-step process.
1. You configure a model, by choosing a particular type of algorithm, and defining its parameters or
hyperparameters. Choose any of the following model types:
Classification models, based on neural networks, decision trees, and decision forests, and other
algorithms.
Regression models, which can include standard linear regression, or which use other algorithms,
including neural networks and Bayesian regression.
2. Provide a dataset that is labeled, and has data compatible with the algorithm. Connect both the data
and the model to Train Model.
What training produces is a specific binary format, the iLearner, that encapsulates the statistical
patterns learned from the data. You cannot directly modify or read this format; however, other modules
can use this trained model.
You can also view properties of the model. For more information, see the Results section.
3. After training is completed, use the trained model with one of the scoring modules, to make
predictions on new data.
How to use Train Model

1. In Azure Machine Learning, configure a classification model or regression model.
2. Add the Train Model module to the experiment. You can find this module under the Machine
Learning category. Expand Train, and then drag the Train Model module into your experiment.
3. On the left input, attach the untrained mode. Attach the training dataset to the right-hand input of
Train Model.
The training dataset must contain a label column. Any rows without labels are ignored.
4. For Label column, click Launch column selector, and choose a single column that contains
outcomes the model can use for training.
For classification problems, the label column must contain either categorical values or
discrete values. Some examples might be a yes/no rating, a disease classification code or name,
or an income group. If you pick a noncategorical column, the module will return an error during
training.
For regression problems, the label column must contain numeric data that represents the
response variable. Ideally the numeric data represents a continuous scale.
Examples might be a credit risk score, the projected time to failure for a hard drive, or the forecasted
number of calls to a call center on a given day or time. If you do not choose a numeric column, you
might get an error.
If you do not specify which label column to use, Azure Machine Learning will try to infer which is
the appropriate label column, by using the metadata of the dataset. If it picks the wrong column,
use the column selector to correct it.
TIP
If you have trouble using the Column Selector, see the article Select Columns in Dataset for tips. It describes
some common scenarios and tips for using the WITH RULES and BY NAME options.
5. Run the experiment. If you have a lot of data, this can take a while.
Results
After the model is trained:
To view the model parameters and feature weights, right-click the output and select Visualize.
To use the model in other experiments, right-click the model and select Save Model. Type a name for
the model.
This saves the model as a snapshot that is not updated by repeated runs of the experiment.
To use the model in predicting new values, connect it to the Score Model module, together with new
input data.
Next steps
Use this module to train a clustering model.
The module takes an untrained clustering model that you have already configured using the K-Means Clustering
module, and trains the model using a labeled or unlabeled data set. The module creates both a trained model that
you can use for prediction, and a set of cluster assignments for each case in the training data.
NOTE
A clustering model cannnot be trained using the Train Model module, which is the generic module for training machine
learning models. That is because Train Model works only with supervised learning algorithms. K-means and other clustering
algorithms allow unsupervised learning, meaning that the algorithm can learn from unlabeled data.
How to use Train Clustering Model

1. Add the Train Clustering Model module to your experiment in Studio. You can find the module under
Machine Learning Modules, in the Train category.
2. Add the K-Means Clustering module, or another custom module that creates a compatible clustering
model, and set the parameters of the clustering model.
3. Attach a training dataset to the right-hand input of Train Clustering Model.
4. In Column Set, select the columns from the dataset to use in building clusters. Be sure to select columns
that make good features: for example, avoid using IDs or other columns that have unique values, or
columns that have all the same values.
If a label is available, you can either use it as a feature, or leave it out.
5. Select the option, Check for Append or Uncheck for Result Only, if you want to output the training data
together with the new cluster label.
If you deselect this option, only the cluster assignments are output.
6. Run the experiment, or click the Train Clustering Model module and select Run Selected.
Results
After training has completed:
To view the values in the dataset, right-click the module, select Result datasets, and click Visualize.
To save the trained model for later re-use, right-click the module, select Trained model, and click Save As
Trained Model.
To generate scores from the model, use Assign Data to Clusters.
Next steps
Evaluate Model module
Use this module to measure the accuracy of a trained model. You provide a dataset containing scores generated
from a model, and the Evaluate Model module computes a set of industry-standard evaluation metrics.
The metrics returned by Evaluate Model depend on the type of model that you are evaluating:
Classification Models
Regression Models
TIP
If you are new to model evaluation, we recommend the video series by Dr. Stephen Elston, as part of the machine learning
course from EdX.
There are three ways to use the Evaluate Model module:

Generate scores over your training data, and evaluate the model based on these scores
Generate scores on the model, but compare those scores to scores on a reserved testing set
Compare scores for two different but related models, using the same set of data
Use the training data

To evaluate a model, you must connect a dataset that contains a set of input columns and scores. If no other data is
available, you can use your original dataset.
1. Connect the Scored dataset output of the Score Model to the input of Evaluate Model.
2. Click Evaluate Model module, and run the experiment to generate the evaluation scores.
Use testing data

A common scenario in machine learning is to separate your original data set into training and testing datasets,
using the Split module, or the Partition and Sample module.
2. Connect the output of the Split Data module that contains the testing data to the right-hand input of Evaluate
Model.
3. Click Evaluate Model module, and select Run selected to generate the evaluation scores.
Compare scores from two models

You can also connect a second set of scores to Evaluate Model. The scores might be a shared evaluation set that
has known results, or a set of results from a different model for the same data.
This feature is useful because you can easily compare results from two different models on the same data. Or, you
might compare scores from two different runs over the same data with different parameters.
2. Connect the output of the Score Model module for the second model to the right-hand input of Evaluate
Model.
3. Right-click Evaluate Model, and select Run selected to generate the evaluation scores.
Results
After you run Evaluate Model, right-click the module and select Evaluation results to see the results. You can:
Save the results as a dataset, for easier analysis with other tools
Generate a visualization in the interface
If you connect datasets to both inputs of Evaluate Model, the results will contain metrics for both set of data, or
both models. The model or data attached to the left port is presented first in the report, followed by the metrics for
the dataset, or model attached on the right port.
For example, the following image represents a comparison of results from two clustering models that were built
on the same data, but with different parameters.
Because this is a clustering model, the evaluation results are different than if you compared scores from two
regression models, or compared two classification models. However, the overall presentation is the same.
Metrics
This section describes the metrics returned for the specific types of models supported for use with Evaluate
Model:
classification models
regression models
Metrics for classification models
The following metrics are reported when evaluating classification models. If you compare models, they are ranked
by the metric you select for evaluation.
Accuracy measures the goodness of a classification model as the proportion of true results to total cases.
Precision is the proportion of true results over all positive results.
Recall is the fraction of all correct results returned by the model.
F-score is computed as the weighted average of precision and recall between 0 and 1, where the ideal F -
score value is 1.
AUC measures the area under the curve plotted with true positives on the y axis and false positives on the x
axis. This metric is useful because it provides a single number that lets you compare models of different
types.
Average log loss is a single score used to express the penalty for wrong results. It is calculated as the
difference between two probability distributions – the true one, and the one in the model.
Training log loss is a single score that represents the advantage of the classifier over a random prediction.
The log loss measures the uncertainty of your model by comparing the probabilities it outputs to the
known values (ground truth) in the labels. You want to minimize log loss for the model as a whole.
Metrics for regression models

The metrics returned for regression models are generally designed to estimate the amount of error. A model is
considered to fit the data well if the difference between observed and predicted values is small. However, looking
at the pattern of the residuals (the difference between any one predicted point and its corresponding actual value)
can tell you a lot about potential bias in the model.
The following metrics are reported for evaluating regression models. When you compare models, they are ranked
by the metric you select for evaluation.
Mean absolute error (MAE ) measures how close the predictions are to the actual outcomes; thus, a lower
score is better.
Root mean squared error (RMSE ) creates a single value that summarizes the error in the model. By
squaring the difference, the metric disregards the difference between over-prediction and under-prediction.
Relative absolute error (RAE ) is the relative absolute difference between expected and actual values;
relative because the mean difference is divided by the arithmetic mean.
Relative squared error (RSE ) similarly normalizes the total squared error of the predicted values by
dividing by the total squared error of the actual values.
Mean Zero One Error (MZOE ) indicates whether the prediction was correct or not. In other words:
ZeroOneLoss(x,y) = 1 when x!=y ; otherwise 0 .
Coefficient of determination, often referred to as R 2, represents the predictive power of the model as a
value between 0 and 1. Zero means the model is random (explains nothing); 1 means there is a perfect fit.
However, caution should be used in interpreting R 2 values, as low values can be entirely normal and high
values can be suspect.
Next steps
Apply Transformation module
Use this module to modify an input dataset based on a previously computed transformation.
For example, if you used z-scores to normalize your training data by using the Normalize Data module, you
would want to use the z-score value that was computed for training during the scoring phase as well. In Azure
Machine Learning, you can save the normalization method as a transform, and then using Apply
Transformation to apply the z-score to the input data before scoring.
Azure Machine Learning provides support for creating and then applying many different kinds of custom
transformations. For example, you might want to save and then reuse transformations to:
Remove or replace missing values, using Clean Missing Data
Normalize data, using Normalize Data
How to use Apply Transformation

1. Add the Apply Transformation module to your experiment. You can find this module under Machine
Learning, in the Score category.
2. Locate an existing transformation to use as an input. Previously saved transformations can be found in the
Transforms group in the left navigation pane.
3. Connect the dataset that you want to transform. The dataset should have exactly the same schema
(number of columns, column names, data types) as the dataset for which the transformation was first
designed.
4. No other parameters need to be set since all customization is done when defining the transformation.
5. To apply a transformation to the new dataset, run the experiment.
Next steps
Linear Regression module
Use this module to create a linear regression model for use in an experiment. Linear regression attempts to
establish a linear relationship between one or more independent variables and a numeric outcome, or dependent
variable.
You use this module to define a linear regression method, and then train a model using a labeled dataset. The
trained model can then be used to make predictions.
About linear regression

Linear regression is a common statistical method, which has been adopted in machine learning and enhanced with
many new methods for fitting the line and measuring error. In the most basic sense, regression refers to prediction
of a numeric target. Linear regression is still a good choice when you want a simple model for a basic predictive
task. Linear regression also tends to work well on high-dimensional, sparse data sets lacking complexity.
Azure Machine Learning supports a variety of regression models, in addition to linear regression. However, the
term "regression" can be interpreted loosely, and some types of regression provided in other tools are not
supported.
The classic regression problem involves a single independent variable and a dependent variable. This is
called simple regression. This module supports simple regression.
Multiple linear regression involves two or more independent variables that contribute to a single dependent
variable. Problems in which multiple inputs are used to predict a single numeric outcome are also called
multivariate linear regression.
The Linear Regression module can solve these problems, as can most of the other regression modules.
Multi-label regression is the task of predicting multiple dependent variables within a single model. For
example, in multi-label logistic regression, a sample can be assigned to multiple different labels. (This is
different from the task of predicting multiple levels within a single class variable.)
This type of regression is not supported in Azure Machine Learning. To predict multiple variables, create a
separate learner for each output that you wish to predict.
For years statisticians have been developing increasingly advanced methods for regression. This is true even for
linear regression. This module supports two methods to measure error and fit the regression line: ordinary least
squares method, and gradient descent.
Gradient descent is a method that minimizes the amount of error at each step of the model training
process. There are many variations on gradient descent and its optimization for various learning problems
has been extensively studied. If you choose this option for Solution method, you can set a variety of
parameters to control the step size, learning rate, and so forth. This option also supports use of an
integrated parameter sweep.
Ordinary least squares is one of the most commonly used techniques in linear regression. For example,
least squares is the method that is used in the Analysis Toolpak for Microsoft Excel.
Ordinary least squares refers to the loss function, which computes error as the sum of the square of
distance from the actual value to the predicted line, and fits the model by minimizing the squared error. This
method assumes a strong linear relationship between the inputs and the dependent variable.
Configure Linear Regression

This module supports two methods for fitting a regression model, with different options:
Create a regression model using online gradient descent
Gradient descent is a better loss function for models that are more complex, or that have too little training
data given the number of variables.
Fit a regression model using ordinary least squares
For small datasets, it is best to select ordinary least squares. This should give similar results to Excel.
Create a regression model using ordinary least squares

1. Add the Linear Regression Model module to your experiment in the interface.
You can find this module in the Machine Learning category. Expand Initialize Model, expand
Regression, and then drag the Linear Regression Model module to your experiment.
2. In the Properties pane, in the Solution method dropdown list, select Ordinary Least Squares. This
option specifies the computation method used to find the regression line.
3. In L2 regularization weight, type the value to use as the weight for L2 regularization. We recommend that
you use a non-zero value to avoid overfitting.
To learn more about how regularization affects model fitting, see this article: L1 and L2 Regularization for
Machine Learning
4. Select the option, Include intercept term, if you want to view the term for the intercept.
Deselect this option if you don't need to review the regression formula.
5. For Random number seed, you can optionally type a value to seed the random number generator used by
the model.
Using a seed value is useful if you want to maintain the same results across different runs of the same
experiment. Otherwise, the default is to use a value from the system clock.
6. Add the Train Model module to your experiment, and connect a labeled dataset.
Results for ordinary least squares model

After training is complete:
To view the model's parameters, right-click the trainer output and select Visualize.
To make predictions, connect the trained model to the Score Model module, along with a dataset of new
values.
Create a regression model using online gradient descent

1. Add the Linear Regression Model module to your experiment in the interface.
You can find this module in the Machine Learning category. Expand Initialize Model, expand
Regression, and drag the Linear Regression Model module to your experiment
2. In the Properties pane, in the Solution method dropdown list, choose Online Gradient Descent as the
computation method used to find the regression line.
3. For Create trainer mode, indicate whether you want to train the model with a predefined set of
parameters, or if you want to optimize the model by using a parameter sweep.
Single Parameter: If you know how you want to configure the linear regression network, you can
provide a specific set of values as arguments.
4. For Learning rate, specify the initial learning rate for the stochastic gradient descent optimizer.
5. For Number of training epochs, type a value that indicates how many times the algorithm should iterate
through examples. For datasets with a small number of examples, this number should be large to reach
convergence.
6. Normalize features: If you have already normalized the numeric data used to train the model, you can
deselect this option. By default, the module normalizes all numeric inputs to a range between 0 and 1.
NOTE
Remember to apply the same normalization method to new data used for scoring.
7. In L2 regularization weight, type the value to use as the weight for L2 regularization. We recommend that
you use a non-zero value to avoid overfitting.
To learn more about how regularization affects model fitting, see this article: L1 and L2 Regularization for
Machine Learning
8. Select the option, Decrease learning rate, if you want the learning rate to decrease as iterations progress.
9. For Random number seed, you can optionally type a value to seed the random number generator used by
the model. Using a seed value is useful if you want to maintain the same results across different runs of the
same experiment.
10. Add a labeled dataset and one of the training modules.
If you are not using a parameter sweep, use the Train Model module.
Results for online gradient descent

To make predictions, connect the trained model to the Score Model module, together with new input data.
Next steps
Neural Network Regression module
Creates a regression model using a neural network algorithm

Category: Machine Learning / Initialize Model / Regression
Module overview
Use this module to create a regression model using a customizable neural network algorithm.
Although neural networks are widely known for use in deep learning and modeling complex problems such as
image recognition, they are easily adapted to regression problems. Any class of statistical models can be termed a
neural network if they use adaptive weights and can approximate non-linear functions of their inputs. Thus neural
network regression is suited to problems where a more traditional regression model cannot fit a solution.
Neural network regression is a supervised learning method, and therefore requires a tagged dataset, which
includes a label column. Because a regression model predicts a numerical value, the label column must be a
numerical data type.
You can train the model by providing the model and the tagged dataset as an input to Train Model. The trained
model can then be used to predict values for the new input examples.
Configure Neural Network Regression

Neural networks can be extensively customized. This section describes how to create a model using two methods:
Create a neural network model using the default architecture
If you accept the default neural network architecture, use the Properties pane to set parameters that control
the behavior of the neural network, such as the number of nodes in the hidden layer, learning rate, and
normalization.
Start here if you are new to neural networks. The module supports many customizations, as well as model
tuning, without deep knowledge of neural networks.
Define a custom architecture for a neural network
Use this option if you want to add extra hidden layers, or fully customize the network architecture, its
connections, and activation functions.
This option is best if you are already somewhat familiar with neural networks. You use the Net# language to
define the network architecture.
Create a neural network model using the default architecture

1. Add the Neural Network Regression module to your experiment in the interface. You can find this module
under Machine Learning, Initialize, in the Regression category.
2. Indicate how you want the model to be trained, by setting the Create trainer mode option.
Single Parameter: Choose this option if you already know how you want to configure the model.
3. In Hidden layer specification, select Fully connected case. This option creates a model using the default
neural network architecture, which for a neural network regression model, has these attributes:
The network has exactly one hidden layer.
The output layer is fully connected to the hidden layer and the hidden layer is fully connected to the input
layer.
The number of nodes in the hidden layer can be set by the user (default value is 100).
Because the number of nodes in the input layer is determined by the number of features in the training data,
in a regression model there can be only one node in the output layer.
4. For Number of hidden nodes, type the number of hidden nodes. The default is one hidden layer with 100
nodes. (This option is not available if you define a custom architecture using Net#.)
5. For Learning rate, type a value that defines the step taken at each iteration, before correction. A larger
value for learning rate can cause the model to converge faster, but it can overshoot local minima.
6. For Number of learning iterations, specify the maximum number of times the algorithm processes the
training cases.
7. For **The initial learning weights diameter, type a value that determines the node weights at the start of the
learning process.
8. For The momentum, type a value to apply during learning as a weight on nodes from previous iterations.
9. Select the option, Shuffle examples, to change the order of cases between iterations. If you deselect this
option, cases are processed in exactly the same order each time you run the experiment.
10. For Random number seed, you can optionally type a value to use as the seed. Specifying a seed value is
useful when you want to ensure repeatability across runs of the same experiment.
11. Connect a training dataset and one of the training modules:
If you set Create trainer mode to Single Parameter, use Train Model.
Results
To see a summary of the model's parameters, together with the feature weights learned from training, and
other parameters of the neural network, right-click the output of Train Model, and select Visualize.
To save a snapshot of the trained model, right-click the Trained model output and select Save As Trained
Model. This model is not updated on successive runs of the same experiment.
Next steps
Decision Forest Regression module
Use this module to create a regression model based on an ensemble of decision trees.
After you have configured the model, you must train the model using a labeled dataset and the Train Model
module. The trained model can then be used to make predictions.
How it works
Decision trees are non-parametric models that perform a sequence of simple tests for each instance, traversing a
binary tree data structure until a leaf node (decision) is reached.
Decision trees have these advantages:
They are efficient in both computation and memory usage during training and prediction.
They can represent non-linear decision boundaries.
They perform integrated feature selection and classification and are resilient in the presence of noisy
features.
This regression model consists of an ensemble of decision trees. Each tree in a regression decision forest outputs a
Gaussian distribution as a prediction. An aggregation is performed over the ensemble of trees to find a Gaussian
distribution closest to the combined distribution for all trees in the model.
For more information about the theoretical framework for this algorithm and its implementation, see this article:
Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and
Semi-Supervised Learning
How to configure Decision Forest Regression Model

1. Add the Decision Forest Regression module to the experiment. You can find the module in the interface
under Machine Learning, Initialize Model, and Regression.
2. Open the module properties, and for Resampling method, choose the method used to create the
individual trees. You can choose from Bagging or Replicate.
Bagging: Bagging is also called bootstrap aggregating. Each tree in a regression decision forest
outputs a Gaussian distribution by way of prediction. The aggregation is to find a Gaussian whose
first two moments match the moments of the mixture of Gaussians given by combining all Gaussians
returned by individual trees.
For more information, see the Wikipedia entry for Bootstrap aggregating.
Replicate: In replication, each tree is trained on exactly the same input data. The determination of
which split predicate is used for each tree node remains random and the trees will be diverse.
For more information about the training process with the Replicate option, see Decision Forests for
Computer Vision and Medical Image Analysis. Criminisi and J. Shotton. Springer 2013..
3. Specify how you want the model to be trained, by setting the Create trainer mode option.
Single Parameter
If you know how you want to configure the model, you can provide a specific set of values as
arguments. You might have learned these values by experimentation or received them as guidance.
4. For Number of decision trees, indicate the total number of decision trees to create in the ensemble. By
creating more decision trees, you can potentially get better coverage, but training time will increase.
TIP
This value also controls the number of trees displayed when visualizing the trained model. if you want to see or print
a single tree, you can set the value to 1; however, this means that only one tree will be produced (the tree with the
initial set of parameters) and no further iterations will be performed.
5. For Maximum depth of the decision trees, type a number to limit the maximum depth of any decision
tree. Increasing the depth of the tree might increase precision, at the risk of some overfitting and increased
training time.
6. For Number of random splits per node, type the number of splits to use when building each node of the
tree. A split means that features in each level of the tree (node) are randomly divided.
7. For Minimum number of samples per leaf node, indicate the minimum number of cases that are
required to create any terminal node (leaf) in a tree.
By increasing this value, you increase the threshold for creating new rules. For example, with the default
value of 1, even a single case can cause a new rule to be created. If you increase the value to 5, the training
data would have to contain at least five cases that meet the same conditions.
8. Connect a labeled dataset, select a single label column containing no more than two outcomes, and connect
to Train Model.
If you set Create trainer mode option to Single Parameter, train the model by using the Train Model
module.
Results
To see the tree that was created on each iteration, right-click the output of the training module, and select
Visualize.
To see the rules for each node, click each tree and drill down into the splits.
To save a snapshot of the trained model, right-click the output of the training module, and select Save As
Trained Model. This copy of the model is not updated on successive runs of the experiment.
Next steps
Boosted Decision Tree Regression module
Use this module to create an ensemble of regression trees using boosting. Boosting means that each tree is
dependent on prior trees. The algorithm learns by fitting the residual of the trees that preceded it. Thus, boosting in
a decision tree ensemble tends to improve accuracy with some small risk of less coverage.
This regression method is a supervised learning method, and therefore requires a labeled dataset. The label
column must contain numerical values.
NOTE
Use this module only with datasets that use numerical variables.
After you have defined the model, train it by using the Train Model.
TIP
Want to know more about the trees that were created? After the model has been trained, right-click the output of the Train
Model module and select Visualize to see the tree that was created on each iteration. You can drill down into the splits for
each tree and see the rules for each node.
More about boosted regression trees

Boosting is one of several classic methods for creating ensemble models, along with bagging, random forests, and
so forth. In Azure Machine Learning, boosted decision trees use an efficient implementation of the MART gradient
boosting algorithm. Gradient boosting is a machine learning technique for regression problems. It builds each
regression tree in a step-wise fashion, using a predefined loss function to measure the error in each step and
correct for it in the next. Thus the prediction model is actually an ensemble of weaker prediction models.
In regression problems, boosting builds a series of trees in a step-wise fashion, and then selects the optimal tree
using an arbitrary differentiable loss function.
For additional information, see these articles:
https://wikipedia.org/wiki/Gradient_boosting#Gradient_tree_boosting
This Wikipedia article on gradient boosting provides some background on boosted trees.
https://research.microsoft.com/apps/pubs/default.aspx?id=132652
Microsoft Research: From RankNet to LambdaRank to LambdaMART: An Overview. By J.C. Burges.
The gradient boosting method can also be used for classification problems by reducing them to regression with a
suitable loss function. For more information about the boosted trees implementation for classification tasks, see
Two-Class Boosted Decision Tree.
How to configure Boosted Decision Tree Regression

1. Add the Boosted Decision Tree module to your experiment. You can find this module under Machine
Learning, Initialize, under the Regression category.
Single Parameter: Select this option if you know how you want to configure the model, and provide a
specific set of values as arguments.
3. Maximum number of leaves per tree: Indicate the maximum number of terminal nodes (leaves) that can
be created in any tree.
By increasing this value, you potentially increase the size of the tree and get better precision, at the risk of
overfitting and longer training time.
4. Minimum number of samples per leaf node: Indicate the minimum number of cases required to create
any terminal node (leaf) in a tree.
data would have to contain at least 5 cases that meet the same conditions.
5. Learning rate: Type a number between 0 and 1 that defines the step size while learning. The learning rate
determines how fast or slow the learner converges on the optimal solution. If the step size is too big, you
might overshoot the optimal solution. If the step size is too small, training takes longer to converge on the
best solution.
6. Number of trees constructed: Indicate the total number of decision trees to create in the ensemble. By
creating more decision trees, you can potentially get better coverage, but training time increases.
This value also controls the number of trees displayed when visualizing the trained model. if you want to see
or print an ingle tree, you can set the value to 1; however, only one tree is produced (the tree with the initial
set of parameters) and no further iterations are performed.
7. Random number seed: Type an optional non-negative integer to use as the random seed value. Specifying
a seed ensures reproducibility across runs that have the same data and parameters.
By default, the random seed is set to 0, which means the initial seed value is obtained from the system clock.
8. Allow unknown categorical levels: Select this option to create a group for unknown values in the
training and validation sets. If you deselect this option, the model can accept only the values that are
contained in the training data. The model might be less precise for known values, but it can provide better
predictions for new (unknown) values.
9. Add a training dataset, and one of the training modules:
If you set Create trainer mode option to Single Parameter, use the Train Model module.
Results
To see the tree that was created on each iteration, right-click the output of the Train Model module and
select Visualize.
Click each tree to drill down into the splits and see the rules for each node.
To use the model for scoring, connect it to Score Model, to predict values for new input examples.
To save a snapshot of the trained model, right-click the Trained model output of the training module and
select Save As. The copy of the trained model that you save is not updated on successive runs of the
experiment.
Next steps
Two-Class Boosted Decision Tree module
Use this module to create a machine learning model that is based on the boosted decision trees algorithm.
A boosted decision tree is an ensemble learning method in which the second tree corrects for the errors of the first
tree, the third tree corrects for the errors of the first and second trees, and so forth. Predictions are based on the
entire ensemble of trees together that makes the prediction.
Generally, when properly configured, boosted decision trees are the easiest methods with which to get top
performance on a wide variety of machine learning tasks. However, they are also one of the more memory-
intensive learners, and the current implementation holds everything in memory. Therefore, a boosted decision tree
model might not be able to process the large datasets that some linear learners can handle.
How to configure
This module creates an untrained classification model. Because classification is a supervised learning method, to
train the model, you need a tagged dataset that includes a label column with a value for all rows.
You can train this type of model using Train Model.
1. In Azure Machine Learning, add the Boosted Decision Tree module to your experiment.
Single Parameter: If you know how you want to configure the model, you can provide a specific set of
values as arguments.
3. For Maximum number of leaves per tree, indicate the maximum number of terminal nodes (leaves) that
can be created in any tree.
By increasing this value, you potentially increase the size of the tree and get better precision, at the risk of
overfitting and longer training time.
4. For Minimum number of samples per leaf node, indicate the number of cases required to create any
terminal node (leaf) in a tree.
5. For Learning rate, type a number between 0 and 1 that defines the step size while learning.
The learning rate determines how fast or slow the learner converges on the optimal solution. If the step size
is too large, you might overshoot the optimal solution. If the step size is too small, training takes longer to
converge on the best solution.
6. For Number of trees constructed, indicate the total number of decision trees to create in the ensemble.
By creating more decision trees, you can potentially get better coverage, but training time will increase.
This value also controls the number of trees displayed when visualizing the trained model. if you want to
see or print a single tree, set the value to 1. However, when you do so, only one tree is produced (the tree
with the initial set of parameters) and no further iterations are performed.
7. For Random number seed, optionally type a non-negative integer to use as the random seed value.
Specifying a seed ensures reproducibility across runs that have the same data and parameters.
The random seed is set by default to 0, which means the initial seed value is obtained from the system
clock. Successive runs using a random seed can have different results.
8. Train the model.
If you set Create trainer mode to Single Parameter, connect a tagged dataset and the Train Model
module.
Results
After model training is complete, right-click the output of Train Model to view the results:
To see the tree that was created on each iteration, select Visualize.
To drill down into the splits and see the rules for each node, click each tree.
Next steps
Two-Class Logistic Regression module
Use this module to create a logistic regression model that can be used to predict two (and only two) outcomes.
Logistic regression is a well-known statistical technique that is used for modeling many kinds of problems. This
algorithm is a supervised learning method; therefore, you must provide a dataset that already contains the
outcomes to train the model.
About logistic regression
Logistic regression is a well-known method in statistics that is used to predict the probability of an outcome, and is
especially popular for classification tasks. The algorithm predicts the probability of occurrence of an event by fitting
data to a logistic function.
In this module, the classification algorithm is optimized for dichotomous or binary variables. if you need to classify
multiple outcomes, use the Multiclass Logistic Regression module.
How to configure
To train this model, you must provide a dataset that contains a label or class column. Because this module is
intended for two-class problems, the label or class column must contain exactly two values.
For example, the label column might be [Voted] with possible values of "Yes" or "No". Or, it might be [Credit Risk],
with possible values of "High" or "Low".
1. Add the Two-Class Logistic Regression module to your experiment.
3. For Optimization tolerance, specify a threshold value to use when optimizing the model. If the
improvement between iterations falls below the specified threshold, the algorithm is considered to have
converged on a solution, and training stops.
4. For L1 regularization weight and L2 regularization weight, type a value to use for the regularization
parameters L1 and L2. A non-zero value is recommended for both.
Regularization is a method for preventing overfitting by penalizing models with extreme coefficient values.
Regularization works by adding the penalty that is associated with coefficient values to the error of the
hypothesis. Thus, an accurate model with extreme coefficient values would be penalized more, but a less
accurate model with more conservative values would be penalized less.
L1 and L2 regularization have different effects and uses.
L1 can be applied to sparse models, which is useful when working with high-dimensional data.
In contrast, L2 regularization is preferable for data that is not sparse.
This algorithm supports a linear combination of L1 and L2 regularization values: that is, if x = L1 and
y = L2 , then ax + by = c defines the linear span of the regularization terms.
NOTE
Want to learn more about L1 and L2 regularization? The following article provides a discussion of how L1 and L2
regularization are different and how they affect model fitting, with code samples for logistic regression and neural
network models: L1 and L2 Regularization for Machine Learning
Different linear combinations of L1 and L2 terms have been devised for logistic regression models: for example, elastic
net regularization. We suggest that you reference these combinations to define a linear combination that is effective
in your model.
5. For Memory size for L -BFGS, specify the amount of memory to use for L -BFGS optimization.
L -BFGS stands for "limited memory Broyden-Fletcher-Goldfarb-Shanno". It is an optimization algorithm
that is popular for parameter estimation. This parameter indicates the number of past positions and
gradients to store for the computation of the next step.
This optimization parameter limits the amount of memory that is used to compute the next step and
direction. When you specify less memory, training is faster but less accurate.
6. For Random number seed, type an integer value. Defining a seed value is important if you want the
results to be reproducible over multiple runs of the same experiment.
7. Add a tagged dataset to the experiment, and connect one of the training modules.
If you set Create trainer mode to Single Parameter, use the Train Model module.
Results
To see a summary of the model's parameters, together with the feature weights learned from training, right-
click the output of Train Model and select Visualize.
To make predictions on new data, use the trained model and new data as input to the Score Model module.
Next steps
Multiclass Logistic Regression module
Use this module to create a logistic regression model that can be used to predict multiple values.
Classification using logistic regression is a supervised learning method, and therefore requires a labeled dataset.
You train the model by providing the model and the labeled dataset as an input to a module such as Train Model.
The trained model can then be used to predict values for new input examples.
Azure Machine Learning also provides a Two-Class Logistic Regression module, which is suited for classification
of binary or dichotomous variables.
About multiclass logistic regression

Logistic regression is a well-known method in statistics that is used to predict the probability of an outcome, and is
popular for classification tasks. The algorithm predicts the probability of occurrence of an event by fitting data to a
logistic function.
In multiclass logistic regression, the classifier can be used to predict multiple outcomes.
Configure a multiclass logistic regression

1. Add the Multiclass Logistic Regression module to the experiment.
Single Parameter: Use this option if you know how you want to configure the model, and provide a
specific set of values as arguments.
Parameter Range: Use this option if you are not sure of the best parameters, and want to use a
parameter sweep.
3. Optimization tolerance, specify the threshold value for optimizer convergence. If the improvement
between iterations is less than the threshold, the algorithm stops and returns the current model.
4. L1 regularization weight, L2 regularization weight: Type a value to use for the regularization
parameters L1 and L2. A non-zero value is recommended for both.
Regularization is a method for preventing overfitting by penalizing models with extreme coefficient values.
Regularization works by adding the penalty that is associated with coefficient values to the error of the
hypothesis. An accurate model with extreme coefficient values would be penalized more, but a less accurate
model with more conservative values would be penalized less.
L1 and L2 regularization have different effects and uses. L1 can be applied to sparse models, which is useful
when working with high-dimensional data. In contrast, L2 regularization is preferable for data that is not
sparse. This algorithm supports a linear combination of L1 and L2 regularization values: that is, if x = L1
and y = L2 , ax + by = c defines the linear span of the regularization terms.
Different linear combinations of L1 and L2 terms have been devised for logistic regression models, such as
elastic net regularization.
5. Random number seed: Type an integer value to use as the seed for the algorithm if you want the results
to be repeatable over runs. Otherwise, a system clock value is used as the seed, which can produce slightly
different results in runs of the same experiment.
6. Connect a labeled dataset, and one of the train modules:
Results
After training is complete, you can see a summary of the model's parameters, together with the feature weights
learned from training, right-click the output of the Train Model module, and select Visualize.
Next steps
Multiclass Neural Network module
Use this module to create a neural network model that can be used to predict a target that has multiple values.
For example, neural networks of this kind might be used in complex computer vision tasks, such as digit or letter
recognition, document classification, and pattern recognition.
Classification using neural networks is a supervised learning method, and therefore requires a tagged dataset that
includes a label column.
You can train the model by providing the model and the tagged dataset as an input to Train Model. The trained
model can then be used to predict values for the new input examples.
About neural networks

A neural network is a set of interconnected layers. The inputs are the first layer, and are connected to an output
layer by an acyclic graph comprised of weighted edges and nodes.
Between the input and output layers you can insert multiple hidden layers. Most predictive tasks can be
accomplished easily with only one or a few hidden layers. However, recent research has shown that deep neural
networks (DNN ) with many layers can be effective in complex tasks such as image or speech recognition. The
successive layers are used to model increasing levels of semantic depth.
The relationship between inputs and outputs is learned from training the neural network on the input data. The
direction of the graph proceeds from the inputs through the hidden layer and to the output layer. All nodes in a
layer are connected by the weighted edges to nodes in the next layer.
To compute the output of the network for a particular input, a value is calculated at each node in the hidden layers
and in the output layer. The value is set by calculating the weighted sum of the values of the nodes from the
previous layer. An activation function is then applied to that weighted sum.
Configure Multiclass Neural Network

1. Add the MultiClass Neural Network module to your experiment in the interface. You can find this module
under Machine Learning, Initialize, in the Classification category.
2. Create trainer mode: Use this option to specify how you want the model to be trained:
3. Hidden layer specification: Select the type of network architecture to create.
Fully connected case: Select this option to create a model using the default neural network
architecture. For multiclass neural network models, the defaults are as follows:
One hidden layer
The output layer is fully connected to the hidden layer.
The hidden layer is fully connected to the input layer.
The number of nodes in the input layer is determined by the number of features in the training
data.
The number of nodes in the hidden layer can be set by the user. The default is 100.
The number of nodes in the output layer depends on the number of classes.
4. Number of hidden nodes: This option lets you customize the number of hidden nodes in the default
architecture. Type the number of hidden nodes. The default is one hidden layer with 100 nodes.
5. The learning rate: Define the size of the step taken at each iteration, before correction.A larger value for
learning rate can cause the model to converge faster, but it can overshoot local minima.
6. Number of learning iterations: Specify the maximum number of times the algorithm should process the
training cases.
7. The initial learning weights diameter: Specify the node weights at the start of the learning process.
8. The momentum: Specify a weight to apply during learning to nodes from previous iterations.
9. Shuffle examples: Select this option to shuffle cases between iterations.
If you deselect this option, cases are processed in exactly the same order each time you run the experiment.
10. Random number seed: Type a value to use as the seed, if you want to ensure repeatability across runs of
the same experiment.
11. Connect a training dataset and one of the training modules:
If you set Create trainer mode to Single Parameter, use Train Model.
Results
other parameters of the neural network, right-click the output of Train Model and select Visualize.
Next steps
Multiclass Decision Forest module
Use this module to create a machine learning model based on the decision forest algorithm. A decision forest is an
ensemble model that rapidly builds a series of decision trees, while learning from tagged data.
More about decision forests

The decision forest algorithm is an ensemble learning method for classification. The algorithm works by building
multiple decision trees and then voting on the most popular output class. Voting is a form of aggregation, in which
each tree in a classification decision forest outputs a non-normalized frequency histogram of labels. The
aggregation process sums these histograms and normalizes the result to get the “probabilities” for each label. The
trees that have high prediction confidence have a greater weight in the final decision of the ensemble.
Decision trees in general are non-parametric models, meaning they support data with varied distributions. In each
tree, a sequence of simple tests is run for each class, increasing the levels of a tree structure until a leaf node
(decision) is reached.
Decision trees have many advantages:
They can represent non-linear decision boundaries.
They are efficient in computation and memory usage during training and prediction.
They perform integrated feature selection and classification.
They are resilient in the presence of noisy features.
The decision forest classifier in Azure Machine Learning consists of an ensemble of decision trees. Generally,
ensemble models provide better coverage and accuracy than single decision trees. For more information, see
Decision trees.
How to configure Multiclass Decision Forest

1. Add the Multiclass Decision Forest module to your experiment in the interface. You can find this module
under Machine Learning, Initialize Model, and Classification.
2. Double-click the module to open the Properties pane.
3. For Resampling method, choose the method used to create the individual trees. You can choose from
bagging or replication.
Bagging: Bagging is also called bootstrap aggregating. In this method, each tree is grown on a new
sample, created by randomly sampling the original dataset with replacement until you have a dataset
the size of the original. The outputs of the models are combined by voting, which is a form of
aggregation. For more information, see the Wikipedia entry for Bootstrap aggregating.
which split predicate is used for each tree node remains random, creating diverse trees.
Single Parameter: Select this option if you know how you want to configure the model, and provide a
set of values as arguments.
5. Number of decision trees: Type the maximum number of decision trees that can be created in the
ensemble. By creating more decision trees, you can potentially get better coverage, but training time might
increase.
This value also controls the number of trees displayed in the results, when visualizing the trained model. To
see or print a single tree, you can set the value to 1; however, this means that only one tree can be produced
(the tree with the initial set of parameters), and no further iterations are performed.
6. Maximum depth of the decision trees: Type a number to limit the maximum depth of any decision tree.
Increasing the depth of the tree might increase precision, at the risk of some overfitting and increased
training time.
7. Number of random splits per node: Type the number of splits to use when building each node of the
8. Minimum number of samples per leaf node: Indicate the minimum number of cases that are required to
create any terminal node (leaf) in a tree. By increasing this value, you increase the threshold for creating new
rules.
For example, with the default value of 1, even a single case can cause a new rule to be created. If you
increase the value to 5, the training data would have to contain at least five cases that meet the same
conditions.
9. Connect a labeled dataset, and one of the training modules:
Results
To see the tree that was created on each iteration, right-click the output of the Train Model module, and select
Visualize.
To see the rules for each node, click each tree to drill down into the splits.
Next steps
Multiclass Boosted Decision Tree
Use this module to create a machine learning model that is based on the boosted decision trees algorithm.
A boosted decision tree is an ensemble learning method in which the second tree corrects for the errors of the first
tree, the third tree corrects for the errors of the first and second trees, and so forth. Predictions are based on the
ensemble of trees together.
How to configure
This module creates an untrained classification model. Because classification is a supervised learning method, you
need a labeled dataset that includes a label column with a value for all rows.
You can train this type of model by using the Train Model.
1. Add the Multiclass Boosted Decision Tree module to your experiment.
2. Specify how you want the model to be trained by setting the Create trainer mode option.
Maximum number of leaves per tree limits the maximum number of terminal nodes (leaves) that
can be created in any tree.
By increasing this value, you potentially increase the size of the tree and achieve higher precision, at
the risk of overfitting and longer training time.
Minimum number of samples per leaf node indicates the number of cases required to create any
terminal node (leaf) in a tree.
By increasing this value, you increase the threshold for creating new rules. For example, with the
default value of 1, even a single case can cause a new rule to be created. If you increase the value to 5,
the training data would have to contain at least five cases that meet the same conditions.
Learning rate defines the step size while learning. Enter a number between 0 and 1.
The learning rate determines how fast or slow the learner converges on an optimal solution. If the
step size is too large, you might overshoot the optimal solution. If the step size is too small, training
takes longer to converge on the best solution.
Number of trees constructed indicates the total number of decision trees to create in the
ensemble. By creating more decision trees, you can potentially get better coverage, but training time
will increase.
Random number seed optionally sets a non-negative integer to use as the random seed value.
Specifying a seed ensures reproducibility across runs that have the same data and parameters.
The random seed is set by default to 42. Successive runs using different random seeds can have
different results.
NOTE
If you set Create trainer mode to Single Parameter, connect a tagged dataset and the Train Model module.
Next steps
Two-Class Averaged Perceptron module
Use this module to create a machine learning model based on the averaged perceptron algorithm.
This classification algorithm is a supervised learning method, and requires a tagged dataset, which includes a label
column. You can train the model by providing the model and the tagged dataset as an input to Train Model. The
trained model can then be used to predict values for the new input examples.
About averaged perceptron models
The averaged perceptron method is an early and simple version of a neural network. In this approach, inputs are
classified into several possible outputs based on a linear function, and then combined with a set of weights that are
derived from the feature vector—hence the name "perceptron."
The simpler perceptron models are suited to learning linearly separable patterns, whereas neural networks
(especially deep neural networks) can model more complex class boundaries. However, perceptrons are faster, and
because they process cases serially, perceptrons can be used with continuous training.
How to configure Two-Class Averaged Perceptron

1. Add the Two-Class Averaged Perceptron module to your experiment.
Single Parameter: If you know how you want to configure the model, provide a specific set of values as
arguments.
3. For Learning rate, specify a value for the learning rate. The learning rate values control the size of the step
that is used in stochastic gradient descent each time the model is tested and corrected.
By making the rate smaller, you test the model more often, with the risk that you might get stuck in a local
plateau. By making the step larger, you can converge faster, at the risk of overshooting the true minima.
4. For Maximum number of iterations, type the number of times you want the algorithm to examine the
training data.
Stopping early often provides better generalization. Increasing the number of iterations improves fitting, at
the risk of overfitting.
5. For Random number seed, optionally type an integer value to use as the seed. Using a seed is
recommended if you want to ensure reproducibility of the experiment across runs.
6. Connect a training dataset, and one of the training modules:
Results
click the output of Train Model.
Next steps
Two-Class Decision Forest module
Use this module to create a machine learning model based on the decision forests algorithm.
Decision forests are fast, supervised ensemble models. This module is a good choice if you want to predict a target
with a maximum of two outcomes.
Understanding decision forests

This decision forest algorithm is an ensemble learning method intended for classification tasks. Ensemble methods
are based on the general principle that rather than relying on a single model, you can get better results and a more
generalized model by creating multiple related models and combining them in some way. Generally, ensemble
models provide better coverage and accuracy than single decision trees.
There are many ways to create individual models and combine them in an ensemble. This particular
implementation of a decision forest works by building multiple decision trees and then voting on the most
popular output class. Voting is one of the better-known methods for generating results in an ensemble model.
Many individual classification trees are created, using the entire dataset, but different (usually randomized)
starting points. This differs from the random forest approach, in which the individual decision trees might only
use some randomized portion of the data or features.
Each tree in the decision forest tree outputs a non-normalized frequency histogram of labels.
The aggregation process sums these histograms and normalizes the result to get the “probabilities” for each
label.
The trees that have high prediction confidence will have a greater weight in the final decision of the ensemble.
Decision trees in general have many advantages for classification tasks:
They can capture non-linear decision boundaries.
You can train and predict on lots of data, as they are efficient in computation and memory usage.
Feature selection is integrated in the training and classification processes.
Trees can accommodate noisy data and many features.
They are non-parametric models, meaning they can handle data with varied distributions.
However, simple decision trees can overfit on data, and are less generalizable than tree ensembles.
For more information, see Decision Forests.
How to configure
1. Add the Two-Class Decision Forest module to your experiment in Azure Machine Learning, and open the
Properties pane of the module.
You can find the module under Machine Learning. Expand Initialize, and then Classification.
2. For Resampling method, choose the method used to create the individual trees. You can choose from
Bagging or Replicate.
Bagging: Bagging is also called bootstrap aggregating. In this method, each tree is grown on a new
sample, created by randomly sampling the original dataset with replacement until you have a dataset
the size of the original.
The outputs of the models are combined by voting, which is a form of aggregation. Each tree in a
classification decision forest outputs an unnormalized frequency histogram of labels. The
aggregation is to sum these histograms and normalize to get the “probabilities” for each label. In this
manner, the trees that have high prediction confidence will have a greater weight in the final decision
of the ensemble.
For more information, see the Wikipedia entry for Bootstrap aggregating.
which split predicate is used for each tree node remains random and the trees will be diverse.
4. For Number of decision trees, type the maximum number of decision trees that can be created in the
ensemble. By creating more decision trees, you can potentially get better coverage, but training time
increases.
NOTE
This value also controls the number of trees displayed when visualizing the trained model. If you want to see or print
a single tree, you can set the value to 1. However, only one tree can be produced (the tree with the initial set of
parameters) and no further iterations are performed.
5. For Maximum depth of the decision trees, type a number to limit the maximum depth of any decision
tree. Increasing the depth of the tree might increase precision, at the risk of some overfitting and increased
training time.
6. For Number of random splits per node, type the number of splits to use when building each node of the
7. For Minimum number of samples per leaf node, indicate the minimum number of cases that are
required to create any terminal node (leaf) in a tree.
8. Select the Allow unknown values for categorical features option to create a group for unknown values
in the training or validation sets. The model might be less precise for known values, but it can provide better
predictions for new (unknown) values.
If you deselect this option, the model can accept only the values that are contained in the training data.
9. Attach a labeled dataset, and one of the training modules:
Results
To see the tree that was created on each iteration, right-click the output of the Train Model module, and
select Visualize.
Click each tree to drill down into the splits and see the rules for each node.
To save a snapshot of the model, right-click the Trained Model output, and select Save Model. The saved
model is not updated on successive runs of the experiment.
To use the model for scoring, add the Score Model module to an experiment.
Next steps
Two-Class Neural Network module
Use this module to create a neural network model that can be used to predict a target that has only two values.
Classification using neural networks is a supervised learning method, and therefore requires a tagged dataset,
which includes a label column. For example, you could use this neural network model to predict binary outcomes
such as whether or not a patient has a certain disease, or whether a machine is likely to fail within a specified
window of time.
After you define the model, train it by providing a tagged dataset and the model as an input to Train Model. The
trained model can then be used to predict values for new inputs.
More about neural networks
A neural network is a set of interconnected layers. The inputs are the first layer, and are connected to an output
layer by an acyclic graph comprised of weighted edges and nodes.
Between the input and output layers you can insert multiple hidden layers. Most predictive tasks can be
accomplished easily with only one or a few hidden layers. However, recent research has shown that deep neural
networks (DNN ) with many layers can be effective in complex tasks such as image or speech recognition. The
successive layers are used to model increasing levels of semantic depth.
The relationship between inputs and outputs is learned from training the neural network on the input data. The
direction of the graph proceeds from the inputs through the hidden layer and to the output layer. All nodes in a
layer are connected by the weighted edges to nodes in the next layer.
To compute the output of the network for a particular input, a value is calculated at each node in the hidden layers
and in the output layer. The value is set by calculating the weighted sum of the values of the nodes from the
previous layer. An activation function is then applied to that weighted sum.
How to configure
1. Add the Two-Class Neural Network module to your experiment. You can find this module under
Machine Learning, Initialize, in the Classification category.
3. For Hidden layer specification, select the type of network architecture to create.
Fully connected case: Uses the default neural network architecture, defined for two-class neural
networks as follows:
Has one hidden layer.
The output layer is fully connected to the hidden layer, and the hidden layer is fully connected
to the input layer.
The number of nodes in the input layer equals the number of features in the training data.
The number of nodes in the hidden layer is set by the user. The default value is 100.
The number of nodes equals the number of classes. For a two-class neural network, this
means that all inputs must map to one of two nodes in the output layer.
4. For Learning rate, define the size of the step taken at each iteration, before correction. A larger value for
learning rate can cause the model to converge faster, but it can overshoot local minima.
5. For Number of learning iterations, specify the maximum number of times the algorithm should process
the training cases.
6. For The initial learning weights diameter, specify the node weights at the start of the learning process.
7. For The momentum, specify a weight to apply during learning to nodes from previous iterations
8. Select the Shuffle examples option to shuffle cases between iterations. If you deselect this option, cases
are processed in exactly the same order each time you run the experiment.
9. For Random number seed, type a value to use as the seed.
Specifying a seed value is useful when you want to ensure repeatability across runs of the same experiment.
Otherwise, a system clock value is used as the seed, which can cause slightly different results each time you
run the experiment.
10. Add a tagged dataset to the experiment, and connect one of the training modules.
Results
other parameters of the neural network, right-click the output of Train Model, and select Visualize.
Next steps
Two-Class Support Vector Machine module
Use this module to create a model that is based on the support vector machine algorithm.
Support vector machines (SVMs) are a well-researched class of supervised learning methods. This particular
implementation is suited to prediction of two possible outcomes, based on either continuous or categorical
variables.
After defining the model parameters, train the model by using the training modules, and providing a tagged
dataset that includes a label or outcome column.
About support vector machines

Support vector machines are among the earliest of machine learning algorithms, and SVM models have been used
in many applications, from information retrieval to text and image classification. SVMs can be used for both
classification and regression tasks.
This SVM model is a supervised learning model that requires labeled data. In the training process, the algorithm
analyzes input data and recognizes patterns in a multi-dimensional feature space called the hyperplane. All input
examples are represented as points in this space, and are mapped to output categories in such a way that
categories are divided by as wide and clear a gap as possible.
For prediction, the SVM algorithm assigns new examples into one category or the other, mapping them into that
same space.
How to configure
For this model type, it is recommended that you normalize the dataset before using it to train the classifier.
1. Add the Two-Class Support Vector Machine module to your experiment.
3. For Number of iterations, type a number that denotes the number of iterations used when building the
model.
This parameter can be used to control trade-off between training speed and accuracy.
4. For Lambda, type a value to use as the weight for L1 regularization.
This regularization coefficient can be used to tune the model. Larger values penalize more complex models.
5. Select the option, Normalize features, if you want to normalize features before training.
If you apply normalization, before training, data points are centered at the mean and scaled to have one unit
of standard deviation.
6. Select the option, Project to the unit sphere, to normalize coefficients.
Projecting values to unit space means that before training, data points are centered at 0 and scaled to have
one unit of standard deviation.
7. In Random number seed, type an integer value to use as a seed if you want to ensure reproducibility
across runs. Otherwise, a system clock value is used as a seed, which can result in slightly different results
across runs.
8. Connect a labeled dataset, and one of the training modules:
Results
click the output of Train Model, and select Visualize.
To use the trained models to make predictions, connect the trained model to the Score Model module.
Next steps
Module: K-Means Clustering
This article describes how to use the K -Means Clustering module in Azure Machine Learning Studio to create an
untrained K-means clustering model.
K-means is one of the simplest and the best known unsupervised learning algorithms. You can use the algorithm
for a variety of machine learning tasks, such as:
Detecting abnormal data.
Clustering text documents.
Analyzing datasets before you use other classification or regression methods.
To create a clustering model, you:
Add this module to your experiment.
Connect a dataset.
Set parameters, such as the number of clusters you expect, the distance metric to use in creating the clusters,
and so forth.
After you've configured the module hyperparameters, you connect the untrained model to the Train Clustering
Model. Because the K-means algorithm is an unsupervised learning method, a label column is optional.
If your data includes a label, you can use the label values to guide selection of the clusters and optimize the
model.
If your data has no label, the algorithm creates clusters representing possible categories, based solely on
the data.
Understand K-means clustering

In general, clustering uses iterative techniques to group cases in a dataset into clusters that possess similar
characteristics. These groupings are useful for exploring data, identifying anomalies in the data, and eventually for
making predictions. Clustering models can also help you identify relationships in a dataset that you might not
logically derive by browsing or simple observation. For these reasons, clustering is often used in the early phases
of machine learning tasks, to explore the data and discover unexpected correlations.
When you configure a clustering model by using the K-means method, you must specify a target number k that
indicates the number of centroids you want in the model. The centroid is a point that's representative of each
cluster. The K-means algorithm assigns each incoming data point to one of the clusters by minimizing the within-
cluster sum of squares.
When it processes the training data, the K-means algorithm begins with an initial set of randomly chosen
centroids. Centroids serve as starting points for the clusters, and they apply Lloyd's algorithm to iteratively refine
their locations. The K-means algorithm stops building and refining clusters when it meets one or more of these
conditions:
The centroids stabilize, meaning that the cluster assignments for individual points no longer change and
the algorithm has converged on a solution.
The algorithm completed running the specified number of iterations.
After you've completed the training phase, you use the Assign Data to Clusters module to assign new cases to one
of the clusters that you found by using the K-means algorithm. You perform cluster assignment by computing the
distance between the new case and the centroid of each cluster. Each new case is assigned to the cluster with the
nearest centroid.
Configure the K-Means Clustering module

1. Add the K-Means Clustering module to your experiment.
2. To specify how you want the model to be trained, select the Create trainer mode option.
Single Parameter: If you know the exact parameters you want to use in the clustering model, you can
provide a specific set of values as arguments.
3. For Number of Centroids, type the number of clusters you want the algorithm to begin with.
The model isn't guaranteed to produce exactly this number of clusters. The algorithm starts with this
number of data points and iterates to find the optimal configuration.
4. The properties Initialization is used to specify the algorithm that's used to define the initial cluster
configuration.
First N: Some initial number of data points are chosen from the dataset and used as the initial
means.
This method is also called the Forgy method.
Random: The algorithm randomly places a data point in a cluster and then computes the initial
mean to be the centroid of the cluster's randomly assigned points.
This method is also called the random partition method.
K-Means++: This is the default method for initializing clusters.
The K-means++ algorithm was proposed in 2007 by David Arthur and Sergei Vassilvitskii to avoid
poor clustering by the standard K-means algorithm. K-means++ improves upon standard K-means
by using a different method for choosing the initial cluster centers.
5. For Random number seed, optionally type a value to use as the seed for the cluster initialization. This
value can have a significant effect on cluster selection.
6. For Metric, choose the function to use for measuring the distance between cluster vectors, or between new
data points and the randomly chosen centroid. Azure Machine Learning supports the following cluster
distance metrics:
Euclidean: The Euclidean distance is commonly used as a measure of cluster scatter for K-means
clustering. This metric is preferred because it minimizes the mean distance between points and the
centroids.
7. For Iterations, type the number of times the algorithm should iterate over the training data before it
finalizes the selection of centroids.
You can adjust this parameter to balance accuracy against training time.
8. For Assign label mode, choose an option that specifies how a label column, if it's present in the dataset,
should be handled.
Because K-means clustering is an unsupervised machine learning method, labels are optional. However, if
your dataset already has a label column, you can use those values to guide the selection of the clusters, or
you can specify that the values be ignored.
Ignore label column: The values in the label column are ignored and are not used in building the
model.
Fill missing values: The label column values are used as features to help build the clusters. If any
rows are missing a label, the value is imputed by using other features.
Overwrite from closest to center: The label column values are replaced with predicted label
values, using the label of the point that is closest to the current centroid.
9. Select the Normalize features option if you want to normalize features before training.
If you apply normalization, before training, the data points are normalized to [0,1] by
MinMaxNormalizer.
10. Train the model.
If you set Create trainer mode to Single Parameter, add a tagged dataset and train the model by
using the Train Clustering Model module.
Results
After you've finished configuring and training the model, you have a model that you can use to generate scores.
However, there are multiple ways to train the model, and multiple ways to view and use the results:
Capture a snapshot of the model in your workspace
If you used the Train Clustering Model module:
1. Right-click the Train Clustering Model module.
2. Select Trained model, and then select Save as Trained Model.
The saved model represents the training data at the time you saved the model. If you later update the training
data used in the experiment, it doesn't update the saved model.
See the clustering result dataset
If you used the Train Clustering Model module:
1. Right-click the Train Clustering Model module.
2. Select Results dataset, and then select Visualize.
Tips for generating the best clustering model
It is known that the seeding process that's used during clustering can significantly affect the model. Seeding
means the initial placement of points into potential centroids.
For example, if the dataset contains many outliers, and an outlier is chosen to seed the clusters, no other data
points would fit well with that cluster, and the cluster could be a singleton. That is, it might have only one point.
You can avoid this problem in a couple of ways:
Change the number of centroids and try multiple seed values.
Create multiple models, varying the metric or iterating more.
In general, with clustering models, it's possible that any given configuration will result in a locally optimized set of
clusters. In other words, the set of clusters that's returned by the model suits only the current data points and isn't
generalizable to other data. If you use a different initial configuration, the K-means method might find a different,
perhaps superior, configuration.
Execute Python Script module
Use this module to run Python code. For more information about the architecture and design principles of Python,
see the following article.
With Python, you can perform tasks that aren't currently supported by existing modules such as:
Visualizing data using matplotlib
Using Python libraries to enumerate datasets and models in your workspace
Reading, loading, and manipulating data from sources not supported by the Import Data module
Run your own deep learning code
Azure Machine Learning uses the Anaconda distribution of Python, which includes many common utilities for
data processing. We will update Anaconda version automatically. Current version is:
Anaconda 4.5+ distribution for Python 3.6
The pre-installed packages are:
asn1crypto==0.24.0
attrs==19.1.0
azure-common==1.1.18
azure-storage-blob==1.5.0
azure-storage-common==1.4.0
certifi==2019.3.9
cffi==1.12.2
chardet==3.0.4
cryptography==2.6.1
distro==1.4.0
idna==2.8
jsonschema==3.0.1
lightgbm==2.2.3
more-itertools==6.0.0
numpy==1.16.2
pandas==0.24.2
Pillow==6.0.0
pip==19.0.3
pyarrow==0.12.1
pycparser==2.19
pycryptodomex==3.7.3
pyrsistent==0.14.11
python-dateutil==2.8.0
pytz==2018.9
requests==2.21.0
scikit-learn==0.20.3
scipy==1.2.1
setuptools==40.8.0
six==1.12.0
torch==1.0.1.post2
torchvision==0.2.2.post3
urllib3==1.24.1
wheel==0.33.1
To install other packages not in the pre-installed list, for example scikit-misc, add the following code to your script:
import os
os.system(f"pip install scikit-misc")
How to use
The Execute Python Script module contains sample Python code that you can use as a starting point. To
configure the Execute Python Script module, you provide a set of inputs and Python code to execute in the
Python script text box.
1. Add the Execute Python Script module to your experiment.
2. Add and connect on Dataset1 any datasets from the interface that you want to use for input. Reference this
dataset in your Python script as DataFrame1.
Use of a dataset is optional, if you want to generate data using Python, or use Python code to import the
data directly into the module.
This module supports addition of a second dataset on Dataset2. Reference the second dataset in your
Python script as DataFrame2.
Datasets stored in Azure Machine Learning are automatically converted to pandas data.frames when
loaded with this module.
3. To include new Python packages or code, add the zipped file containing these custom resources on Script
bundle. The input to Script bundle must be a zipped file already uploaded to your workspace.
Any file contained in the uploaded zipped archive can be used during experiment execution. If the archive
includes a directory structure, the structure is preserved, but you must prepend a directory called src to the
path.
4. In the Python script text box, type or paste valid Python script.
The Python script text box is pre-populated with some instructions in comments, and sample code for
data access and output. You must edit or replace this code. Be sure to follow Python conventions about
indentation and casing.
The script must contain a function named azureml_main as the entry point for this module.
The entry point function can contain up to two input arguments: Param<dataframe1> and
Param<dataframe2>
Zipped files connected to the third input port are unzipped and stored in the directory, .\Script Bundle ,
which is also added to the Python sys.path .
Therefore, if your zip file contains mymodule.py , import it using import mymodule .
Two datasets can be returned to the interface, which must be a sequence of type pandas.DataFrame . You
can create other outputs in your Python code and write them directly to Azure storage.
5. Run the experiment, or select the module and click Run selected to run just the Python script.
All of the data and code is loaded into a virtual machine, and run using the specified Python environment.
Results
The results of any computations performed by the embedded Python code must be provided as a
pandas.DataFrame, which is automatically converted to the Azure Machine Learning dataset format, so that you
can use the results with other modules in the experiment.
The module returns two datasets:
Results Dataset 1, defined by the first returned pandas dataframe in Python script
Result Dataset 2, defined by the second returned pandas dataframe in Python script
Next steps
Create Python Model
This article describes how to use the Create Python Model module to create an untrained model from a Python
script.
You can base the model on any learner that is included in an Python package in the Azure Machine Learning
environment.
After you create the model, you can use Train Model to train the model on a dataset, like any other learner in Azure
Machine Learning. The trained model can be passed to Score Model to use the model to make predictions. The
trained model can then be saved, and the scoring workflow can be published as a web service.
WARNING
Currently it is not possible to pass the scored results of a Python model to Evaluate Model. If you need to evaluate a model,
you can write custom Python script and run it using the Execute Python Script module.
How to configure Create Python Model

Use of this module requires intermediate or expert knowledge of Python. The module supports use of any learner
that is included in the Python packages already installed in Azure Machine Learning. See pre-installed Python
package list in Execute Python Script.
This article will show how to use the Create Python Model with a simple experiment. Below is the graph of the
experiment.
1. Click Create Python Model, edit the script to implement your modelling or data management process.
You can base the model on any learner that is included in a Python package in the Azure Machine Learning
environment.
Below is a sample code of two-class Naive Bayes classifier by using the popular sklearn package.
# The script MUST define a class named AzureMLModel.
# This class MUST at least define the following three methods:
# __init__: in which self.model must be assigned,
# train: which trains self.model, the two input arguments must be pandas DataFrame,
# predict: which generates prediction result, the input argument and the prediction result MUST be pandas
DataFrame.
# The signatures (method names and argument names) of all these methods MUST be exactly the same as the
following example.
import pandas as pd
from sklearn.naive_bayes import GaussianNB
class AzureMLModel:
def __init__(self):
self.model = GaussianNB()
self.feature_column_names = list()
def train(self, df_train, df_label):

self.feature_column_names = df_train.columns.tolist()
self.model.fit(df_train, df_label)
def predict(self, df):

return pd.DataFrame(
{'Scored Labels': self.model.predict(df[self.feature_column_names]),
'probabilities': self.model.predict_proba(df[self.feature_column_names])[:, 1]}
)
2. Connect the Create Python Model module you just created to a Train Model and Score Model
3. If you need to evaluate the model, add a Execute Python Script and edit the Python script to implement
evaluation.
Below is sample evaluation code.
# The script MUST contain a function named azureml_main
# which is the entry point for this module.
# imports up here can be used to

import pandas as pd
# The entry point function can contain up to two input arguments:

# Param<dataframe1>: a pandas.DataFrame
# Param<dataframe2>: a pandas.DataFrame
from sklearn.metrics import accuracy_score, precision_score, recall_score, roc_auc_score, roc_curve

import pandas as pd
import numpy as np
scores = dataframe1.ix[:, ("income", "Scored Labels", "probabilities")]

ytrue = np.array([0 if val == '<=50K' else 1 for val in scores["income"]])
ypred = np.array([0 if val == '<=50K' else 1 for val in scores["Scored Labels"]])
probabilities = scores["probabilities"]
accuracy, precision, recall, auc = \

accuracy_score(ytrue, ypred),\
precision_score(ytrue, ypred),\
recall_score(ytrue, ypred),\
roc_auc_score(ytrue, probabilities)
metrics = pd.DataFrame();
metrics["Metric"] = ["Accuracy", "Precision", "Recall", "AUC"];
metrics["Value"] = [accuracy, precision, recall, auc]
return metrics,
Execute R Script
This article describes how to use the Execute R Script module to run R code in your visual interface experiment.
With R, you can perform tasks that aren't currently supported by existing modules such as:
Create custom data transformations
Use your own metrics to evaluate predictions
Build models using algorithms that aren't implemented as standalone modules in visual interface
R version support
The Azure Machine Learning service visual interface uses the CRAN (Comprehensive R Archive Network)
distribution of R. The currently used version is CRAN 3.5.1.
Supported R packages
The R environment is pre-installed with over 100 packages. For a complete list, see the section Pre-installed R
packages.
You can also add the following code to any Execute R Script module and to see the installed packages.
azureml_main <- function(dataframe1, dataframe2){

print("R script run.")
dataframe1 <- data.frame(installed.packages())
return(list(dataset1=dataframe1, dataset2=dataframe2))
}
Installing R packages
To install additional R packages, use the install.packages() method. Be sure to specify the CRAN repository.
Packages are installed for each Execute R Script module, and aren't shared across other Execute R Script
modules.
This sample shows how to install Zoo:
# R version: 3.5.1

# Param<dataframe1>: a R DataFrame
if(!require(zoo)) install.packages("zoo",repos = "http://cran.us.r-project.org")

library(zoo)
# Return datasets as a Named List
}
NOTE
Please check if the package already exists before install it to avoid repeat installing. Like
if(!require(zoo)) install.packages("zoo",repos = "http://cran.us.r-project.org") in above sample code. Repeat
installing may cause web service request timeout.
How to configure Execute R Script

The Execute R Script module contains sample code that you can use as a starting point. To configure the Execute
R Script module, provide a set of inputs and code to execute.
Datasets stored in visual interface are automatically converted to an R data frame when loaded with this module.
1. Add the Execute R Script module to your experiment.
2. Connect any inputs needed by the script. Inputs are optional and can include data and additional R code.
Dataset1: Reference the first input as dataframe1 . The input dataset must be formatted as a CSV,
TSV, ARFF, or you can connect an Azure Machine Learning dataset.
Dataset2: Reference the second input as dataframe2 . This dataset also must be formatted as a CSV,
TSV, ARFF file, or as an Azure Machine Learning dataset.
Script Bundle: The third input accepts ZIP files. The zipped file can contain multiple files and
multiple file types.
3. In the R script text box, type or paste valid R script.
To help you get started, the R Script text box is pre-populated with sample code, which you can edit or
replace.
# R version: 3.5.1

# If a zip file is connected to the third input port, it is

# unzipped under "./Script Bundle". This directory is added
# to sys.path.

}
The script must contain a function named azureml_main , which is the entry point for this module.
The entry point function can contain up to two input arguments: Param<dataframe1> and Param<dataframe2>
NOTE
The data passed to the Execute R Script module is referenced as dataframe1 and dataframe2 , which is different
from Azure Machine Learning Studio (Studio reference as dataset1 , dataset2 ). Please check to make sure input
data is referneced correctly in your script.
NOTE
Existing R code may need minor changes to run in a visual interface experiment. For example, input data that you
provide in CSV format should be explicitly converted to a dataset before you can use it in your code. Data and
column types used in the R language also differ in some ways from the data and column types used in the visual
interface.
1. Random Seed: Type a value to use inside the R environment as the random seed value. This parameter is
equivalent to calling set.seed(value) in R code.
Results
The Execute R Script modules can return multiple outputs, but they must be provided as R data frames. Data
frames are automatically converted to visual interface datasets for compatibility with other modules.
Standard messages and errors from R are returned to the module's log.
Sample scripts
There are many ways that you can extend your experiment by using custom R script. This section provides sample
code for common tasks.
Add R script as an input
The Execute R Script module supports arbitrary R script files as inputs. To do so, they must be uploaded to your
workspace as part of the ZIP file.
1. To upload a ZIP file containing R code to your workspace, click New, click Dataset, and then select From
local file and the Zip file option.
2. Verify that the zipped file is available in the Saved Datasets list.
3. Connect the dataset to the Script Bundle input port.
4. All files that are contained in the ZIP file are available during experiment run time.
If the script bundle file contained a directory structure, the structure is preserved. However, you must alter
your code to prepend the directory ./Script Bundle to the path.
Process data
The following sample shows how to scale and normalize input data:
# R version: 3.5.1
# If a zip file is connected to the third input port, it is
# unzipped under "./Script Bundle". This directory is added
# to sys.path.
series <- dataframe1$width
# find the maximum and minimum values of width column in dataframe1
max_v <- max(series)
min_v <- min(series)
# calculate the scale and bias
scale <- max_v - min_v
bias <- min_v / dis
# apply min-max normalizing
dataframe1$width <- dataframe1$width / scale - bias
dataframe2$width <- dataframe2$width / scale - bias
}
Read a ZIP file as input

This sample shows how to use a dataset in a ZIP file as an input to the Execute R Script module.
1. Create the data file in CSV format, and name it “mydatafile.csv”.
2. Create a ZIP file and add the CSV file to the archive.
3. Upload the zipped file to your Azure Machine Learning workspace.
4. Connect the resulting dataset to the ScriptBundle input of your Execute R Script module.
5. Using the following code to read the CSV data from the zipped file.

mydataset<-read.csv("./Script Bundle/mydatafile.csv",encoding="UTF-8");
return(list(dataset1=mydataset, dataset2=dataframe2))
}
Replicate rows
This sample shows how to replicate positive records in a dataset to balance the sample:
data.set <- dataframe1[dataframe1[,1]==-1,]
# positions of the positive samples
pos <- dataframe1[dataframe1[,1]==1,]
# replicate the positive samples to balance the sample
for (i in 1:20) data.set <- rbind(data.set,pos)
row.names(data.set) <- NULL
return(list(dataset1=data.set, dataset2=dataframe2))
}
Pass R objects between Execute R Script modules

You can pass R objects between instances of the Execute R Script module by using the internal serialization
mechanism. This example assumes that you want to move the R object named A between two Execute R Script
modules.
1. Add the first Execute R Script module to your experiment, and type the following code in the R Script text
box to create a serialized object A as a column in the module’s output Data Table:

# some codes generated A
serialized <- as.integer(serialize(A,NULL))

data.set <- data.frame(serialized,stringsAsFactors=FALSE)
return(list(dataset1=data.set, dataset2=dataframe2))
}
The explicit conversion to integer type is done because the serialization function outputs data in the R Raw
format, which isn't supported by the visual interface.
2. Add a second instance of the Execute R Script module, and connect it to the output port of the previous
module.
3. Type the following code in the R Script text box to extract object A from the input Data Table.

A <- unserialize(as.raw(dataframe1$serialized))
}
Pre-installed R Packages
The current list of pre-installed R Packages available to use:
Package Version
askpass 1.1
assertthat 0.2.1
backports 1.1.4
base 3.5.1
base64enc 0.1-3
BH 1.69.0-1
bindr 0.1.1
bindrcpp 0.2.2
bitops 1.0-6
boot 1.3-22
broom 0.5.2
callr 3.2.0
caret 6.0-84
caTools 1.17.1.2
cellranger 1.1.0
class 7.3-15
cli 1.1.0
clipr 0.6.0
cluster 2.0.7-1
codetools 0.2-16
colorspace 1.4-1
compiler 3.5.1
crayon 1.3.4
curl 3.3
data.table 1.12.2
datasets 3.5.1
DBI 1.0.0
dbplyr 1.4.1
digest 0.6.19
dplyr 0.7.6
e1071 1.7-2
evaluate 0.14
fansi 0.4.0
forcats 0.3.0
foreach 1.4.4
foreign 0.8-71
fs 1.3.1
gdata 2.18.0
generics 0.0.2
ggplot2 3.2.0
glmnet 2.0-18
glue 1.3.1
gower 0.2.1
gplots 3.0.1.1
graphics 3.5.1
grDevices 3.5.1
grid 3.5.1
gtable 0.3.0
gtools 3.8.1
haven 2.1.0
highr 0.8
hms 0.4.2
htmltools 0.3.6
httr 1.4.0
ipred 0.9-9
iterators 1.0.10
jsonlite 1.6
KernSmooth 2.23-15
knitr 1.23
labeling 0.3
lattice 0.20-38
lava 1.6.5
lazyeval 0.2.2
lubridate 1.7.4
magrittr 1.5
markdown 1
MASS 7.3-51.4
Matrix 1.2-17
methods 3.5.1
mgcv 1.8-28
mime 0.7
ModelMetrics 1.2.2
modelr 0.1.4
munsell 0.5.0
nlme 3.1-140
nnet 7.3-12
numDeriv 2016.8-1.1
openssl 1.4
parallel 3.5.1
pillar 1.4.1
pkgconfig 2.0.2
plogr 0.2.0
plyr 1.8.4
prettyunits 1.0.2
processx 3.3.1
prodlim 2018.04.18
progress 1.2.2
ps 1.3.0
purrr 0.3.2
quadprog 1.5-7
quantmod 0.4-15
R6 2.4.0
randomForest 4.6-14
RColorBrewer 1.1-2
Rcpp 1.0.1
RcppRoll 0.3.0
readr 1.3.1
readxl 1.3.1
recipes 0.1.5
rematch 1.0.1
reprex 0.3.0
reshape2 1.4.3
reticulate 1.12
rlang 0.4.0
rmarkdown 1.13
ROCR 1.0-7
rpart 4.1-15
rstudioapi 0.1
rvest 0.3.4
scales 1.0.0
selectr 0.4-1
spatial 7.3-11
splines 3.5.1
SQUAREM 2017.10-1
stats 3.5.1
stats4 3.5.1
stringi 1.4.3
stringr 1.3.1
survival 2.44-1.1
sys 3.2
tcltk 3.5.1
tibble 2.1.3
tidyr 0.8.3
tidyselect 0.2.5
tidyverse 1.2.1
timeDate 3043.102
tinytex 0.13
tools 3.5.1
tseries 0.10-47
TTR 0.23-4
utf8 1.1.4
utils 3.5.1
vctrs 0.1.0
viridisLite 0.3.0
whisker 0.3-2
withr 2.1.2
xfun 0.8
xml2 1.2.0
xts 0.11-2
yaml 2.2.0
zeallot 0.1.0
zoo 1.8-6
Next steps
Exceptions and error codes for Algorithm & Module
Reference
Learn about the error messages and exception codes you might encounter using modules in Azure Machine
Learning Studio.
To resolve the issue, look for the error in this article to read about common causes. There are two ways to get the
full text of an error message in Studio:
Click the link, View Output Log, in the right pane and scroll to the bottom. The detailed error message is
displayed in the last two lines of the window.
Select the module that has the error, and click the red X. Only the pertinent error text is displayed.
If the error message text is not helpful, send us information about the context and any desired additions or
changes. You can either submit feedback on the error topic, or visit the Azure Machine Learning STUDIO forum
and post a question.
Error 0001
Exception occurs if one or more specified columns of data set couldn't be found.
You will receive this error if a column selection is made for a module, but the selected column(s) do not exist in the
input data set. This error may occur if you have manually typed in a column name or if the column selector has
provided a suggested column that did not exist in your dataset when you ran the experiment.
Resolution: Revisit the module throwing this exception and validate that the column name or names are correct
and that all referenced columns do exist.
EXCEPTION MESSAGES
One or more specified columns were not found
Column with name or index "{0}" not found
Column with name or index "{0}" does not exist in "{1}"
Error 0002
Exception occurs if one or more parameters could not be parsed or converted from specified type into required by
target method type.
This error occurs in Azure Machine Learning when you specify a parameter as input and the value type is different
from the type that is expected, and implicit conversion cannot be performed.
Resolution: Check the module requirements and determine which value type is required (string, integer, double,
etc.)
EXCEPTION MESSAGES
Failed to parse parameter
Failed to parse "{0}" parameter
Failed to parse (convert) "{0}" parameter to "{1}"
Failed to convert "{0}" parameter from "{1}" to "{2}"
Failed to convert "{0}" parameter value "{1}" from "{2}" to "{3}"
Failed to convert value "{0}" in column "{1}" from "{2}" to "{3}" with usage of the format "{4}" provided
Error 0003
Exception occurs if one or more of inputs are null or empty.
You will receive this error in Azure Machine Learning if any inputs or parameters to a module are null or empty.
This error might occur, for example, when you did not type in any value for a parameter. It can also happen if you
chose a dataset that has missing values, or an empty dataset.
Resolution:
Open the module that produced the exception and verify that all inputs have been specified. Ensure that all
required inputs are specified.
Make sure that data that is loaded from Azure storage is accessible, and that the account name or key has not
changed.
Check the input data for missing values, or nulls.
If using a query on a data source, verify that data is being returned in the format you expect.
Check for typos or other changes in the specification of data.
EXCEPTION MESSAGES
One or more of inputs are null or empty
Input "{0}" is null or empty
Error 0004
Exception occurs if parameter is less than or equal to specific value.
You will receive this error in Azure Machine Learning if the parameter in the message is below a boundary value
required for the module to process the data.
Resolution: Revisit the module throwing the exception and modify the parameter to be greater than the specified
value.
EXCEPTION MESSAGES
Parameter should be greater than boundary value.
Parameter "{0}" value should be greater than {1}.

EXCEPTION MESSAGES
Parameter "{0}" has value "{1}" which should be greater than {2}
Error 0005
Exception occurs if parameter is less than a specific value.
You will receive this error in Azure Machine Learning if the parameter in the message is below or equal to a
boundary value required for the module to process the data.
Resolution: Revisit the module throwing the exception and modify the parameter to be greater than or equal to
the specified value.
EXCEPTION MESSAGES
Parameter should be greater than or equal to boundary value.
Parameter "{0}" value should be greater than or equal to {1}.
Parameter "{0}" has value "{1}" which should be greater than or equal to {2}.
Error 0006
Exception occurs if parameter is greater than or equal to the specified value.
You will receive this error in Azure Machine Learning if the parameter in the message is greater than or equal to a
boundary value required for the module to process the data.
Resolution: Revisit the module throwing the exception and modify the parameter to be less than the specified
value.
EXCEPTION MESSAGES
Parameters mismatch. One of the parameters should be less than another.
Parameter "{0}" value should be less than parameter "{1}" value.
Parameter "{0}" has value "{1}" which should be less than {2}.
Error 0007
Exception occurs if parameter is greater than a specific value.
You will receive this error in Azure Machine Learning if, in the properties for the module, you specified a value that
is greater than is allowed. For example, you might specify a data that is outside the range of supported dates, or
you might indicate that five columns be used when only three columns are available.
You might also see this error if you are specifying two sets of data that need to match in some way. For example, if
you are renaming columns, and specify the columns by index, the number of names you supply must match the
number of column indices. Another example might be a math operation that uses two columns, where the columns
must have the same number of rows.
Resolution:
Open the module in question and review any numeric property settings.
Ensure that any parameter values fall within the supported range of values for that property.
If the module takes multiple inputs, ensure that inputs are of the same size.
Check whether the dataset or data source has changed. Sometimes a value that worked with a previous version
of the data will fail after the number of columns, the column data types, or the size of the data has changed.
EXCEPTION MESSAGES
Parameters mismatch. One of the parameters should be less than or equal to another.
Parameter "{0}" value should be less than or equal to parameter "{1}" value.
Parameter "{0}" has value "{1}" which should be less than or equal to {2}.
Error 0008
Exception occurs if parameter is not in range.
You will receive this error in Azure Machine Learning if the parameter in the message is outside the bounds
required for the module to process the data.
For example, this error is displayed if you try to use Add Rows to combine two datasets that have a different
number of columns.
Resolution: Revisit the module throwing the exception and modify the parameter to be within the specified range.
EXCEPTION MESSAGES
Parameter value is not in the specified range.
Parameter "{0}" value is not in range.
Parameter "{0}" value should be in the range of [{1}, {2}].
Error 0009
Exception occurs when the Azure storage account name or container name is specified incorrectly.
This error occurs in Azure Machine Learning Studio when you specify parameters for an Azure storage account,
but the name or password cannot be resolved. Errors on passwords or account names can happen for many
reasons:
The account is the wrong type. Some new account types are not supported for use with Machine Learning
Studio. See Import Data for details.
You entered the incorrect account name
The account no longer exists
The password for the storage account is wrong or has changed
You didn't specify the container name, or the container does not exist
You didn't fully specify the file path (path to the blob)
Resolution:
Such problems often occur when you try to manually enter the account name, password, or container path. We
recommend that you use the new wizard for the Import Data module, which helps you look up and check names.
Also check whether the account, container, or blob has been deleted. Use another Azure storage utility to verify that
the account name and password have been entered correctly, and that the container exists.
Some newer account types are not supported by Azure Machine Learning. For example, the new "hot" or "cold"
storage types cannot be used for machine learning. Both classic storage accounts and storage accounts created as
"General purpose" work fine.
If the complete path to a blob was specified, verify that the path is specified as container/blobname, and that
both the container and the blob exist in the account.
The path should not contain a leading slash. For example /container/blob is incorrect and should be entered as
container/blob.
EXCEPTION MESSAGES
The Azure storage account name or container name is incorrect.
The Azure storage account name "{0}" or container name "{1}" is incorrect; a container name of the format container/blob was
expected.
Error 0010
Exception occurs if input datasets have column names that should match but do not.
You will receive this error in Azure Machine Learning if the column index in the message has different column
names in the two input datasets.
Resolution: Use Edit Metadata or modify the original dataset to have the same column name for the specified
column index.
EXCEPTION MESSAGES
Columns with corresponding index in input datasets have different names.
Column names are not the same for column {0} (zero-based) of input datasets ({1} and {2} respectively).
Error 0011
Exception occurs if passed column set argument does not apply to any of dataset columns.
You will receive this error in Azure Machine Learning if the specified column selection does not match any of the
columns in the given dataset.
You can also get this error if you haven't selected a column and at least one column is required for the module to
work.
Resolution: Modify the column selection in the module so that it will apply to the columns in the dataset.
If the module requires that you select a specific column, such as a label column, verify that the right column is
selected.
If inappropriate columns are selected, remove them and rerun the experiment.
EXCEPTION MESSAGES
Specified column set does not apply to any of dataset columns.
Specified column set "{0}" does not apply to any of dataset columns.
Error 0012
Exception occurs if instance of class could not be created with passed set of arguments.
Resolution: This error is not actionable by the user and will be deprecated in a future release.
EXCEPTION MESSAGES
Untrained model, train model first.
Untrained model ({0}), use trained model.
Error 0013
Exception occurs if the learner passed to the module is an invalid type.
This error occurs whenever a trained model is incompatible with the connected scoring module.
Resolution:
Determine the type of learner that is produced by the training module, and determine the scoring module that is
appropriate for the learner.
If the model was trained using any of the specialized training modules, connect the trained model only to the
corresponding specialized scoring module.
MODEL TYPE TRAINING MODULE SCORING MODULE
any classifier Train Model Score Model
any regression model Train Model Score Model
EXCEPTION MESSAGES
Learner of invalid type is passed.
Learner "{0}" has invalid type.
Error 0014
Exception occurs if the count of column unique values is greater than allowed.
This error occurs when a column contains too many unique values. For example, you might see this error if you
specify that a column be handled as categorical data, but there are too many unique values in the column to allow
processing to complete. You might also see this error if there is a mismatch between the number of unique values
in two inputs.
Resolution:
Open the module that generated the error, and identify the columns used as inputs. For some modules, you can
right-click the dataset input and select Visualize to get statistics on individual columns, including the number of
unique values and their distribution.
For columns that you intend to use for grouping or categorization, take steps to reduce the number of unique
values in columns. You can reduce in different ways, depending on the data type of the column.
TIP
Unable to find a resolution that matches your scenario? You can provide feedback on this topic that includes the name of the
module that generated the error, and the data type and cardinality of the column. We will use the information to provide
more targeted troubleshooting steps for common scenarios.
EXCEPTION MESSAGES
Number of column unique values is greater than allowed.
Number of unique values in column: "{0}" exceeds tuple count of {1}.
Error 0015
Exception occurs if database connection has failed.
You will receive this error if you enter an incorrect SQL account name, password, database server, or database
name, or if a connection with the database cannot be established due to problems with the database or server.
Resolution: Verify that the account name, password, database server, and database have been entered correctly,
and that the specified account has the correct level of permissions. Verify that the database is currently accessible.
EXCEPTION MESSAGES
Error making database connection.
Error making database connection: {0}.
Error 0016
Exception occurs if input datasets passed to the module should have compatible column types but do not.
You will receive this error in Azure Machine Learning if the types of the columns passed in two or more datasets
are not compatible with each other.
Resolution: Use Edit Metadata or modify the original input dataset to ensure that the types of the columns are
compatible.
EXCEPTION MESSAGES
Columns with corresponding index in input datasets do have incompatible types.
Columns {0} and {1} are incompatible.
Column element types are not compatible for column {0} (zero-based) of input datasets ({1} and {2} respectively).
Error 0017
Exception occurs if a selected column uses a data type that is not supported by the current module.
For example, you might receive this error in Azure Machine Learning if your column selection includes a column
with a data type that cannot be processed by the module, such as a string column for a math operation, or a score
column where a categorical feature column is required.
Resolution:
1. Identify the column that is the problem.
2. Review the requirements of the module.
3. Modify the column to make it conform to requirements. You might need to use several of the following
modules to make changes, depending on the column and the conversion you are attempting:
Use Edit Metadata to change the data type of columns, or to change the column usage from feature to
numeric, categorical to non-categorical, and so forth.
4. As a last resort, you might need to modify the original input dataset.
TIP
Unable to find a resolution that matches your scenario? You can provide feedback on this topic that includes the name of the
module that generated the error, and the data type and cardinality of the column. We will use the information to provide
more targeted troubleshooting steps for common scenarios.
EXCEPTION MESSAGES
Cannot process column of current type. The type is not supported by the module.
Cannot process column of type {0}. The type is not supported by the module.
Cannot process column "{1}" of type {0}. The type is not supported by the module.
Cannot process column "{1}" of type {0}. The type is not supported by the module. Parameter name: {2}
Error 0018
Exception occurs if input dataset is not valid.
Resolution: This error in Azure Machine Learning can appear in many contexts, so there is not a single resolution.
In general, the error indicates that the data provided as input to a module has the wrong number of columns, or
that the data type does not match requirements of the module. For example:
The module requires a label column, but no column is marked as a label, or you have not selected a label
column yet.
The module requires that data be categorical but your data is numeric.
The data is in the wrong format.
Imported data contains invalid characters, bad values, or out of range values.
The column is empty or contains too many missing values.
To determine the requirements and how your data might, review the help topic for the module that will be
consuming the dataset as input.
.
EXCEPTION MESSAGES
Dataset is not valid.
{0} contains invalid data.
{0} and {1} should be consistent column wise.
Error 0019
Exception occurs if column is expected to contain sorted values, but it does not.
You will receive this error in Azure Machine Learning if the specified column values are out of order.
Resolution: Sort the column values by manually modifying the input dataset and rerun the module.
EXCEPTION MESSAGES
Values in column are not sorted.
Values in column "{0}" are not sorted.
Values in column "{0}" of dataset "{1}" are not sorted.
Error 0020
Exception occurs if number of columns in some of the datasets passed to the module is too small.
You will receive this error in Azure Machine Learning if not enough columns have been selected for a module.
Resolution: Revisit the module and ensure that column selector has correct number of columns selected.
EXCEPTION MESSAGES
Number of columns in input dataset is less than allowed minimum.
Number of columns in input dataset is less than allowed minimum of {0} column(s).
Number of columns in input dataset "{0}" is less than allowed minimum of {1} column(s).
Error 0021
Exception occurs if number of rows in some of the datasets passed to the module is too small.
This error in seen in Azure Machine Learning when there are not enough rows in the dataset to perform the
specified operation. For example, you might see this error if the input dataset is empty, or if you are trying to
perform an operation that requires some minimum number of rows to be valid. Such operations can include (but
are not limited to) grouping or classification based on statistical methods, certain types of binning, and learning
with counts.
Resolution:
Open the module that returned the error, and check the input dataset and module properties.
Verify that the input dataset is not empty and that there are enough rows of data to meet the requirements
described in module help.
If your data is loaded from an external source, make sure that the data source is available and that there is no
error or change in the data definition that would cause the import process to get fewer rows.
If you are performing an operation on the data upstream of the module that might affect the type of data or the
number of values, such as cleaning, splitting, or join operations, check the outputs of those operations to
determine the number of rows returned.
Error 0022
Exception occurs if number of selected columns in input dataset does not equal to the expected number.
This error in Azure Machine Learning can occur when the downstream module or operation requires a specific
number of columns or inputs, and you have provided too few or too many columns or inputs. For example:
You specify a single label column or key column and accidentally selected multiple columns.
You are renaming columns, but provided more or fewer names than there are columns.
The number of columns in the source or destination has changed or doesn't match the number of columns
used by the module.
You have provided a comma-separated list of values for inputs, but the number of values does not match, or
multiple inputs are not supported.
Resolution: Revisit the module and check the column selection to ensure that the correct number of columns is
selected. Verify the outputs of upstream modules, and the requirements of downstream operations.
If you used one of the column selection options that can select multiple columns (column indices, all features, all
numeric, etc.), validate the exact number of columns returned by the selection.
Verify that the number or type of upstream columns has not changed.
If you are using a recommendation dataset to train a model, remember that the recommender expects a limited
number of columns, corresponding to user-item pairs or user-item-rankings. Remove additional columns before
training the model or splitting recommendation datasets. For more information, see Split Data.
EXCEPTION MESSAGES
Number of selected columns in input dataset does not equal to the expected number.
Number of selected columns in input dataset does not equal to {0}.
Column selection pattern "{0}" provides number of selected columns in input dataset not equal to {1}.
Column selection pattern "{0}" is expected to provide {1} column(s) selected in input dataset, but {2} column(s) is/are provided.
Error 0023
Exception occurs if target column of input dataset is not valid for the current trainer module.
This error in Azure Machine Learning occurs if the target column (as selected in the module parameters) is not of
the valid data-type, contained all missing values, or was not categorical as expected.
Resolution: Revisit the module input to inspect the content of the label/target column. Make sure it does not have
all missing values. If the module is expecting target column to be categorical, make sure that there are more than
one distinct values in the target column.
EXCEPTION MESSAGES
Input dataset has unsupported target column.
Input dataset has unsupported target column "{0}".
Input dataset has unsupported target column "{0}" for learner of type {1}.
Error 0024
Exception occurs if dataset does not contain a label column.
This error in Azure Machine Learning occurs when the module requires a label column and the dataset does not
have a label column. For example, evaluation of a scored dataset usually requires that a label column is present to
compute accuracy metrics.
It can also happen that a label column is present in the dataset, but not detected correctly by Azure Machine
Learning.
Resolution:
Open the module that generated the error, and determine if a label column is present. The name or data type of
the column doesn't matter, as long as the column contains a single outcome (or dependent variable) that you
are trying to predict. If you are not sure which column has the label, look for a generic name such as Class or
Target.
If the dataset does not include a label column, it is possible that the label column was explicitly or accidentally
removed upstream. It could also be that the dataset is not the output of an upstream scoring module.
To explicitly mark the column as the label column, add the Edit Metadata module and connect the dataset.
Select only the label column, and select Label from the Fields dropdown list.
If the wrong column is chosen as the label, you can select Clear label from the Fields to fix the metadata on
the column.
EXCEPTION MESSAGES
There is no label column in dataset.
There is no label column in "{0}".
Error 0025
Exception occurs if dataset does not contain a score column.
This error in Azure Machine Learning occurs if the input to the evaluate model does not contain valid score
columns. For example, the user attempts to evaluate a dataset before it was scored with a correct trained model, or
the score column was explicitly dropped upstream. This exception also occurs if the score columns on the two
datasets are incompatible. For example, you might be trying to compare the accuracy of a linear regressor with
that of a binary classifier.
Resolution: Revisit the input to the evaluate model and examine if it contains one or more score columns. If not,
the dataset was not scored or the score columns were dropped in an upstream module.
EXCEPTION MESSAGES
There is no score column in dataset.
There is no score column in "{0}".
There is no score column in "{0}" that is produced by a "{1}". Score the dataset using the correct type of learner.
Error 0026
Exception occurs if columns with the same name are not allowed.
This error in Azure Machine Learning occurs if multiple columns have the same name. One way you may receive
this error is if the dataset does not have a header row and column names are automatically assigned: Col0, Col1,
etc.
Resolution: If columns have same name, insert a Edit Metadata module between the input dataset and the
module. Use the column selector in Edit Metadata to select columns to rename, typing the new names into the
New column names textbox.
EXCEPTION MESSAGES
Equal column names are specified in arguments. Equal column names are not allowed by module.
Equal column names in arguments "{0}" and "{1}" are not allowed. Specify different names.
Error 0027
Exception occurs in case when two objects have to be of the same size but are not.
This is an common error in Azure Machine Learning and can be caused by many conditions.
Resolution: There is no specific resolution. However, you can check for conditions such as the following:
If you are renaming columns, make sure that each list (the input columns and the list of new names) has the
same number of items.
If you are joining or concatenating two datasets, make sure they have the same schema.
If you are joining two datasets that have multiple columns, make sure that the key columns have the same
data type, and select the option Allow duplicates and preserve column order in selection.
EXCEPTION MESSAGES
The size of passed objects is inconsistent.
The size of "{0}" is inconsistent with size of "{1}".
Error 0028
Exception occurs in the case when column set contains duplicated column names and it is not allowed.
This error in Azure Machine Learning occurs when column names are duplicated; that is, not unique.
Resolution: If any columns have same name, add an instance of Edit Metadata between the input dataset and the
module raising the error. Use the Column Selector in Edit Metadata to select columns to rename, and type the new
columns names into the New column names textbox. If you are renaming multiple columns, ensure that the
values you type in the New column names are unique.
EXCEPTION MESSAGES
Column set contains duplicated column name(s).
The name "{0}" is duplicated.
The name "{0}" is duplicated in "{1}".
Error 0029
Exception occurs in case when invalid URI is passed.
This error in Azure Machine Learning occurs in case when invalid URI is passed. You will receive this error if any of
the following conditions are true:, or.
The Public or SAS URI provided for Azure Blob Storage for read or write contains an error.
The time window for the SAS has expired.
The Web URL via HTTP source represents a file or a loopback URI.
The Web URL via HTTP contains an incorrectly formatted URL.
The URL cannot be resolved by the remote source.
Resolution: Revisit the module and verify the format of the URI. If the data source is a Web URL via HTTP, verify
that the intended source is not a file or a loopback URI (localhost).
EXCEPTION MESSAGES
Invalid Uri is passed.
Error 0030
Exception occurs in the case when it is not possible to download a file.
This exception in Azure Machine Learning occurs when it is not possible to download a file. You will receive this
exception when an attempted read from an HTTP source has failed after three (3) retry attempts.
Resolution: Verify that the URI to the HTTP source is correct and that the site is currently accessible via the
Internet.
EXCEPTION MESSAGES
Unable to download a file.
Error while downloading the file: {0}.
Error 0031
Exception occurs if number of columns in column set is less than needed.
This error in Azure Machine Learning occurs if the number of columns selected is less than needed. You will
receive this error if the minimum required number of columns are not selected.
Resolution: Add additional columns to the column selection by using the Column Selector.
EXCEPTION MESSAGES
Number of columns in column set is less than required.
{0} column(s) should be specified. The actual number of specified columns is {1}.
Error 0032
Exception occurs if argument is not a number.
You will receive this error in Azure Machine Learning if the argument is a double or NaN.
Resolution: Modify the specified argument to use a valid value.
EXCEPTION MESSAGES
Argument is not a number.
"{0}" is not a number.
Error 0033
Exception occurs if argument is Infinity.
This error in Azure Machine Learning occurs if the argument is infinite. You will receive this error if the argument is
either double.NegativeInfinity or double.PositiveInfinity .
Resolution: Modify the specified argument to be a valid value.
EXCEPTION MESSAGES
Argument is must be finite.
"{0}" is not finite.
Error 0034
Exception occurs if more than one rating exists for a given user-item pair.
This error in Azure Machine Learning occurs in recommendation if a user-item pair has more than one rating
value.
Resolution: Ensure that the user-item pair possesses one rating value only.
EXCEPTION MESSAGES
More than one rating exists for the value(s) in dataset.
More than one rating for user {0} and item {1} in rating prediction data table.
Error 0035
Exception occurs if no features were provided for a given user or item.
This error in Azure Machine Learning occurs you are trying to use a recommendation model for scoring but a
feature vector cannot be found.
Resolution:
The Matchbox recommender has certain requirements that must be met when using either item features or user
features. This error indicates that a feature vector is missing for a user or item that you provided as input. You must
ensure that a vector of features is available in the data for each user or item.
For example, if you trained a recommendation model using features such as the user's age, location, or income, but
now want to create scores for new users who were not seen during training, you must provide some equivalent set
of features (namely, age, location, and income values) for the new users in order to make appropriate predictions
for them.
If you do not have any features for these users, consider feature engineering to generate appropriate features. For
example, if you do not have individual user age or income values, you might generate approximate values to use
for a group of users.
TIP
Resolution not applicable to your case? You are welcome to send feedback on this article and provide information about the
scenario, including the module and the number of rows in the column. We will use this information to provide more detailed
troubleshooting steps in the future.
EXCEPTION MESSAGES
No features were provided for a required user or item.
Features for {0} required but not provided.
Error 0036
Exception occurs if multiple feature vectors were provided for a given user or item.
This error in Azure Machine Learning occurs if a feature vector is defined more than once.
Resolution: Ensure that the feature vector is not defined more than once.
EXCEPTION MESSAGES
Duplicate feature definition for a user or item.
Duplicate feature definition for {0}.
Error 0037
Exception occurs if multiple label columns are specified and just one is allowed.
This error in Azure Machine Learning occurs if more than one column is selected to be the new label column. Most
supervised learning algorithms require a single column to be marked as the target or label.
Resolution: Make sure to select a single column as the new label column.
EXCEPTION MESSAGES
Multiple label columns are specified.
Error 0038
Exception occurs if number of elements expected should be an exact value, but is not.
This error in Azure Machine Learning occurs if the number of elements expected should be an exact value, but is
not. You will receive this error if the number of elements is not equal to the valid expected value.
Resolution: Modify the input to have the correct number of elements.
EXCEPTION MESSAGES
Number of elements is not valid.
Number of elements in "{0}" is not valid.
Number of elements in "{0}" is not equal to valid number of {1} element(s).
Error 0039
Exception occurs if an operation has failed.
This error in Azure Machine Learning occurs when an internal operation cannot be completed.
Resolution: This error is caused by many conditions and there is no specific remedy.
The following table contains generic messages for this error, which are followed by a specific description of the
condition.
If no details are available, send feedback and provide information about the modules that generated the error and
related conditions.
EXCEPTION MESSAGES
Operation failed.
Error while completing operation: {0}.
Error 0040
Exception occurs when calling a deprecated module.
This error in Azure Machine Learning is produced when calling a deprecated module.
Resolution: Replace the deprecated module with a supported one. See the module output log for the information
about which module to use instead.
EXCEPTION MESSAGES
Accessing deprecated module.

EXCEPTION MESSAGES
Module "{0}" is deprecated. Use module "{1}" instead.
Error 0041
Exception occurs when calling a deprecated module.
This error in Azure Machine Learning is produced when calling a deprecated module.
Resolution: Replace the deprecated module with a set of supported ones. This information should be visible in the
module output log.
EXCEPTION MESSAGES
Module "{0}" is deprecated. Use the modules "{1}" for requested functionality.
Error 0042
Exception occurs when it is not possible to convert column to another type.
This error in Azure Machine Learning occurs when it is not possible to convert column to the specified type. You
will receive this error if a module requires a particular data type, such as datetime, text, a floating point number, or
integer, but it is not possible to convert an existing column to the required type.
For example, you might select a column and try to convert it to a numeric data type for use in a math operation,
and get this error if the column contained invalid data.
Another reason you might get this error if you try to use a column containing floating point numbers or many
unique values as a categorical column.
Resolution:
Open the help page for the module that generated the error, and verify the data type requirements.
Review the data types of the columns in the input dataset.
Inspect data originating in so-called schema-less data sources.
Check the dataset for missing values or special characters that might block conversion to the desired data type.
Numeric data types should be consistent: for example, check for floating point numbers in a column of
integers.
Look for text strings or NA values in a number column.
Boolean values can be converted to an appropriate representation depending on the required data type.
Examine text columns for non-unicode characters, tab characters, or control characters
Datetime data should be consistent to avoid modeling errors, but cleanup can be complex owing to the
many formats. Consider using Execute Python Script modules to perform cleanup.
If necessary, modify the values in the input dataset so that the column can be converted successfully.
Modification might include binning, truncation or rounding operations, elimination of outliers, or imputation of
missing values. See the following articles for some common data transformation scenarios in machine learning:
Clean Missing Data
Normalize Data
TIP
Resolution unclear, or not applicable to your case? You are welcome to send feedback on this article and provide information
about the scenario, including the module and the data type of the column. We will use this information to provide more
detailed troubleshooting steps in the future.
EXCEPTION MESSAGES
Not allowed conversion.
Could not convert column of type {0} to column of type {1}.
Could not convert column "{2}" of type {0} to column of type {1}.
Could not convert column "{2}" of type {0} to column "{3}" of type {1}.
Error 0043
Exception occurs when element type does not explicitly implement Equals.
This error in Azure Machine Learning is unused and will be deprecated.
Resolution: None.
EXCEPTION MESSAGES
No accessible explicit method Equals found.
Cannot compare values for column \"{0}\" of type {1}. No accessible explicit method Equals found.
Error 0044
Exception occurs when it is not possible to derive element type of column from the existing values.
This error in Azure Machine Learning occurs when it is not possible to infer the type of a column or columns in a
dataset. This typically happens when concatenating two or more datasets with different element types. If Azure
Machine Learning is unable to determine a common type that is able to represent all the values in a column or
columns without loss of information, it will generate this error.
Resolution: Ensure that all values in a given column in both datasets being combined are either of the same type
(numeric, Boolean, categorical, string, date, etc.) or can be coerced to the same type.
EXCEPTION MESSAGES
Cannot derive element type of the column.
Cannot derive element type for column "{0}" -- all the elements are null references.
Cannot derive element type for column "{0}" of dataset "{1}" -- all the elements are null references.
Error 0045
Exception occurs when it is not possible to create a column because of mixed element types in the source.
This error in Azure Machine Learning is produced when the element types of two datasets being combined are
different.
Resolution: Ensure that all values in a given column in both datasets being combined are of the same type
(numeric, Boolean, categorical, string, date, etc.).
EXCEPTION MESSAGES
Cannot create column with mixed element types.
Cannot create column with ID "{0}" of mixed element types:\n\tType of data[{1}, {0}] is {2}\n\tType of data[{3}, {0}] is {4}.
Error 0046
Exception occurs when it is not possible to create directory on specified path.
This error in Azure Machine Learning occurs when it is not possible to create a directory on the specified path. You
will receive this error if any part of the path to the output directory for a Hive Query is incorrect or inaccessible.
Resolution: Revisit the module and verify that the directory path is correctly formatted and that it is accessible
with the current credentials.
EXCEPTION MESSAGES
Specify a valid output directory.
Directory: {0} cannot be created. Specify valid path.
Error 0047
Exception occurs if number of feature columns in some of the datasets passed to the module is too small.
This error in Azure Machine Learning occurs if the input dataset to training does not contain the minimum number
of columns required by the algorithm. Typically either the dataset is empty or only contains training columns.
Resolution: Revisit the input dataset to make sure there one or more additional columns apart from the label
column.
EXCEPTION MESSAGES
Number of feature columns in input dataset is less than allowed minimum.
Number of feature columns in input dataset is less than allowed minimum of {0} column(s).
Number of feature columns in input dataset "{0}" is less than allowed minimum of {1} column(s).
Error 0048
Exception occurs in the case when it is not possible to open a file.
This error in Azure Machine Learning occurs when it is not possible to open a file for read or write. You might
receive this error for these reasons:
The container or the file (blob) does not exist
The access level of the file or container does not allow you to access the file
The file is too large to read or the wrong format
Resolution: Revisit the module and the file you are trying to read.
Verify that the names of the container and file are correct.
Use the Azure classic portal or an Azure storage tool to verify that you have permission to access the file.
EXCEPTION MESSAGES
Unable to open a file.
Error while opening the file: {0}.
Error 0049
Exception occurs in the case when it is not possible to parse a file.
This error in Azure Machine Learning occurs when it is not possible to parse a file. You will receive this error if the
file format selected in the Import Data module does not match the actual format of the file, or if the file contains an
unrecognizable character.
Resolution: Revisit the module and correct the file format selection if it does not match the format of the file. If
possible, inspect the file to confirm that it does not contain any illegal characters.
EXCEPTION MESSAGES
Unable to parse a file.
Error while parsing the file: {0}.
Error 0050
Exception occurs in the case when input and output files are the same.
Resolution: This error in Azure Machine Learning is unused and will be deprecated.
EXCEPTION MESSAGES
Specified files for input and output cannot be the same.
Error 0051
Exception occurs in the case when several output files are the same.
EXCEPTION MESSAGES
Specified files for outputs cannot be the same.
Error 0052
Exception occurs if Azure storage account key is specified incorrectly.
This error in Azure Machine Learning occurs if the key used to access the Azure storage account is incorrect. For
example, you might see this error if the Azure storage key was truncated when copied and pasted, or if the wrong
key was used.
For more information about how to get the key for an Azure storage account, see View, copy, and regenerate
storage access keys.
Resolution: Revisit the module and verify that the Azure storage key is correct for the account; copy the key again
from the Azure classic portal if necessary.
EXCEPTION MESSAGES
The Azure storage account key is incorrect.
Error 0053
Exception occurs in the case when there are no user features or items for matchbox recommendations.
This error in Azure Machine Learning is produced when a feature vector cannot be found.
Resolution: Ensure that a feature vector is present in the input dataset.
EXCEPTION MESSAGES
User features or/and items are required but not provided.
Error 0054
Exception occurs if there is too few distinct values in the column to complete operation.
EXCEPTION MESSAGES
Data has too few distinct values in the specified column to complete operation.
Data has too few distinct values in the specified column to complete operation. The required minimum is {0} elements.
Data has too few distinct values in the column "{1}" to complete operation. The required minimum is {0} elements.
Error 0055
Exception occurs when calling a deprecated module. This error in Azure Machine Learning appears if you try to call
a module that has been deprecated.
Resolution:
EXCEPTION MESSAGES
Module "{0}" is deprecated.

Error 0056
Exception occurs if the columns you selected for an operation violates requirements.
This error in Azure Machine Learning occurs when you are choosing columns for an operation that requires the
column be of a particular data type.
This error can also happen if the column is the correct data type, but the module you are using requires that the
column also be marked as a feature, label, or categorical column.
Resolution:
1. Review the data type of the columns that are currently selected.
2. Ascertain whether the selected columns are categorical, label, or feature columns.
3. Review the help topic for the module in which you made the column selection, to determine if there are
specific requirements for data type or column usage.
4. Use Edit Metadata to change the column type for the duration of this operation. Be sure to change the
column type back to its original value, using another instance of Edit Metadata, if you need it for
downstream operations.
EXCEPTION MESSAGES
One or more selected columns were not in an allowed category.
Column with name "{0}" is not in an allowed category.
Error 0057
Exception occurs when attempting to create a file or blob that already exists.
This exception occurs when you are using the Export Data module or other module to save results of an
experiment in Azure Machine Learning to Azure blob storage, but you attempt to create a file or blob that already
exists.
Resolution:
You will receive this error only if you previously set the property Azure blob storage write mode to Error. By
design, this module raises an error if you attempt to write a dataset to a blob that already exists.
Open the module properties and change the property Azure blob storage write mode to Overwrite.
Alternatively, you can type the name of a different destination blob or file and be sure to specify a blob that
does not already exist.
EXCEPTION MESSAGES
File or Blob already exists.
File or Blob "{0}" already exists.
Error 0058
This error in Azure Machine Learning occurs if the dataset does not contain the expected label column.
This exception can also occur when the label column provided does not match the data or datatype expected by the
learner, or has the wrong values. For example, this exception is produced when using a real-valued label column
when training a binary classifier.
Resolution: The resolution depends on the learner or trainer that you are using, and the data types of the columns
in your dataset. First, verify the requirements of the machine learning algorithm or training module.
Revisit the input dataset. Verify that the column you expect to be treated as the label has the right data type for the
model you are creating.
Check inputs for missing values and eliminate or replace them if necessary.
If necessary, add the Edit Metadata module and ensure that the label column is marked as a label.
EXCEPTION MESSAGES
The label column is not as expected
The label column is not as expected in "{0}".
The label column "{0}" is not expected in "{1}".
Error 0059
Exception occurs if a column index specified in a column picker cannot be parsed.
This error in Azure Machine Learning occurs if a column index specified when using the Column Selector cannot
be parsed. You will receive this error when the column index is in an invalid format that cannot be parsed.
Resolution: Modify the column index to use a valid index value.
EXCEPTION MESSAGES
One or more specified column indexes or index ranges could not be parsed.
Column index or range "{0}" could not be parsed.
Error 0060
Exception occurs when an out of range column range is specified in a column picker.
This error in Azure Machine Learning occurs when an out-of-range column range is specified in the Column
Selector. You will receive this error if the column range in the column picker does not correspond to the columns in
the dataset.
Resolution: Modify the column range in the column picker to correspond to the columns in the dataset.
EXCEPTION MESSAGES
Invalid or out of range column index range specified.
Column range "{0}" is invalid or out of range.
Error 0061
Exception occurs when attempting to add a row to a DataTable that has a different number of columns than the
table.
This error in Azure Machine Learning occurs when you attempt to add a row to a dataset that has a different
number of columns than the dataset. You will receive this error if the row that is being added to the dataset has a
different number of columns from the input dataset. The row cannot be appended to the dataset if the number of
columns is different.
Resolution: Modify the input dataset to have the same number of columns as the row added, or modify the row
added to have the same number of columns as the dataset.
EXCEPTION MESSAGES
All tables must have the same number of columns.
Error 0062
Exception occurs when attempting to compare two models with different learner types.
This error in Azure Machine Learning is produced when evaluation metrics for two different scored datasets
cannot be compared. In this case, it is not possible to compare the effectiveness of the models used to produce the
two scored datasets.
Resolution: Verify that the scored results are produced by the same kind of machine learning model (binary
classification, regression, multi-class classification, recommendation, clustering, anomaly detection, etc.) All models
that you compare must have the same learner type.
EXCEPTION MESSAGES
All models must have the same learner type.
EXCEPTION MESSAGES
Error during evaluation of R script.
The following error occurred during evaluation of R script: ---------- Start of error message from R ---------- {0} ----------- End
of error message from R -----------
During the evaluation of R script "{1}" the following error occurred: ---------- Start of error message from R ---------- {0} --------
--- End of error message from R -----------
Error 0064
Exception occurs if Azure storage account name or storage key is specified incorrectly.
This error in Azure Machine Learning occurs if the Azure storage account name or storage key is specified
incorrectly. You will receive this error if you enter an incorrect account name or password for the storage account.
This may occur if you manually enter the account name or password. It may also occur if the account has been
deleted.
Resolution: Verify that the account name and password have been entered correctly, and that the account exists.
EXCEPTION MESSAGES
The Azure storage account name or storage key is incorrect.

EXCEPTION MESSAGES
The Azure storage account name "{0}" or storage key for the account name is incorrect.
Error 0065
Exception occurs if Azure blob name is specified incorrectly.
This error in Azure Machine Learning occurs if the Azure blob name is specified incorrectly. You will receive the
error if:
The blob cannot be found in the specified container.
Only the container was specified as the source in a Import Data request when the format was Excel or CSV
with encoding; concatenation of the contents of all blobs within a container is not allowed with these
formats.
A SAS URI does not contain the name of a valid blob.
Resolution: Revisit the module throwing the exception. Verify that the specified blob does exist in the container in
the storage account and that permissions allow you to see the blob. Verify that the input is of the form
containername/filename if you have Excel or CSV with encoding formats. Verify that a SAS URI contains the
name of a valid blob.
EXCEPTION MESSAGES
The Azure storage blob is incorrect.
The Azure storage blob name "{0}" is incorrect
Error 0066
Exception occurs if a resource could not be uploaded to an Azure Blob.
This error in Azure Machine Learning occurs if a resource could not be uploaded to an Azure Blob. Both are saved
to the same Azure storage account as the account containing the input file.
Resolution: Revisit the module. Verify that the Azure account name, storage key, and container are correct and
that the account has permission to write to the container.
EXCEPTION MESSAGES
The resource could not be uploaded to Azure storage.
The file "{0}" could not be uploaded to Azure storage as {1}.
Error 0067
Exception occurs if a dataset has a different number of columns than expected.
This error in Azure Machine Learning occurs if a dataset has a different number of columns than expected. You will
receive this error when the number of columns in the dataset are different from the number of columns that the
module expects during execution.
Resolution: Modify the input dataset or the parameters.
EXCEPTION MESSAGES
Unexpected number of columns in the datatable.
Expected "{0}" columns but found "{1}" columns instead.
Error 0068
Exception occurs if the specified Hive script is not correct.
This error in Azure Machine Learning occurs if there are syntax errors in a Hive QL script, or if the Hive interpreter
encounters an error while executing the query or script.
Resolution:
The error message from Hive is normally reported back in the Error Log so that you can take action based on the
specific error.
Open the module and inspect the query for mistakes.
Verify that the query works correctly outside of Azure Machine Learning by logging in to the Hive console of
your Hadoop cluster and running the query.
Try placing comments in your Hive script in a separate line as opposed to mixing executable statements and
comments in a single line.
Resources
See the following articles for help with Hive queries for machine learning:
Create Hive tables and load data from Azure Blob Storage
Explore data in tables with Hive queries
Create features for data in an Hadoop cluster using Hive queries
Hive for SQL Users Cheat Sheet (PDF )
EXCEPTION MESSAGES
Hive script is incorrect.
Hive script {0} is not correct.
Error 0069
Exception occurs if the specified SQL script is not correct.
This error in Azure Machine Learning occurs if the specified SQL script has syntax problems, or if the columns or
table specified in the script is not valid.
You will receive this error if the SQL engine encounters any error while executing the query or script. The SQL
error message is normally reported back in the Error Log so that you can take action based on the specific error.
Resolution: Revisit the module and inspect the SQL query for mistakes.
Verify that the query works correctly outside of Azure ML by logging in to the database server directly and
running the query.
If there is a SQL generated message reported by the module exception, take action based on the reported error.
For example, the error messages sometimes include specific guidance on the likely error:
No such column or missing database, indicating that you might have typed a column name wrong. If you are
sure the column name is correct, try using brackets or quotation marks to enclose the column identifier.
SQL logic error near <SQL keyword>, indicating that you might have a syntax error before the specified
keyword
EXCEPTION MESSAGES
SQL script is incorrect.
SQL query "{0}" is not correct.
SQL query "{0}" is not correct: {1}
Error 0070
Exception occurs when attempting to access non-existent Azure table.
This error in Azure Machine Learning occurs when you attempt to access a non-existent Azure table. You will
receive this error if you specify a table in Azure storage, which does not exist when reading from or writing to
Azure Table Storage. This can happen if you mistype the name of the desired table, or you have a mismatch
between the target name and the storage type. For example, you intended to read from a table but entered the
name of a blob instead.
Resolution: Revisit the module to verify that the name of the table is correct.
EXCEPTION MESSAGES
Azure table does not exist.
Azure table "{0}" does not exist.
Error 0071
Exception occurs if provided credentials are incorrect.
This error in Azure Machine Learning occurs if the provided credentials are incorrect.
You might also receive this error if the module cannot connect to an HDInsight cluster.
Resolution: Review the inputs to the module and verify the account name and password.
Check for the following issues that can cause an error:
The schema of the dataset does not match the schema of the destination datatable.
Column names are missing or misspelled
You are writing to a table that has column names with illegal characters. Ordinarily you can enclose such
column names in square brackets, but if that does not work, edit column names to use only letters and
underscores (_)
Strings that you are trying to write contain single quotation marks
If you are trying to connect to an HDInsight cluster, verify that the target cluster is accessible with the supplied
credentials.
EXCEPTION MESSAGES
Incorrect credentials are passed.
Incorrect username "{0}" or password is passed
Error 0072
Exception occurs in the case of connection timeout.
This error in Azure Machine Learning occurs when a connection times out. You will receive this error if there are
currently connectivity issues with the data source or destination, such as slow internet connectivity, or if the dataset
is large and/or the SQL query to read in the data performs complicated processing.
Resolution: Determine whether there are currently issues with slow connections to Azure storage or the internet.
EXCEPTION MESSAGES
Connection timeout occurred.
Error 0073
Exception occurs if an error occurs while converting a column to another type.
This error in Azure Machine Learning occurs when it is not possible to convert column to another type. You will
receive this error if a module requires a particular type and it is not possible to convert the column to the new type.
Resolution: Modify the input dataset so that the column can be converted based on the inner exception.
EXCEPTION MESSAGES
Failed to convert column.
Failed to convert column to {0}.
Error 0074
Exception occurs when the Edit Metadata tries to convert a sparse column to categorical.
This error in Azure Machine Learning occurs when the Edit Metadata tries to convert a sparse column to
categorical. You will receive this error when trying to convert sparse columns to categorical with the Make
categorical option. Azure machine Learning does not support sparse categorical arrays, so the module will fail.
EXCEPTION MESSAGES
Sparse columns cannot be converted to Categorical.
Error 0075
Exception occurs when an invalid binning function is used when quantizing a dataset.
This error in Azure Machine Learning occurs when you are trying to bin data using an unsupported method, or
when the parameter combinations are invalid.
Resolution:
Error handling for this event was introduced in an earlier version of Azure Machine Learning that allowed more
customization of binning methods. Currently all binning methods are based on a selection from a dropdown list, so
technically it should no longer be possible to get this error.
EXCEPTION MESSAGES
Invalid binning function used.
Error 0077
Exception occurs when unknown blob file writes mode passed.
This error in Azure Machine Learning occurs if an invalid argument is passed in the specifications for a blob file
destination or source.
Resolution: In almost all modules that import or export data to and from Azure blob storage, parameter values
controlling the write mode are assigned by using a dropdown list; therefore, it is not possible to pass an invalid
value, and this error should not appear. This error will be deprecated in a later release.
EXCEPTION MESSAGES
Unsupported blob writes mode.
Unsupported blob writes mode: {0}.
Error 0078
Exception occurs when the HTTP option for Import Data receives a 3xx status code indicating redirection.
This error in Azure Machine Learning occurs when the HTTP option for Import Data receives a 3xx (301, 302, 304,
etc.) status code indicating redirection. You will receive this error if you attempt to connect to an HTTP source that
redirects the browser to another page. For security reasons, redirecting websites are not allowed as data sources
for Azure Machine Learning.
Resolution: If the website is a trusted website, enter the redirected URL directly.
EXCEPTION MESSAGES
Http redirection not allowed
Error 0079
Exception occurs if Azure storage container name is specified incorrectly.
This error in Azure Machine Learning occurs if the Azure storage container name is specified incorrectly. You will
receive this error if you have not specified both the container and the blob (file) name using the Path to blob
beginning with container option when writing to Azure Blob Storage.
Resolution: Revisit the Export Data module and verify that the specified path to the blob contains both the
container and the file name, in the format container/filename.
EXCEPTION MESSAGES
The Azure storage container name is incorrect.
The Azure storage container name "{0}" is incorrect; a container name of the format container/blob was expected.
Error 0080
Exception occurs when column with all values missing is not allowed by module.
This error in Azure Machine Learning is produced when one or more of the columns consumed by the module
contains all missing values. For example, if a module is computing aggregate statistics for each column, it cannot
operate on a column containing no data. In such cases, module execution is halted with this exception.
Resolution: Revisit the input dataset and remove any columns that contain all missing values.
EXCEPTION MESSAGES
Columns with all values missing are not allowed.
Column {0} has all values missing.
Error 0081
Exception occurs in PCA module if number of dimensions to reduce to is equal to number of feature columns in
input dataset, containing at least one sparse feature column.
This error in Azure Machine Learning is produced if the following conditions are met: (a) the input dataset has at
least one sparse column and (b) the final number of dimensions requested is the same as the number of input
dimensions.
Resolution: Consider reducing the number of dimensions in the output to be fewer than the number of
dimensions in the input. This is typical in applications of PCA.
EXCEPTION MESSAGES
For dataset containing sparse feature columns number of dimensions to reduce should be less than number of feature columns.
Error 0082
Exception occurs when a model cannot be successfully deserialized.
This error in Azure Machine Learning occurs when a saved machine learning model or transform cannot be loaded
by a newer version of the Azure Machine Learning runtime as a result of a breaking change.
Resolution: The training experiment that produced the model or transform must be rerun and the model or
transform must be resaved.
EXCEPTION MESSAGES
Model could not be deserialized because it is likely serialized with an older serialization format. Retrain and resave the model.
Error 0083
Exception occurs if dataset used for training cannot be used for concrete type of learner.
This error in Azure Machine Learning is produced when the dataset is incompatible with the learner being trained.
For example, the dataset might contain at least one missing value in each row, and as a result, the entire dataset
would be skipped during training. In other cases, some machine learning algorithms such as anomaly detection do
not expect labels to be present and can throw this exception if labels are present in the dataset.
Resolution: Consult the documentation of the learner being used to check requirements for the input dataset.
Examine the columns to see all required columns are present.
EXCEPTION MESSAGES
Dataset used for training is invalid.
{0} contains invalid data for training.
{0} contains invalid data for training. Learner type: {1}.
Error 0084
Exception occurs when scores produced from an R Script are evaluated. This is currently unsupported.
This error in Azure Machine Learning occurs if you try to use one of the modules for evaluating a model with
output from an R script that contains scores.
Resolution:
EXCEPTION MESSAGES
Evaluating scores produced by R is currently unsupported.
Error 0085
Exception occurs when script evaluation fails with an error.
This error in Azure Machine Learning occurs when you are running custom script that contains syntax errors.
Resolution: Review your code in an external editor and check for errors.
EXCEPTION MESSAGES
Error during evaluation of script.
The following error occurred during script evaluation, view the output log for more information: ---------- Start of error message
from {0} interpreter ---------- {1} ---------- End of error message from {0} interpreter ----------
Error 0086
Exception occurs when a counting transform is invalid.
This error in Azure Machine Learning occurs when you select a transformation based on a count table, but the
selected transform is incompatible with the current data, or with the new count table.
Resolution: The module supports saving the counts and rules that make up the transformation in two different
formats. If you are merging count tables, verify that both tables you intend to merge use the same format.
In general, a count-based transform can only be applied to datasets that have the same schema as the dataset on
which the transform was originally created.
EXCEPTION MESSAGES
Invalid counting transform specified.
The counting transform at input port '{0}' is invalid.
The counting transform at input port '{0}' cannot be merged with the counting transform at input port '{1}'. Check to verify the
metadata used for counting matches.
Error 0087
Exception occurs when an invalid count table type is specified for learning with counts modules.
This error in Azure Machine Learning occurs when you try to import an existing count table, but the table is
incompatible with the current data, or with the new count table.
Resolution: There are different formats for saving the counts and rules that make up the transformation. If you
are merging count tables, verify that both use the same format.
Generally, a count-based transform can only be applied to datasets that have the same schema as the dataset on
which the transform was originally created.
Error 0088
Exception occurs when an invalid counting type is specified for learning with counts modules.
This error in Azure Machine Learning occurs when you try to use a different counting method than is supported
for count-based featurization.
Resolution: In general, counting methods are chosen from a dropdown list, so you should not see this error.
EXCEPTION MESSAGES
Invalid counting type is specified.
The specified counting type '{0}' is not a valid counting type.
Error 0089
Exception occurs when the specified number of classes is less than the actual number of classes in a dataset used
for counting.
This error in Azure Machine Learning occurs when you are creating a count table and the label column contains a
different number of classes than you specified in the module parameters.
Resolution: Check your dataset and find out exactly how many distinct values (possible classes) there are in the
label column. When you create the count table, you must specify at least this number of classes.
The count table cannot automatically determine the number of classes available.
When you create the count table, you cannot specify 0 or any number that is less than the actual number of classes
in the label column.
EXCEPTION MESSAGES
The number of classes is incorrect. Make sure that the number of classes you specify in the parameter pane is greater than or
equal to the number of classes in the label column.
The number of classes specified is '{0}', which is not greater than a label value '{1}' in the data set used to count. Make sure that
the number of classes you specify in the parameter pane is greater than or equal to the number of classes in the label column.
Error 0090
Exception occurs when Hive table creation fails.
This error in Azure Machine Learning occurs when you are using Export Data or another option to save data to an
HDInsight cluster and the specified Hive table cannot be created.
Resolution: Check the Azure storage account name associated with the cluster and verify that you are using the
same account in the module properties.
EXCEPTION MESSAGES
The Hive table could not be created. For a HDInsight cluster, ensure the Azure storage account name associated with cluster is
the same as what is passed in through the module parameter.
The Hive table "{0}" could not be created. For a HDInsight cluster, ensure the Azure storage account name associated with cluster
is the same as what is passed in through the module parameter.
The Hive table "{0}" could not be created. For a HDInsight cluster, ensure the Azure storage account name associated with cluster
is "{1}".
Error 0100
Exception occurs when an unsupported language is specified for a custom module.
This error in Azure Machine Learning occurs when building a custom module and the name property of the
Language element in a custom module xml definition file has an invalid value. Currently, the only valid value for
this property is R . For example:
<Language name="R" sourceFile="CustomAddRows.R" entryPoint="CustomAddRows" />
Resolution: Verify that the name property of the Language element in the custom module xml definition file is
set to R . Save the file, update the custom module zip package, and try to add the custom module again.
EXCEPTION MESSAGES
Unsupported custom module language specified
Error 0101
All port and parameter IDs must be unique.
This error in Azure Machine Learning occurs when one or more ports or parameters are assigned the same ID
value in a custom module XML definition file.
Resolution: Check that the ID values across all ports and parameters are unique. Save the xml file, update the
custom module zip package, and try to add the custom module again.
EXCEPTION MESSAGES
All port and parameter IDs for a module must be unique
Module '{0}' has duplicate port/argument IDs. All port/argument IDs must be unique for a module.
Error 0102
Thrown when a ZIP file cannot be extracted.
This error in Azure Machine Learning occurs when you are importing a zipped package with the .zip extension, but
the package is either not a zip file, or the file does not use a supported zip format.
Resolution: Make sure the selected file is a valid .zip file, and that it was compressed by using one of the
supported compression algorithms.
If you get this error when importing datasets in compressed format, verify that all contained files use one of the
supported file formats, and are in Unicode format.
Try readding the desired files to a new compressed zipped folder and try to add the custom module again.
EXCEPTION MESSAGES
Given ZIP file is not in the correct format
Error 0103
Thrown when a ZIP file does not contain any .xml files
This error in Azure Machine Learning occurs when the custom module zip package does not contain any module
definition (.xml) files. These files need to reside in the root of the zip package (for example, not within a subfolder.)
Resolution: Verify that one or more xml module definition files are in the root folder of the zip package by
extracting it to a temporary folder on your disk drive. Any xml files should be directly in the folder you extracted
the zip package to. Make sure when you create the zip package that you do not select a folder that contains xml
files to zip as this will create a sub folder within the zip package with the same name as the folder you selected to
zip.
EXCEPTION MESSAGES
Given ZIP file does not contain any module definition files (.xml files)
Error 0104
Thrown when a module definition file references a script that cannot be located
This error in Azure Machine Learning is thrown when a custom module xml definition file references a script file in
the Language element that does not exist in the zip package. The script file path is defined in the sourceFile
property of the Language element. The path to the source file is relative to the root of the zip package (same
location as the module xml definition files). If the script file is in a sub folder, the relative path to the script file must
be specified. For instance, if all scripts were stored in a myScripts folder within the zip package, the Language
element would have to add this path to the sourceFile property as below. For example:
<Language name="R" sourceFile="myScripts/CustomAddRows.R" entryPoint="CustomAddRows" />
Resolution: Make sure that the value of the sourceFile property in the Language element of the custom module
xml definition is correct and that the source file exists in the correct relative path in the zip package.
EXCEPTION MESSAGES
Referenced R script file does not exist.
Referenced R script file '{0}' cannot be found. Ensure that the relative path to the file is correct from the definitions location.
Error 0105
This error is displayed when a module definition file contains an unsupported parameter type
This error in Azure Machine Learning is produced when the you create a custom module xml definition and the
type of a parameter or argument in the definition does not match a supported type.
Resolution: Make sure that the type property of any Arg element in the custom module xml definition file is a
supported type.
EXCEPTION MESSAGES
Unsupported parameter type.
Unsupported parameter type '{0}' specified.
Error 0106
Thrown when a module definition file defines an unsupported input type
This error in Azure Machine Learning is produced when the type of an input port in a custom module XML
definition does not match a supported type.
Resolution: Make sure that the type property of an Input element in the custom module XML definition file is a
supported type.
EXCEPTION MESSAGES
Unsupported input type.
Unsupported input type '{0}' specified.
Error 0107
Thrown when a module definition file defines an unsupported output type
This error in Azure Machine Learning is produced when the type of an output port in a custom module xml
definition does not match a supported type.
Resolution: Make sure that the type property of an Output element in the custom module xml definition file is a
supported type.
EXCEPTION MESSAGES
Unsupported output type.

EXCEPTION MESSAGES
Unsupported output type '{0}' specified.
Error 0108
Thrown when a module definition file defines more input or output ports than are supported
This error in Azure Machine Learning is produced when too many input or output ports are defined in a custom
module xml definition.
Resolution: Makes sure the maximum number of input and output ports defined in the custom module xml
definition does not exceed the maximum number of supported ports.
EXCEPTION MESSAGES
Exceeded supported number of input or output ports.
Exceeded number of supported '{0}' ports. Maximum allowed number of '{0}' ports is '{1}'.
Error 0109
Thrown when a module definition file defines a column picker incorrectly
This error in Azure Machine Learning is produced when the syntax for a column picker argument contains an error
in a custom module xml definition.
Resolution: This error is produced when the syntax for a column picker argument contains an error in a custom
module xml definition.
EXCEPTION MESSAGES
Unsupported syntax for column picker.
Error 0110
Thrown when a module definition file defines a column picker that references a non-existent input port ID
This error in Azure Machine Learning is produced when the portId property within the Properties element of an
Arg of type ColumnPicker does not match the ID value of an input port.
Resolution: Make sure the portId property matches the ID value of an input port defined in the custom module
xml definition.
EXCEPTION MESSAGES
Column picker references a non-existent input port ID.
Column picker references a non-existent input port ID '{0}'.
Error 0111
Thrown when a module definition file defines an invalid property
This error in Azure Machine Learning is produced when an invalid property is assigned to an element in the
custom module XML definition.
Resolution: Make sure the property is supported by the custom module element.
EXCEPTION MESSAGES
Property definition is invalid.
Property definition '{0}' is invalid.
Error 0112
Thrown when a module definition file cannot be parsed
This error in Azure Machine Learning is produced when there is an error in the xml format that prevents the
custom module XML definition from being parsed as a valid XML file.
Resolution: Ensure that each element is opened and closed correctly. Make sure that there are no errors in the
XML formatting.
EXCEPTION MESSAGES
Unable to parse module definition file.
Unable to parse module definition file '{0}'.
Error 0113
Thrown when a module definition file contains errors.
This error in Azure Machine Learning is produced when the custom module XML definition file can be parsed but
contains errors, such as definition of elements not supported by custom modules.
Resolution: Make sure the custom module definition file defines elements and properties that are supported by
custom modules.
EXCEPTION MESSAGES
Module definition file contains errors.
Module definition file '{0}' contains errors.
Module definition file '{0}' contains errors. {1}
Error 0114
Thrown when building a custom module fails.
This error in Azure Machine Learning is produced when a custom module build fails. This occurs when one or
more custom module-related errors are encountered while adding the custom module. The additional errors are
reported within this error message.
Resolution: Resolve the errors reported within this exception message.
EXCEPTION MESSAGES
Failed to build custom module.
Custom module builds failed with error(s): {0}
Error 0115
Thrown when a custom module default script has an unsupported extension.
This error in Azure Machine Learning occurs when you provide a script for a custom module that uses an
unknown filename extension.
Resolution: Verify the file format and filename extension of any script files included in the custom module.
EXCEPTION MESSAGES
Unsupported extention for default script.
Unsupported file extention {0} for default script.
Error 0121
Thrown when SQL writes fails because the table is unwriteable
This error in Azure Machine Learning is produced when you are using the Export Data module to save results to a
table in a SQL database, and the table cannot be written to. Typically, you will see this error if the Export Data
module successfully establishes a connection with the SQL Server instance, but is then unable to write the contents
of the Azure ML dataset to the table.
Resolution:
Open the Properties pane of the Export Data module and verify that the database and table names are entered
correctly.
Review the schema of the dataset you are exporting, and make sure that the data is compatible with the
destination table.
Verify that the SQL sign in associated with the user name and password has permissions to write to the table.
If the exception contains additional error information from SQL Server, use that information to make
corrections.
EXCEPTION MESSAGES
Connected to server, unable to write to table.
Unable to write to Sql table: {0}
Error 0122
Exception occurs if multiple weight columns are specified and just one is allowed.
This error in Azure Machine Learning occurs when too many columns have been selected as weight columns.
Resolution: Review the input dataset and its metadata. Ensure that only one column contains weights.
EXCEPTION MESSAGES
Multiple weight columns are specified.
Error 0123
Exception occurs if column of vectors is specified to Label column.
This error in Azure Machine Learning occurs if you use a vector as the label column.
Resolution: Change the data format of the column if necessary, or choose a different column.
EXCEPTION MESSAGES
Column of vectors is specified as Label column.
Error 0124
Exception occurs if non-numeric columns are specified to be the weight column.
Resolution:
EXCEPTION MESSAGES
Non-numeric column is specified as the weight column.
Error 0125
Thrown when schema for multiple datasets does not match.
Resolution:
EXCEPTION MESSAGES
Dataset schema does not match.
Error 0126
Exception occurs if the user specifies a SQL domain that is not supported in Azure ML.
This error is produced when the user specifies a SQL domain that is not supported in Azure Machine Learning.
You will receive this error if you are attempting to connect to a database server in a domain that is not whitelisted.
Currently, the allowed SQL domains are: ".database.windows.net", ".cloudapp.net", or
".database.secure.windows.net". That is, the server must be an Azure SQL server or a server in a virtual machine
on Azure.
Resolution: Revisit the module. Verify that the SQL database server belongs to one of the accepted domains:
.database.windows.net
.cloudapp.net
.database.secure.windows.net
EXCEPTION MESSAGES
Unsupported SQL domain.
The SQL domain {0} is not currently supported in Azure ML
Error 0127
Image pixel size exceeds allowed limit
This error occurs if you are reading images from an image dataset for classification and the images are larger than
the model can handle.
EXCEPTION MESSAGES
Image pixel size exceeds allowed limit.
Image pixel size in the file '{0}' exceeds allowed limit: '{1}'
Error 0128
Number of conditional probabilities for categorical columns exceeds limit.
Resolution:
EXCEPTION MESSAGES
Number of conditional probabilities for categorical columns exceeds limit.
Number of conditional probabilities for categorical columns exceeds limit. Columns '{0}' and '{1}' are the problematic pair.
Error 0129
Number of columns in the dataset exceeds allowed limit.
Resolution:
EXCEPTION MESSAGES
Number of columns in the dataset exceeds allowed limit.
Number of columns in the dataset in '{0}' exceeds allowed.'
Number of columns in the dataset in '{0}' exceeds allowed limit of '{1}'.'
Number of columns in the dataset in '{0}' exceeds allowed '{1}' limit of '{2}'.'
Error 0130
Exception occurs when all rows in the training dataset contain missing values.
This occurs when some column in the training dataset is empty.
Resolution: Use the Clean Missing Data module to remove columns with all missing values.
EXCEPTION MESSAGES
All rows in training dataset contain missing values. Consider using the Clean Missing Data module to remove missing values.
Error 0131
Exception occurs if one or more datasets in a zip file fails to be unzipped and registered correctly
This error is produced when one or more datasets in a zip file fails to be unzipped and read correctly. You will
receive this error if the unpacking fails because the zip file itself or one of the files in it is corrupt, or there is a
system error while trying to unpack and expand a file.
Resolution: Use the details provided in the error message to determine how to proceed.
EXCEPTION MESSAGES
Upload zipped datasets failed
Zipped dataset {0} failed with the following message: {1}
Zipped dataset {0} failed with a {1} exception with message: {2}
Error 0132
No file name was specified for unpacking; multiple files were found in zip file.
This error is produced when no file name was specified for unpacking; multiple files were found in zip file. You will
receive this error if the .zip file contains more than one compressed file, but you did not specify a file for extraction
in the Dataset to Unpack text box, in the Property pane of the module. Currently, only one file can be extracted
each time the module is run.
Resolution: The error message provides a list of the files found in the .zip file. Copy the name of the desired file
and paste it into the Dataset to Unpack text box.
EXCEPTION MESSAGES
Zip file contains multiple files; you must specify the file to expand.
The file contains more than one file. Specify the file to expand. The following files were found: {0}
Error 0133
The specified file was not found in the zip file
This error is produced when the filename entered in the Dataset to Unpack field of the Property pane does not
match the name of any file found in the .zip file. The most common causes of this error are a typing error or
searching the wrong archive file for the file to expand.
Resolution: Revisit the module. If the name of the file you intended to decompress appears in the list of files
found, copy the file name and paste it into the Dataset to Unpack property box. If you do not see the desired file
name in the list, verify that you have the correct .zip file and the correct name for the desired file.
EXCEPTION MESSAGES
The specified file was not found int the zip file.
The specified file was not found. Found the following file(s): {0}
Error 0134
Exception occurs when label column is missing or has insufficient number of labeled rows.
This error occurs when the module requires a label column, but you did not include one in the column selection, or
the label column is missing too many values.
This error can also occur when a previous operation changes the dataset such that insufficient rows are available to
a downstream operation. For example, suppose you use an expression in the Partition and Sample module to
divide a dataset by values. If no matches are found for your expression, one of the datasets resulting from the
partition would be empty.
Resolution:
If you include a label column in the column selection but it isn’t recognized, use the Edit Metadata module to mark
it as a label column.
Then, you can use the [Clean Missing Data](clean-missing-data.md) module to remove rows with missing values in
the label column.
Check your input datasets to make sure that they contain valid data, and enough rows to satisfy the requirements
of the operation. Many algorithms will generate an error message if they require some minimum number rows of
data, but the data contains only a few rows, or only a header.
EXCEPTION MESSAGES
Exception occurs when label column is missing or has insufficient number of labeled rows.
Exception occurs when label column is missing or has less than {0} labeled rows
Error 0135
Only centroid-based cluster is supported.
Resolution: You might encounter this error message if you have attempted to evaluate a clustering model that is
based on a custom clustering algorithm that does not use centroids to initialize the cluster.
EXCEPTION MESSAGES
Only centroid-based cluster is supported.
Error 0136
No file name was returned; unable to process the file as a result.
Resolution:
EXCEPTION MESSAGES
No file name was returned; unable to process the file as a result.
Error 0137
Azure Storage SDK encountered an error converting between table properties and dataset columns during read or
write.
Resolution:
EXCEPTION MESSAGES
Conversion error between Azure table storage property and dataset column.
Conversion error between Azure table storage property and dataset column. Additional information: {0}
Error 0138
Memory has been exhausted, unable to complete running of module. Downsampling the dataset may help to
alleviate the problem.
This error occurs when the module that is running requires more memory than is available in the Azure container.
This can happen if you are working with a large dataset and the current operation cannot fit into memory.
Resolution: If you are trying to read a large dataset and the operation cannot be completed, downsampling the
dataset might help.
EXCEPTION MESSAGES
Memory has been exhausted, unable to complete running of module.
Error 0139
Exception occurs when it is not possible to convert a column to another type.
This error in Azure Machine Learning occurs when you try to convert a column to a different data type, but that
type is not supported by the current operation or by the module.
The error might also appear when a module tries to implicitly convert data to meet requirements of the current
module, but the conversion is not possible.
Resolution:
1. Review your input data and determine the exact data type of the column that you want to use, and the data type
of the column that is producing the error. Sometimes you might think the data type is correct, but find that an
upstream operation has modified the data type or usage of a column. Use the Edit Metadata module to reset
column metadata to its original state.
2. Look at the module help page to verify the requirements for the specified operation. Determine which data
types are supported by the current module, and what range of values is supported.
4. Consider whether it is possible to convert or cast the column to a different data type. The following modules all
provide considerable flexibility and power for modifying data:
Execute Python Script.
NOTE
Still not working? Consider providing additional feedback on the problem, to help us develop better troubleshooting
guidance. Just submit feedback on this page and provide the name of the module that generated the error, and the data
type conversion that failed.
EXCEPTION MESSAGES
Not allowed conversion.
Could not convert: {0}.
Could not convert: {0}, on row {1}.
Could not convert column of type {0} to column of type {1} on row {2}.
Could not convert column "{2}" of type {0} to column of type {1} on row {3}.
Could not convert column "{2}" of type {0} to column "{3}" of type {1} on row {4}.
Error 0140
Exception occurs if passed column set argument does not contain other columns except label column.
This error occurs if you connected a dataset to a module that requires multiple columns, including features, but you
have provided only the label column.
Resolution: Choose at least one feature column to include in the dataset.
EXCEPTION MESSAGES
Specified column set does not contain other columns except label column.
Error 0141
Exception occurs if the number of the selected numerical columns and unique values in the categorical and string
columns is too small.
This error in Azure Machine Learning occurs when there are not enough unique values in the selected column to
perform the operation.
Resolution: Some operations perform statistical operations on feature and categorical columns, and if there are
not enough values, the operation might fail or return an invalid result. Check your dataset to see how many values
there are in the feature and label columns, and determine whether the operation you are trying to perform is
statistically valid.
If the source dataset is valid, you might also check whether some upstream data manipulation or metadata
operation has changed the data and removed some values.
If upstream operations include splitting, sampling, or resampling, verify that the outputs contain the expected
number of rows and values.
EXCEPTION MESSAGES
The number of the selected numerical columns and unique values in the categorical and string columns is too small.
The total number of the selected numerical columns and unique values in the categorical and string columns (currently {0}) should
be at least {1}
Error 0142
Exception occurs when the system cannot load certificate to authenticate.
Resolution:
EXCEPTION MESSAGES
The certificate cannot be loaded.
The certificate {0} cannot be loaded. Its thumbprint is {1}.
Error 0143
Can't parse user-provided URL that is supposed to be from GitHub.
This error in Azure Machine Learning occurs when you specify an invalid URL and the module requires a valid
GitHub URL.
Resolution: Verify that the URL refers to a valid GitHub repository. Other site types are not supported.
EXCEPTION MESSAGES
URL is not from github.com.
URL is not from github.com: {0}
Error 0144
User-provided GitHub url is missing the expected part.
This error in Azure Machine Learning occurs when you specify a GitHub file source using an invalid URL format.
Resolution: Check that the URL of the GitHub repository is valid and ends with \blob\ or \tree\.
EXCEPTION MESSAGES
Cannot parse GitHub URL.
Cannot parse GitHub URL (expecting '\blob\' or '\tree\' after the repository name): {0}
Error 0145
Cannot create the replication directory for some reason.
This error in Azure Machine Learning occurs when the module fails to create the specified directory.
Resolution:
EXCEPTION MESSAGES
Cannot create replication directory.
Error 0146
When the user files are unzipped into the local directory, the combined path might be too long.
This error in Azure Machine Learning occurs when you are extracting files but some file names are too long when
unzipped.
Resolution: Edit the file names such that combined path and file name is no longer than 248 characters.
EXCEPTION MESSAGES
Replication path is longer than 248 characters, shorten the script name or path.
Error 0147
Could not download stuff from GitHub for some reason
This error in Azure Machine Learning occurs when you cannot read or download the specified files from GitHub.
Resolution: The issue might be temporary; you might try accessing the files at another time. Or verify that you
have the necessary permissions and that the source is valid.
EXCEPTION MESSAGES
GitHub access error.
GitHub access error. {0}
Error 0148
Unauthorized access issues while extracting data or creating directory.
This error in Azure Machine Learning occurs when you are trying to create a directory or read data from storage
but do not have the necessary permissions.
Resolution:
EXCEPTION MESSAGES
Unauthorized access exception while extracting data.
Error 0149
The user file does not exist inside GitHub bundle.
This error in Azure Machine Learning occurs when the specified file cannot be found.
Resolution:
EXCEPTION MESSAGES
GitHub file is not found.
GitHub file is not found.: {0}
Error 0150
The scripts that come from the user package could not be unzipped, most likely because of a collision with GitHub
files.
This error in Azure Machine Learning occurs when a script cannot be extracted, typically when there is an existing
file of the same name.
Resolution:
EXCEPTION MESSAGES
Unable to unzip the bundle; possible name collision with GitHub files.
Error 0151
There was an error writing to cloud storage. Check the URL.
This error in Azure Machine Learning occurs when the module tries to write data to cloud storage but the URL is
unavailable or invalid.
Resolution: Check the URL and verify that it is writable.
EXCEPTION MESSAGES
Error writing to cloud storage (possibly a bad url).
Error writing to cloud storage: {0}. Check the url.
Error 0152
The Azure cloud type was specified incorrectly in the module context.
EXCEPTION MESSAGES
Bad Azure Cloud Type
Bad Azure Cloud Type: {0}
Error 0153
The storage end point specified is invalid.
EXCEPTION MESSAGES
Bad Azure Cloud Type

EXCEPTION MESSAGES
Bad Storage End Point: {0}
Error 0154
The specified server name could not be resolved
EXCEPTION MESSAGES
The specified server name could not be resolved
The specified server {0}.documents.azure.com could not be resolved
Error 0155
The DocDb Client threw an exception
EXCEPTION MESSAGES
The DocDb Client threw an exception
DocDb Client: {0}
Error 0156
Bad response for HCatalog Server.
EXCEPTION MESSAGES
Bad response for HCatalog Server. Check that all services are running.
Bad response for HCatalog Server. Check that all services are running. Error details: {0}
Error 0157
There was an error reading from Azure Cosmos DB due to inconsistent or different document schemas. Reader
requires all documents to have the same schema.
EXCEPTION MESSAGES
Detected documents with different schemas. Make sure all documents have the same schema
Error 1000
Internal library exception.
This error is provided to capture otherwise unhandled internal engine errors. Therefore, the cause for this error
might be different depending on the module that generated the error.
To get more help, we recommend that you post the detailed message that accompanies the error to the Azure
Machine Learning forum, together with a description of the scenario, including the data used as inputs. This
feedback will help us to prioritize errors and identify the most important issues for further work.
EXCEPTION MESSAGES
Library exception.
Library exception: {0}
{0} library exception: {1}

Azure Machine Learning service release notes
In this article, learn about the Azure Machine Learning service releases. For the full SDK reference content, visit the
Azure Machine Learning's main SDK for Python reference page.
See the list of known issues to learn about known bugs and workarounds.
2019-08-19
Azure Machine Learning SDK for Python v1.0.57
New features
Enabled TabularDataset to be consumed by AutomatedML. To learn more about TabularDataset , please
visit https://aka.ms/azureml/howto/createdatasets.
Bug fixes and improvements
automl-client-core-nativeclient
Fixed the error, raised when training and/or validation labels (y and y_valid) are provided in the
form of pandas dataframe but not as numpy array.
Updated interface to create a RawDataContext to only require the data and the AutoMLBaseSettings
object.
Allow AutoML users to drop training series that are not long enough when forecasting. - Allow
AutoML users to drop grains from the test set that do not exist in the training set when
forecasting.
azure-cli-ml
You can now update the SSL certificate for the scoring endpoint deployed on AKS cluster both for
Microsoft generated and customer certificate.
azureml-automl-core
Fixed an issue in AutoML where rows with missing labels were not removed properly.
Improved error logging in AutoML; full error messages will now always be written to the log file.
AutoML has updated its package pinning to include azureml-defaults , azureml-explain-model ,
and azureml-dataprep . AutoML will no longer warn on package mismatches (except for
azureml-train-automl package).
Fixed an issue in timeseries where cv splits are of unequal size causing bin calculation to fail.
When running ensemble iteration for the Cross-Validation training type, if we ended up having
trouble downloading the models trained on the entire dataset, we were having an inconsistency
between the model weights and the models that were being fed into the voting ensemble.
Fixed the error, raised when training and/or validation labels (y and y_valid) are provided in the
form of pandas dataframe but not as numpy array.
Fixed the issue with the forecasting tasks when None was encountered in the Boolean columns of
input tables.
Allow AutoML users to drop training series that are not long enough when forecasting. - Allow
AutoML users to drop grains from the test set that do not exist in the training set when
forecasting.
azureml-core
Fixed issue with blob_cache_timeout parameter ordering.
Added external fit and transform exception types to system errors.
Added support for Key Vault secrets for remote runs. Add a azureml.core.keyvault.Keyvault class
to add, get and list secrets from the keyvault associated with your workspace. Supported
operations are:
azureml.core.workspace.Workspace.get_default_keyvault()
azureml.core.keyvault.Keyvault.set_secret(name, value)
azureml.core.keyvault.Keyvault.set_secrets(secrets_dict)
azureml.core.keyvault.Keyvault.get_secret(name)
azureml.core.keyvault.Keyvault.get_secrets(secrets_list)
azureml.core.keyvault.Keyvault.list_secrets()
Additional methods to obtain default keyvault and get secrets during remote run:
azureml.core.workspace.Workspace.get_default_keyvault()
azureml.core.run.Run.get_secret(name)
azureml.core.run.Run.get_secrets(secrets_list)
Added additional override parameters to submit-hyperdrive CLI command.
Improve reliability of API calls be expanding retries to common requests library exceptions.
Add support for submitting runs from a submitted run.
Fixed expiring SAS token issue in FileWatcher which caused files to stop being uploaded after
their initial token had expired.
Supported importing HTTP csv/tsv files in dataset python SDK.
Deprecated the Workspace.setup() method. Warning message shown to users suggests to use
create() or get()/from_config() instead.
Added Environment.add_private_pip_wheel(), which enables uploading private custom python
packages (.whl) to the workspace and securely using them to build/materialize the environment.
You can now update the SSL certificate for the scoring endpoint deployed on AKS cluster both for
Microsoft generated and customer certificate.
azureml-explain-model
Added parameter to add a model ID to explanations on upload.
Added is_raw tagging to explanations in memory and upload.
Added pytorch support and tests for azureml-explain-model package.
azureml-opendatasets
Support detecting and logging auto test environment.
Added classes to get US population by county and zip.
azureml-pipeline-core
Added label property to input and output port definitions.
azureml-telemetry
Fixed an incorrect telemetry configuration.
azureml-train-automl
Fixed the bug where on setup failure, error was not getting logged in "errors" field for the setup
run and hence was not stored in parent run "errors".
Fixed an issue in AutoML where rows with missing labels were not removed properly.
Allow AutoML users to drop training series that are not long enough when forecasting.
Allow AutoML users to drop grains from the test set that do not exist in the training set when
forecasting.
Now AutoMLStep passes through automl config to backend to avoid any issues on changes or
additions of new config parameters.
AutoML Data Guardrail is now in public preview. User will see a Data Guardrail report (for
classification/regression tasks) after training and also be able to access it through SDK API.
azureml-train-core
Added torch 1.2 support in PyTorch Estimator.
azureml-widgets
Improved confusion matrix charts for classification training.
Azure Machine Learning Data Prep SDK v1.1.12
New features
Lists of strings can now be passed in as input to read_* methods.
The performance of read_parquet has been significantly improved when running in Spark.
Fixed an issue where column_type_builder failed in case of a single column with ambiguous date
formats.
Azure Portal
Preview Feature
Log and output file streaming is now available for run details pages. The files will stream updates in real
time when the preview toggle is turned on.
Ability to set quota at a workspace level is released in preview. AmlCompute quotas are allocated at the
subscription level, but we now allow you to distribute that quota between workspaces and allocate it for
fair sharing and governance. Just click on the Usages+Quotas blade in the left navigation bar of your
workspace and select the Configure Quotas tab. Note that you must be a subscription admin to be able
to set quotas at the workspace level since this is a cross-workspace operation.
2019-08-05
New features
Token based authentication is now supported for the calls made to the scoring endpoint deployed on
AKS. We will continue to support the current key based authentication and users can use one of these
authentication mechanisms at a time.
Ability to register a blob storage that is behind the virtual network (VNet) as a datastore.
azureml-automl-core
Fixes a bug where validation size for CV splits is small and results in bad predicted vs. true charts
for regression and forecasting.
The logging of forecasting tasks on the remote runs improved, now user is provided with
comprehensive error message if the run was failed.
Fixed failures of Timeseries if preprocess flag is True.
Made some forecasting data validation error messages more actionable.
Reduced memory consumption of AutoML runs by dropping and/or lazy loading of datasets,
especially in between process spawns
azureml-contrib-explain-model
Added model_task flag to explainers to allow user to override default automatic inference logic for
model type
Widget changes: Automatically installs with contrib, no more nbextension install/enable - support
explanation with just global feature importance (eg Permutative)
Dashboard changes: - Box plots and violin plots in addition to beeswarm plot on summary page -
Much faster rerendering of beeswarm plot on 'Top -k' slider change - helpful message explaining
how top-k is computed - Useful customizable messages in place of charts when data not provided
azureml-core
Added Model.package() method to create Docker images and Dockerfiles that encapsulate models
and their dependencies.
Updated local webservices to accept InferenceConfigs containing Environment objects.
Fixed Model.register() producing invalid models when '.' (for the current directory) is passed as
the model_path parameter.
Add Run.submit_child, the functionality mirrors Experiment.submit while specifying the run as the
parent of the submitted child run.
Support configuration options from Model.register in Run.register_model.
Ability to run JAR jobs on existing cluster.
Now supporting instance_pool_id and cluster_log_dbfs_path parameters.
Added support for using an Environment object when deploying a Model to a Webservice. The
Environment object can now be provided as a part of the InferenceConfig object.
Add appinsifht mapping for new regions - centralus - westus - northcentralus
Added documentation for all the attributes in all the Datastore classes.
Added blob_cache_timeout parameter to Datastore.register_azure_blob_container .
Added save_to_directory and load_from_directory methods to
azureml.core.environment.Environment.
Added the "az ml environment download" and "az ml environment register" commands to the
CLI.
Added Environment.add_private_pip_wheel method.
Added dataset tracking to Explanations using the Dataset service (preview ).
Decreased default batch size when streaming global explanations from 10k to 100.
Added model_task flag to explainers to allow user to override default automatic inference logic for
model type.
azureml-mlflow
Fixed bug in mlflow.azureml.build_image where nested directories are ignored.
azureml-pipeline-steps
Added ability to run JAR jobs on existing Azure Databricks cluster.
Added support instance_pool_id and cluster_log_dbfs_path parameters for DatabricksStep step.
Added support for pipeline parameters in DatabricksStep step.
Added docstrings for the Ensemble related files.
Updated docs to more appropriate language for max_cores_per_iteration and
max_concurrent_iterations
The logging of forecasting tasks on the remote runs improved, now user is provided with
comprehensive error message if the run was failed.
Removed get_data from pipeline automlstep notebook.
Started support dataprep in automlstep.
New features
You can now request to execute specific inspectors (e.g. histogram, scatter plot, etc) on specific columns.
Added a parallelize argument to append_columns . If True, data will be loaded into memory but execution
will run in parallel; if False, execution will be streaming but single-threaded.
2019-07-23
New features
Automated Machine Learning now supports training ONNX models on the remote compute target
Azure Machine Learning now provides ability to resume training from a previous run, checkpoint or
model files.
Learn how to use estimators to resume training from a previous run
automl-client-core-nativeclient
Fix the bug about loosing columns types after the transformation (bug linked);
Allow y_query to be an object type containing None(s) at the begin (#459519).
azure-cli-ml
CLI commands "model deploy" and "service update" now accept parameters, config files, or a
combination of the two. Parameters have precedence over attributes in files.
Model description can now be updated after registration
azureml-automl-core
Update NimbusML dependency to 1.2.0 version (current latest).
Adding support for Nimbus ML estimators & pipelines to be used within AutoML estimators.
Fixing a bug in the Ensemble selection procedure which was unnecessarily growing the resulting
ensemble even if the scores remained constant.
Enable re-use of some featurizations across CV Splits for forecasting tasks. This speeds up the
run-time of the setup run by roughly a factor of n_cross_validations for expensive featurizations
like lags and rolling windows.
Addressing an issue if time is out of pandas supported time range. We now raise a DataException
if time is less than pd.Timestamp.min or greater than pd.Timestamp.max
Forecasting now allows different frequencies in train and test sets if they can be aligned. For
example, “quarterly starting in January” and at “quarterly starting in October” can be aligned.
The property "parameters" was added to the TimeSeriesTransformer.
Remove old exception classes.
In forecasting tasks, the target_lags parameter now accepts a single integer value or a list of
integers. If the integer was provided, only one lag will be created. If a list is provided, the unique
values of lags will be taken. target_lags=[1, 2, 2, 4] will create lags of one, 2 and 4 periods.
Fix the bug about losing columns types after the transformation (bug linked);
In model.forecast(X, y_query) , allow y_query to be an object type containing None(s) at the begin
(#459519).
Add expected values to automl output
azureml-contrib-datadrift
Improvements to example notebook including switch to azureml-opendatasets instead of
azureml-contrib-opendatasets and performance improvements when enriching data
azureml-contrib-explain-model
Fixed transformations argument for LIME explainer for raw feature importance in azureml-
contrib-explain-model package
added segmentations to image explanations in image explainer for AzureML -contrib-explain-
model package
add scipy sparse support for LimeExplainer
add batch_size to mimic explainer when include_local=False for streaming global explanations in
batches to improve execution time of DecisionTreeExplainableModel
azureml-contrib-featureengineering
Fix for calling set_featurizer_timeseries_params(): dict value type change and null check - Add
notebook for timeseries featurizer
azureml-core
Added the ability to attach DBFS datastores in the AzureML CLI
Fixed the bug with datastore upload where an empty folder is created if target_path started with
/
Fixed deepcopy issue in ServicePrincipalAuthentication.
Added the "az ml environment show" and "az ml environment list" commands to the CLI.
Environments now support specifying a base_dockerfile as an alternative to an already-built
base_image.
The unused RunConfiguration setting auto_prepare_environment has been marked as
deprecated.
Model description can now be updated after registration
Bugfix: Model and Image delete now provides more information about retrieving upstream
objects that depend on them if delete fails due to an upstream dependency.
Fixed bug that printed blank duration for deployments that occur when creating a workspace for
some environments.
Improved workspace create failure exceptions. Such that users don't see "Unable to create
workspace. Unable to find..." as the message and instead see the actual creation failure.
Add support for token authentication in AKS webservices.
Add get_token() method to Webservice objects.
Added CLI support to manage machine learning datasets.
Datastore.register_azure_blob_container now optionally takes a blob_cache_timeout value (in
seconds) which configures blobfuse's mount parameters to enable cache expiration for this
datastore. The default is no timeout, i.e. when a blob is read, it will stay in the local cache until the
job is finished. Most jobs will prefer this setting, but some jobs need to read more data from a
large dataset than will fit on their nodes. For these jobs, tuning this parameter will help them
succeed. Take care when tuning this parameter: setting the value too low can result in poor
performance, as the data used in an epoch may expire before being used again. This means that
all reads will be done from blob storage (i.e. the network) rather than the local cache, which
negatively impacts training times.
Model description can now properly be updated after registration
Model and Image deletion now provides more information about upstream objects that depend
on them which causes the delete to fail
Improve resource utilization of remote runs using azureml.mlflow.
Fixed transformations argument for LIME explainer for raw feature importance in azureml-
contrib-explain-model package
add scipy sparse support for LimeExplainer
added shap linear explainer wrapper, as well as another level to tabular explainer for explaining
linear models
for mimic explainer in explain model library, fixed error when include_local=False for sparse data
input
add expected values to automl output
fixed permutation feature importance when transformations argument supplied to get raw feature
importance
add batch_size to mimic explainer when include_local=False for streaming global explanations in
batches to improve execution time of DecisionTreeExplainableModel
for model explainability library, fixed blackbox explainers where pandas dataframe input is
required for prediction
Fixed a bug where explanation.expected_values would sometimes return a float rather than a list
with a float in it.
azureml-mlflow
Improve performance of mlflow.set_experiment(experiment_name)
Fix bug in use of InteractiveLoginAuthentication for mlflow tracking_uri
Improve resource utilization of remote runs using azureml.mlflow.
Improve the documentation of the azureml-mlflow package
Patch bug where mlflow.log_artifacts("my_dir") would save artifacts under "my_dir/" instead of ""
Pin pyarrow of opendatasets to old versions (<0.14.0) because of memory issue newly introduced
there.
Move azureml-contrib-opendatasets to azureml-opendatasets. - Allow open dataset classes to be
registered to AML workspace and leverage AML Dataset capabilities seamlessly. - Improve
NoaaIsdWeather enrich performance in non-SPARK version significantly.
DBFS Datastore is now supported for Inputs and Outputs in DatabricksStep.
Updated documentation for Azure Batch Step with regards to inputs/outputs.
In AzureBatchStep, changed delete_batch_job_after_finish default value to true.
azureml-telemetry
Move azureml-contrib-opendatasets to azureml-opendatasets. - Allow open dataset classes to be
registered to AML workspace and leverage AML Dataset capabilities seamlessly. - Improve
NoaaIsdWeather enrich performance in non-SPARK version significantly.
Updated documentation on get_output to reflect the actual return type and provide additional
notes on retrieving key properties.
add expected values to automl output
azureml-train-core
Strings are now accepted as compute target for Automated Hyperparameter Tuning
The unused RunConfiguration setting auto_prepare_environment has been marked as
deprecated.
New features
Added support for reading a file directly from a http or https url.
Improved error message when attempting to read a Parquet Dataset from a remote source (which is not
currently supported).
Fixed a bug when writing to Parquet file format in ADLS Gen 2, and updating the ADLS Gen 2 container
name in the path.
2019-07-09
Visual Interface
Preview features
Added "Execute R script" module in visual interface.
New features
azureml-contrib-opendatasets is now available as azureml-opendatasets. The old package
can still work, but we recommend you using azureml-opendatasets moving forward for richer
capabilities and improvements.
This new package allows you to register open datasets as Dataset in AML workspace, and
leverage whatever functionalities that Dataset offers.
It also includes existing capabilities such as consuming open datasets as Pandas/SPARK
dataframes, and location joins for some dataset like weather.
Preview features
HyperDriveConfig can now accept pipeline object as a parameter to support hyperparameter tuning
using a pipeline.
Fixed the bug about losing columns types after the transformation.
Fixed the bug to allow y_query to be an object type containing None(s) at the beginning.
Fixed the issue in the Ensemble selection procedure which was unnecessarily growing the
resulting ensemble even if the scores remained constant.
Fixed the issue with whitelist_models and blacklist_models settings in AutoMLStep.
Fixed the issue that prevented the usage of preprocessing when AutoML would have been used in
the context of Azure ML Pipelines.
Moved azureml-contrib-opendatasets to azureml-opendatasets.
Allowed open dataset classes to be registered to AML workspace and leverage AML Dataset
capabilities seamlessly.
Improved NoaaIsdWeather enrich performance in non-SPARK version significantly.
Updated online documentation for interpretability objects.
Added batch_size to mimic explainer when include_local=False for streaming global explanations
in batches to improve execution time of DecisionTreeExplainableModel.
Fixed the issue where explanation.expected_values would sometimes return a float rather than a
list with a float in it.
Added expected values to automl output for mimic explainer in explain model library.
Fixed permutation feature importance when transformations argument supplied to get raw
feature importance.
Added batch_size to mimic explainer when include_local=False for streaming global explanations
in batches to improve execution time of DecisionTreeExplainableModel for model explainability
library.
azureml-core
Added the ability to attach DBFS datastores in the AzureML CLI.
Fixed the issue with datastore upload where an empty folder is created if target_path started
with / .
Enabled comparison of two datasets.
Model and Image delete now provides more information about retrieving upstream objects that
depend on them if delete fails due to an upstream dependency.
Deprecated the unused RunConfiguration setting in auto_prepare_environment.
azureml-mlflow
Improved resource utilization of remote runs that use azureml.mlflow.
Improved the documentation of the azureml-mlflow package.
Fixed the issue where mlflow.log_artifacts("my_dir") would save artifacts under "my_dir/artifact-
paths" instead of "artifact-paths".
azureml-pipeline-core
Parameter hash_paths for all pipeline steps is deprecated and will be removed in future. By default
contents of the source_directory is hashed (except files listed in .amlignore or .gitignore)
Continuing improving Module and ModuleStep to support compute type specific modules, in
preparation for RunConfiguration integration and further changes to unlock their usage in
pipelines.
AzureBatchStep: Improved documentation with regards to inputs/outputs.
AzureBatchStep: Changed delete_batch_job_after_finish default value to true.
azureml-train-core
Strings are now accepted as compute target for Automated Hyperparameter Tuning.
Deprecated the unused RunConfiguration setting in auto_prepare_environment.
Deprecated parameters conda_dependencies_file_path and pip_requirements_file_path in favor of
conda_dependencies_file and pip_requirements_file respectively.
Improve NoaaIsdWeather enrich performance in non-SPARK version significantly.
New features
Dataflow objects can now be iterated over, producing a sequence of records. See documentation for
Dataflow.to_record_iterator .

Increased the robustness of DataPrep SDK.
Improved handling of pandas DataFrames with non-string Column Indexes.
Improved the performance of to_pandas_dataframe in Datasets.
Fixed a bug where Spark execution of Datasets failed when run in a multi-node environment.
2019-07-01
We reverted a change that improved performance, as it was causing issues for some customers using Azure
Databricks. If you experienced an issue on Azure Databricks, you can upgrade to version 1.1.7 using one of the
methods below:
1. Run this script to upgrade: %sh /home/ubuntu/databricks/python/bin/pip install azureml-dataprep==1.1.7
2. Recreate the cluster, which will install the latest Data Prep SDK version.
2019-06-25
New features
Add decision tree surrogate model to mimic explainer in azureml-explain-model package
Ability to specify a CUDA version to be installed on Inferencing images. Support for CUDA 9.0, 9.1, and
10.0.
Information about Azure ML training base images are now available at Azure ML Containers GitHub
Repository and DockerHub
Added CLI support for pipeline schedule. Run "az ml pipeline -h" to learn more
Added custom Kubernetes namespace parameter to AKS webservice deployment configuration and CLI.
Deprecated hash_paths parameter for all pipeline steps
Model.register now supports registering multiple individual files as a single model with use of the
child_paths parameter.
Preview features
Scoring explainers can now optionally save conda and pip information for more reliable serialization and
deserialization.
Bug Fix for Auto Feature Selector.
Updated mlflow.azureml.build_image to the new api, patched bugs exposed by the new implementation.
Removed paramiko dependency from azureml-core. Added deprecation warnings for legacy compute
target attach methods.
Improve performance of run.create_children
In mimic explainer with binary classifier, fix the order of probabilities when teacher probability is used for
scaling shap values
Improved error handling and message for Automated machine learning.
Fixed the iteration timeout issue for Automated machine learning.
Improved the time-series transformation performance for Automated machine learning.
2019-06-24
New features
Added summary functions for top values ( SummaryFunction.TOPVALUES ) and bottom values (
SummaryFunction.BOTTOMVALUES ).
Significantly improved the performance of read_pandas_dataframe .
Fixed a bug that would cause get_profile() on a Dataflow pointing to binary files to fail.
Exposed set_diagnostics_collection() to allow for programmatic enabling/disabling of the telemetry
collection.
Changed the behavior of get_profile() . NaN values are now ignored for Min, Mean, Std, and Sum,
which aligns with the behavior of Pandas.
2019-06-10
New features
Azure Machine Learning now provides first-class support for popular machine learning and data
analysis framework Scikit-learn. Using SKLearn estimator, users can easily train and deploy Scikit-learn
models.
Learn how to run hyperparameter tuning with Scikit-learn using HyperDrive.
Added support for creating ModuleStep in pipelines along with Module and ModuleVersion classes to
manage reusable compute units.
ACI webservices now support persistent scoring_uri through updates. The scoring_uri will change from
IP to FQDN. The Dns Name Label for FQDN can be configured by setting the dns_name_label on
deploy_configuration.
Automated machine learning new features:
STL featurizer for forecasting
KMeans clustering is enabled for feature sweeping
AmlCompute Quota approvals just became faster! We have now automated the process to approve your
quota requests within a threshold. For more information on how quotas work, learn how to manage
quotas.
Preview features
Integration with MLflow 1.0.0 tracking through azureml-mlflow package (example notebooks).
Submit Jupyter notebook as a run. API Reference Documentation
Public Preview of Data Drift Detector through azureml-contrib-datadrift package (example notebooks).
Data Drift is one of the top reasons where model accuracy degrades over time. It happens when data
served to model in production is different from the data that the model was trained on. AML Data Drift
detector helps customer to monitor data drift and sends alert whenever drift is detected.
Breaking changes
RunConfiguration load and save supports specifying a full file path with full back-compat for previous
behavior.
Added caching in ServicePrincipalAuthentication, turned off by default.
Enable logging of multiple plots under the same metric name.
Model class now properly importable from azureml.core ( from azureml.core import Model ).
In pipeline steps, hash_path parameter is now deprecated. New behavior is to hash complete
source_directory, except files listed in .amlignore or .gitignore.
In pipeline packages, various get_all and get_all_* methods have been deprecated in favor of list
and list_* , respectively.
azureml.core.get_run no longer requires classes to be imported before returning the original run type.
Fixed an issue where some calls to WebService Update did not trigger an update.
Scoring timeout on AKS webservices should be between 5ms and 300000ms. Max allowed
scoring_timeout_ms for scoring requests has been bumped from 1 min to 5 min.
LocalWebservice objects now have scoring_uri and swagger_uri properties.
Moved outputs directory creation and outputs directory upload out of the user process. Enabled run
history SDK to run in every user process. This should resolve some synchronization issues experienced
by distributed training runs.
The name of the azureml log written from the user process name will now include process name (for
distributed training only) and PID.
For interpreted datetime values that have a 2-digit year format, the range of valid years has been
updated to match Windows May Release. The range has been changed from 1930-2029 to 1950-2049.
When reading in a file and setting handleQuotedLineBreaks=True , \r will be treated as a new line.
Fixed a bug that caused read_pandas_dataframe to fail in some cases.
Improved performance of get_profile .
Improved error messages.
2019-05-28
New features
You can now use the following expression language functions to extract and parse datetime values into
new columns.
RegEx.extract_record() extracts datetime elements into a new column.
create_datetime() creates datetime objects from separate datetime elements.
When calling get_profile() , you can now see that quantile columns are labeled as (est.) to clearly
indicate that the values are approximations.
You can now use ** globbing when reading from Azure Blob Storage.
e.g. dprep.read_csv(path='https://yourblob.blob.core.windows.net/yourcontainer/**/data/*.csv')
Bug fixes
Fixed a bug related to reading a Parquet file from a remote source (Azure Blob).
2019-05-14
Changes
Run configuration auto_prepare_environment option is being deprecated, with auto prepare becoming
the default.
2019-05-08
New features
Added support to read from a PostgresSQL database, either by calling read_postgresql or using a
Datastore.
See examples in how -to guides:
Data Ingestion notebook
Datastore notebook
Fixed issues with column type conversion:
Now correctly converts a boolean or numeric column to a boolean column.
Now does not fail when attempting to set a date column to be date type.
Improved JoinType types and accompanying reference documentation. When joining two dataflows, you
can now specify one of these types of join:
NONE, MATCH, INNER, UNMATCHLEFT, LEFTANTI, LEFTOUTER, UNMATCHRIGHT,
RIGHTANTI, RIGHTOUTER, FULLANTI, FULL.
Improved data type inferencing to recognize more date formats.
2019-05-06
Azure portal
In Azure portal, you can now:
Create and run automated ML experiments
Create a Notebook VM to try out sample Jupyter notebooks or your own.
Brand new Authoring section (Preview ) in the Machine Learning service workspace, which includes Automated
Machine Learning, Visual Interface and Hosted Notebook VMs
Automatically create a model using Automated machine learning
Use a drag and drop Visual Interface to run experiments
Create a Notebook VM to explore data, create models, and deploy services.
Live chart and metric updating in run reports and run details pages
Updated file viewer for logs, outputs, and snapshots in Run details pages.
New and improved report creation experience in the Experiments tab.
Added ability to download the config.json file from the Overview page of the Azure Machine Learning service
workspace.
Support Machine Learning service workspace creation from Azure Databricks workspace
2019-04-26
New features
The Workspace.create method now accepts default cluster configurations for CPU and GPU clusters.
If Workspace creation fails, depended resources are cleaned.
Default Azure Container Registry SKU was switched to basic.
Azure Container Registry is created lazily, when needed for run or image creation.
Support for Environments for training runs.
Notebook Virtual Machine
Use a Notebook VM as a secure, enterprise-ready hosting environment for Jupyter notebooks in which you can
program machine learning experiments, deploy models as web endpoints and perform all other operations
supported by Azure Machine Learning SDK using Python. It provides several capabilities:
Quickly spin up a preconfigured notebook VM that has the latest version of Azure Machine Learning SDK and
related packages.
Access is secured through proven technologies, such as HTTPS, Azure Active Directory authentication and
authorization.
Reliable cloud storage of notebooks and code in your Azure Machine Learning Workspace blob storage
account. You can safely delete your notebook VM without losing your work.
Preinstalled sample notebooks to explore and experiment with Azure Machine Learning service features.
Full customization capabilities of Azure VMs, any VM type, any packages, any drivers.
2019-04-26
Azure Machine Learning SDK for Python v1.0.33 released.
Azure ML Hardware Accelerated Models on FPGAs is generally available.
You can now use the azureml-accel-models package to:
Train the weights of a supported deep neural network (ResNet 50, ResNet 152, DenseNet-121,
VGG -16, and SSD -VGG )
Use transfer learning with the supported DNN
Register the model with Model Management Service and containerize the model
Deploy the model to an Azure VM with an FPGA in an Azure Kubernetes Service (AKS ) cluster
Deploy the container to an Azure Data Box Edge server device
Score your data with the gRPC endpoint with this sample
Automated Machine Learning
Feature sweeping to enable dynamically adding featurizers for performance optimization. New featurizers:
work embeddings, weight of evidence, target encodings, text target encoding, cluster distance
Smart CV to handle train/valid splits inside automated ML
Few memory optimization changes and runtime performance improvement
Performance improvement in model explanation
ONNX model conversion for local run
Added Subsampling support
Intelligent Stopping when no exit criteria defined
Stacked ensembles
Time Series Forecasting
New predict forecast function
You can now use rolling-origin cross validation on time series data
New functionality added to configure time series lags
New functionality added to support rolling window aggregate features
New Holiday detection and featurizer when country code is defined in experiment settings
Azure Databricks
Enabled time series forecasting and model explainabilty/interpretability capability
You can now cancel and resume (continue) automated ML experiments
Added support for multicore processing
MLOps
Local deployment & debugging for scoring containers
You can now deploy an ML model locally and iterate quickly on your scoring file and dependencies to
ensure they behave as expected.
Introduced InferenceConfig & Model.deploy()
Model deployment now supports specifying a source folder with an entry script, the same as a RunConfig.
Additionally, model deployment has been simplified to a single command.
Git reference tracking
Customers have been requesting basic Git integration capabilities for some time as it helps maintain an
end-to-end audit trail. We have implemented tracking across major entities in Azure ML for Git-related
metadata (repo, commit, clean state). This information will be collected automatically by the SDK and CLI.
Model profiling & validation service
Customers frequently complain of the difficulty to properly size the compute associated with their inference
service. With our model profiling service, the customer can provide sample inputs and we will profile across
16 different CPU / memory configurations to determine optimal sizing for deployment.
Bring your own base image for inference
Another common complaint was the difficulty in moving from experimentation to inference RE sharing
dependencies. With our new base image sharing capability, you can now reuse your experimentation base
images, dependencies and all, for inference. This should speed up deployments and reduce the gap from the
inner to the outer loop.
Improved Swagger schema generation experience
Our previous swagger generation method was error prone and impossible to automate. We have a new in-
line way of generating swagger schemas from any Python function via decorators. We have open-sourced
this code and our schema generation protocol is not coupled to the Azure ML platform.
Azure ML CLI is generally available (GA )
Models can now be deployed with a single CLI command. We got common customer feedback that no one
deploys an ML model from a Jupyter notebook. The CLI reference documentation has been updated.
2019-04-22
Azure Machine Learning SDK for Python v1.0.30 released.
The PipelineEndpoint was introduce to add a new version of a published pipeline while maintaining same
endpoint.
2019-04-17
Note: Data Prep Python SDK will no longer install numpy and pandas packages. See updated installation
instructions.
New features
You can now use the Pivot transform.
How -to guide: Pivot notebook
You can now use regular expressions in native functions.
Examples:
dflow.filter(dprep.RegEx('pattern').is_match(dflow['column_name']))
dflow.assert_value('column_name', dprep.RegEx('pattern').is_match(dprep.value))
You can now use to_upper and to_lower functions in expression language.
You can now see the number of unique values of each column in a data profile.
For some of the commonly used reader steps, you can now pass in the infer_column_types argument. If
it is set to True , Data Prep will attempt to detect and automatically convert column types.
inference_arguments is now deprecated.
You can now call Dataflow.shape .
keep_columns now accepts an additional optional argument validate_column_exists , which checks if the
result of keep_columns will contain any columns.
All reader steps (which read from a file) now accept an additional optional argument verify_exists .
Improved performance of reading from pandas dataframe and getting data profiles.
Fixed a bug where slicing a single step from a Dataflow failed with a single index.
2019-04-15
Azure portal
You can now resubmit an existing Script run on an existing remote compute cluster.
You can now run a published pipeline with new parameters on the Pipelines tab.
Run details now supports a new Snapshot file viewer. You can view a snapshot of the directory when you
submitted a specific run. You can also download the notebook that was submitted to start the run.
You can now cancel parent runs from the Azure portal.
2019-04-08
New features
The Azure Machine Learning SDK now supports Python 3.7.
Azure Machine Learning DNN Estimators now provide built-in multi-version support. For example,
TensorFlow estimator now accepts a framework_version parameter, and users can specify version '1.10'
or '1.12'. For a list of the versions supported by your current SDK release, call get_supported_versions()
on the desired framework class (for example, TensorFlow.get_supported_versions() ). For a list of the
versions supported by the latest SDK release, see the DNN Estimator documentation.
New features
You can read multiple Datastore/DataPath/DataReference sources using read_* transforms.
You can perform the following operations on columns to create a new column: division, floor, modulo,
power, length.
Data Prep is now part of the Azure ML diagnostics suite and will log diagnostic information by default.
To turn this off, set this environment variable to true: DISABLE_DPREP_LOGGER
Improved code documentation for commonly used classes and functions.
Fixed a bug in auto_read_file that failed to read Excel files.
Added option to overwrite the folder in read_pandas_dataframe.
Improved performance of dotnetcore2 dependency installation, and added support for Fedora 27/28
and Ubuntu 1804.
Improved the performance of reading from Azure Blobs.
Column type detection now supports columns of type Long.
Fixed a bug where some date values were being displayed as timestamps instead of Python datetime
objects.
Fixed a bug where some type counts were being displayed as doubles instead of integers.
2019-03-25
New features
The azureml.core.Run.create_children method allows low -latency creation of multiple child runs with a
single call.
Breaking changes
The concept of the Data Prep Package has been deprecated and is no longer supported. Instead of
persisting multiple Dataflows in one Package, you can persist Dataflows individually.
How -to guide: Opening and Saving Dataflows notebook
New features
Data Prep can now recognize columns that match a particular Semantic Type, and split accordingly. The
STypes currently supported include: email address, geographic coordinates (latitude & longitude), IPv4
and IPv6 addresses, US phone number, and US zip code.
How -to guide: Semantic Types notebook
Data Prep now supports the following operations to generate a resultant column from two numeric
columns: subtract, multiply, divide, and modulo.
You can call verify_has_data() on a Dataflow to check whether the Dataflow would produce records if
executed.
You can now specify the number of bins to use in a histogram for numeric column profiles.
The read_pandas_dataframe transform now requires the DataFrame to have string- or byte- typed
column names.
Fixed a bug in the fill_nulls transform, where values were not correctly filled in if the column was
missing.
2019-03-11
Changes
The azureml-tensorboard package replaces azureml-contrib-tensorboard.
With this release, you can set up a user account on your managed compute cluster (amlcompute), while
creating it. This can be done by passing these properties in the provisioning configuration. You can find
more details in the SDK reference documentation.
New features
Now supports adding two numeric columns to generate a resultant column using the expression
language.
Improved the documentation and parameter checking for random_split.
2019-02-27
Bug fix
Fixed a Service Principal authentication issue that was caused by an API change.
2019-02-25
New features
Azure Machine Learning now provides first class support for popular DNN framework Chainer. Using
Chainer class users can easily train and deploy Chainer models.
Learn how to run distributed training with ChainerMN
Learn how to run hyperparameter tuning with Chainer using HyperDrive
Azure Machine Learning Pipelines added ability trigger a Pipeline run based on datastore modifications.
The pipeline schedule notebook is updated to showcase this feature.
We have added support Azure Machine Learning Pipelines for setting the source_directory_data_store
property to a desired datastore (such as a blob storage) on RunConfigurations that are supplied to the
PythonScriptStep. By default Steps use Azure File store as the backing datastore, which may run into
throttling issues when a large number of steps are executed concurrently.
Azure portal
New features
New drag and drop table editor experience for reports. Users can drag a column from the well to the
table area where a preview of the table will be displayed. The columns can be rearranged.
New Logs file viewer
Links to experiment runs, compute, models, images, and deployments from the activities tab
New features
Data Prep now supports writing file streams from a dataflow. Also provides the ability to manipulate the
file stream names to create new file names.
How -to guide: Working With File Streams notebook
Improved performance of t-Digest on large data sets.
Data Prep now supports reading data from a DataPath.
One hot encoding now works on boolean and numeric columns.
Other miscellaneous bug fixes.
2019-02-11
New features
Azure Machine Learning Pipelines added AzureBatchStep (notebook), HyperDriveStep (notebook), and
time-based scheduling functionality (notebook).
DataTranferStep updated to work with Azure SQL Server and Azure database for PostgreSQL
(notebook).
Changes
Deprecated PublishedPipeline.get_published_pipeline in favor of PublishedPipeline.get .
Deprecated Schedule.get_schedule in favor of Schedule.get .
New features
Data Prep now supports reading from an Azure SQL database using Datastore.
Changes
Improved the memory performance of certain operations on large data.
read_pandas_dataframe() now requires temp_folder to be specified.
The name property on ColumnProfile has been deprecated - use column_name instead.
2019-01-28
Changes:
Azure ML SDK no longer has azure-cli packages as dependency. Specifically, azure-cli-core and azure-
cli-profile dependencies have been removed from azureml-core. These are the user impacting changes:
If you are performing "az login" and then using azureml-sdk, the SDK will do the browser or
device code log in one more time. It won't use any credentials state created by "az login".
For Azure CLI authentication, such as using "az login", use
azureml.core.authentication.AzureCliAuthentication class. For Azure CLI authentication, do pip
install azure-cli in the Python environment where you have installed azureml-sdk.
If you are doing "az login" using a service principal for automation, we recommend using
azureml.core.authentication.ServicePrincipalAuthentication class, as azureml-sdk won't use
credentials state created by azure CLI.
Bug fixes: This release mostly contains minor bug fixes
Bug fixes
Improved the performance of getting data profiles.
Fixed minor bugs related to error reporting.
Azure portal: new features
New drag and drop charting experience for reports. Users can drag a column or attribute from the well to
the chart area where the system will automatically select an appropriate chart type for the user based on the
type of data. Users can change the chart type to other applicable types or add additional attributes.
Supported Chart Types:
Line Chart
Histogram
Stacked Bar Chart
Box Plot
Scatter Plot
Bubble Plot
The portal now dynamically generates reports for experiments. When a user submits a run to an
experiment, a report will automatically be generated with logged metrics and graphs to allow comparison
across different runs.
2019-01-14
New features
Datastore improvements (documented in Datastore how -to-guide)
Added ability to read from and write to Azure File Share and ADLS Datastores in scale-up.
When using Datastores, Data Prep now supports using service principal authentication instead of
interactive authentication.
Added support for wasb and wasbs urls.
2019-01-09
Bug fixes
Fixed bug with reading from public readable Azure Blob containers on Spark
2018-12-20
New features
to_bool function now allows mismatched values to be converted to Error values. This is the new default
mismatch behavior for to_bool and set_column_types , whereas the previous default behavior was to
convert mismatched values to False.
When calling to_pandas_dataframe , there is a new option to interpret null/missing values in numeric
columns as NaN.
Added ability to check the return type of some expressions to ensure type consistency and fail early.
You can now call parse_json to parse values in a column as JSON objects and expand them into
multiple columns.
Bug fixes
Fixed a bug that crashed set_column_types in Python 3.5.2.
Fixed a bug that crashed when connecting to Datastore using an AML image.
Updates
Example Notebooks for getting started tutorials, case studies, and how -to guides.
2018-12-04: General Availability

Azure Machine Learning service is now generally available.
With this release, we are announcing a new managed compute experience through the Azure Machine Learning
Compute. This compute target replaces Azure Batch AI compute for Azure Machine Learning.
This compute target:
Is used for model training and batch inference/scoring
Is single- to multi-node compute
Does the cluster management and job scheduling for the user
Autoscales by default
Support for both CPU and GPU resources
Enables use of low -priority VMs for reduced cost
Azure Machine Learning Compute can be created in Python, using Azure portal, or the CLI. It must be created in
the region of your workspace, and cannot be attached to any other workspace. This compute target uses a Docker
container for your run, and packages your dependencies to replicate the same environment across all your nodes.
WARNING
We recommend creating a new workspace to use Azure Machine Learning Compute. There is a remote chance that users
trying to create Azure Machine Learning Compute from an existing workspace might see an error. Existing compute in your
workspace should continue to work unaffected.

Breaking changes
With this release, we are removing support for creating a VM from Azure Machine Learning. You can
still attach an existing cloud VM or a remote on-premises server.
We are also removing support for BatchAI, all of which should be supported through Azure Machine
Learning Compute now.
New
For machine learning pipelines:
EstimatorStep
HyperDriveStep
MpiStep
Updated
For machine learning pipelines:
DatabricksStep now accepts runconfig
DataTransferStep now copies to and from a SQL datasource
Schedule functionality in SDK to create and update schedules for running published pipelines
Breaking changes
SummaryFunction.N was renamed to SummaryFunction.Count .
Bug Fixes
Use latest AML Run Token when reading from and writing to datastores on remote runs. Previously, if
the AML Run Token is updated in Python, the Data Prep runtime will not be updated with the updated
AML Run Token.
Additional clearer error messages
to_spark_dataframe() will no longer crash when Spark uses Kryo serialization
Value Count Inspector can now show more than 1000 unique values
Random Split no longer fails if the original Dataflow doesn’t have a name
More information
Azure Machine Learning Data Prep SDK
Docs and notebooks
ML Pipelines
New and updated notebooks for getting started with pipelines, batch scoping, and style transfer
examples: https://aka.ms/aml-pipeline-notebooks
Learn how to create your first pipeline
Learn how to run batch predictions using pipelines
Azure Machine Learning compute target
Sample notebooks are now updated to use the new managed compute.
Learn about this compute
Azure portal: new features
Create and manage Azure Machine Learning Compute types in the portal.
Monitor quota usage and request quota for Azure Machine Learning Compute.
View Azure Machine Learning Compute cluster status in real time.
Virtual network support was added for Azure Machine Learning Compute and Azure Kubernetes Service
creation.
Rerun your published pipelines with existing parameters.
New automated machine learning charts for classification models (lift, gains, calibration, feature importance
chart with model explainability) and regression models (residuals and feature importance chart with model
explainability).
Pipelines can be viewed in Azure portal
2018-11-20
Breaking changes
azureml.train.widgets namespace has moved to azureml.widgets.
azureml.core.compute.AmlCompute deprecates the following classes -
azureml.core.compute.BatchAICompute and azureml.core.compute.DSVMCompute. The latter class will
be removed in subsequent releases. The AmlCompute class has an easier definition now, and simply
needs a vm_size and the max_nodes, and will automatically scale your cluster from 0 to the max_nodes
when a job is submitted. Our sample notebooks have been updated with this information and should
give you usage examples. We hope you like this simplification and lots of more exciting features to come
in a later release!
Learn more about the Data Prep SDK by reading reference docs.
New Features
Created a new DataPrep CLI to execute DataPrep packages and view the data profile for a dataset or
dataflow
Redesigned SetColumnType API to improve usability
Renamed smart_read_file to auto_read_file
Now includes skew and kurtosis in the Data Profile
Can sample with stratified sampling
Can read from zip files that contain CSV files
Can split datasets row -wise with Random Split (for example, into test-train sets)
Can get all the column data types from a dataflow or a data profile by calling .dtypes
Can get the row count from a dataflow or a data profile by calling .row_count
Bug Fixes
Fixed long to double conversion
Fixed assert after any add column
Fixed an issue with FuzzyGrouping, where it would not detect groups in some cases
Fixed sort function to respect multi-column sort order
Fixed and/or expressions to be similar to how pandas handles them
Fixed reading from dbfs path
Made error messages more understandable
Now no longer fails when reading on remote compute target using AML token
Now no longer fails on Linux DSVM
Now no longer crashes when non-string values are in string predicates
Now handles assertion errors when Dataflow should fail correctly
Now supports dbutils mounted storage locations on Azure Databricks
2018-11-05
Azure portal
The Azure portal for the Azure Machine Learning service has the following updates:
A new Pipelines tab for published pipelines.
Added support for attaching an existing HDInsight cluster as a compute target.
Breaking changes
*Workspace.compute_targets, datastores, experiments, images, models, and webservices are properties
instead of methods. For example, replace Workspace.compute_targets() with
Workspace.compute_targets.
Run.get_context deprecates Run.get_submitted_run. The latter method will be removed in subsequent
releases.
PipelineData class now expects a datastore object as a parameter rather than datastore_name. Similarly,
Pipeline accepts default_datastore rather than default_datastore_name.
New features
The Azure Machine Learning Pipelines sample notebook now uses MPI steps.
The RunDetails widget for Jupyter notebooks is updated to show a visualization of the pipeline.
New features
Type Count added to Data Profile
Value Count and Histogram is now available
More percentiles in Data Profile
The Median is available in Summarize
Python 3.7 is now supported
When you save a dataflow that contains datastores to a DataPrep package, the datastore information will
be persisted as part of the DataPrep package
Writing to datastore is now supported
Bug fixed
64-bit unsigned integer overflows are now handled properly on Linux
Fixed incorrect text label for plain text files in smart_read
String column type now shows up in metrics view
Type count now is fixed to show ValueKinds mapped to single FieldType instead of individual ones
Write_to_csv no longer fails when path is provided as a string
When using Replace, leaving “find” blank will no longer fail
2018-10-12
New features
Multi-tenant support when creating new workspace.
Bugs fixed
You no longer need to pin the pynacl library version when deploying web service.
New features
Added method transform_partition_with_file(script_path), which allows users to pass in the path of a
Python file to execute
2018-10-01
Version 0.1.65 includes new features, more documentation, bug fixes, and more sample notebooks.
See the list of known issues to learn about known bugs and workarounds.
Breaking changes
Workspace.experiments, Workspace.models, Workspace.compute_targets, Workspace.images,
Workspace.web_services return dictionary, previously returned list. See azureml.core.Workspace API
documentation.
Automated Machine Learning removed normalized mean square error from the primary metrics.
HyperDrive
Various HyperDrive bug fixes for Bayesian, Performance improvements for get Metrics calls.
Tensorflow 1.10 upgrade from 1.9
Docker image optimization for cold start.
Jobs now report correct status even if they exit with error code other than 0.
RunConfig attribute validation in SDK.
HyperDrive run object supports cancel similar to a regular run: no need to pass any parameters.
Widget improvements for maintaining state of drop-down values for distributed runs and HyperDrive
runs.
TensorBoard and other log files support fixed for Parameter server.
Intel(R ) MPI support on service side.
Bugfix to parameter tuning for distributed run fix during validation in BatchAI.
Context Manager now identifies the primary instance.
Azure portal experience
log_table() and log_row () are supported in Run details.
Automatically create graphs for tables and rows with 1, 2 or 3 numerical columns and an optional
categorical column.
Automated Machine Learning
Improved error handling and documentation
Fixed run property retrieval performance issues.
Fixed continue run issue.
Fixed ensembling iteration issues.
Fixed training hanging bug on MAC OS.
Downsampling macro average PR/ROC curve in custom validation scenario.
Removed extra index logic.
Removed filter from get_output API.
Pipelines
Added a method Pipeline.publish() to publish a pipeline directly, without requiring an execution run first.
Added a method PipelineRun.get_pipeline_runs() to fetch the pipeline runs that were generated from a
published pipeline.
Project Brainwave
Updated support for new AI models available on FPGAs.
Version 0.2.0 includes following features and bug fixes:
New features
Support for one-hot encoding
Support for quantile transform
Bug fixed:
Works with any Tornado version, no need to downgrade your Tornado version
Value counts for all values, not just the top three
2018-09 (Public preview refresh)

A new, refreshed release of Azure Machine Learning: Read more about this release:
https://azure.microsoft.com/blog/what-s-new -in-azure-machine-learning-service/
Next steps
Read the overview for Azure Machine Learning service.
Known issues and troubleshooting Azure Machine
Learning service
This article helps you find and correct errors or failures encountered when using the Azure Machine Learning
service.
Visual interface issues

Visual interface for machine learning service issues.
Long compute preparation time
Create new compute or evoke leaving compute takes time, may be a few minutes or even longer. The team is
working for optimization.
Cannot run an experiment only contains dataset
You might want to run an experiment only contains dataset to visualize the dataset. However, it's not allowed to
run an experiment only contains dataset today. We are actively fixing this issue.
Before the fix, you can connect the dataset to any data transformation module (Select Columns in Dataset, Edit
Metadata, Split Data etc.) and run the experiment. Then you can visualize the dataset.
Below image shows how:
SDK installation issues

Error message: Cannot uninstall 'PyYAML'
Azure Machine Learning SDK for Python: PyYAML is a distutils installed project. Therefore, we cannot accurately
determine which files belong to it if there is a partial uninstall. To continue installing the SDK while ignoring this
error, use:
pip install --upgrade azureml-sdk[notebooks,automl] --ignore-installed PyYAML
Error message: ERROR: No matching distribution found for azureml-dataprep-native
Anaconda's Python 3.7.4 distribution has a bug that breaks azureml-sdk install. This issue is discussed in this
GitHub Issue This can be worked around by creating a new Conda Environment using this command:
conda create -n <env-name> python=3.7.3
Which creates a Conda Environment using Python 3.7.3, which doesn't have the install issue present in 3.7.4.
Trouble creating Azure Machine Learning Compute

There is a rare chance that some users who created their Azure Machine Learning workspace from the Azure
portal before the GA release might not be able to create Azure Machine Learning Compute in that workspace. You
can either raise a support request against the service or create a new workspace through the Portal or the SDK to
unblock yourself immediately.
Image building failure

Image building failure when deploying web service. Workaround is to add "pynacl==1.2.1" as a pip dependency to
Conda file for image configuration.
Deployment failure
If you observe ['DaskOnBatch:context_managers.DaskOnBatch', 'setup.py']' died with <Signals.SIGKILL: 9> , change
the SKU for VMs used in your deployment to one that has more memory.
FPGAs
You will not be able to deploy models on FPGAs until you have requested and been approved for FPGA quota. To
request access, fill out the quota request form: https://aka.ms/aml-real-time-ai

Tensor Flow Automated machine learning does not currently support tensor flow version 1.13. Installing this
version will cause package dependencies to stop working. We are working to fix this issue in a future release.
Experiment Charts
Binary classification charts (precision-recall, ROC, gain curve etc.) shown in automated ML experiment iterations
are not rendering correctly in user interface since 4/12. Chart plots are currently showing inverse results, where
better performing models are shown with lower results. A resolution is under investigation.
Databricks
Databricks and Azure Machine Learning issues.
Failure when installing packages
Azure Machine Learning SDK installation fails on Azure Databricks when more packages are installed. Some
packages, such as psutil , can cause conflicts. To avoid installation errors, install packages by freezing the library
version. This issue is related to Databricks and not to the Azure Machine Learning service SDK. You might
experience this issue with other libraries, too. Example:
psutil cryptography==1.5 pyopenssl==16.0.0 ipython==2.2.0
Alternatively, you can use init scripts if you keep facing install issues with Python libraries. This approach isn't
officially supported. For more information, see Cluster-scoped init scripts.
Cancel an automated machine learning run
When you use automated machine learning capabilities on Azure Databricks, to cancel a run and start a new
experiment run, restart your Azure Databricks cluster.
>10 iterations for automated machine learning
In automated machine learning settings, if you have more than 10 iterations, set show_output to False when you
submit the run.
Widget for the Azure Machine Learning SDK/automated machine learning
The Azure Machine Learning SDK widget isn't supported in a Databricks notebook because the notebooks can't
parse HTML widgets. You can view the widget in the portal by using this Python code in your Azure Databricks
notebook cell:
displayHTML("<a href={} target='_blank'>Azure Portal: {}</a>".format(local_run.get_portal_url(),

local_run.id))
Import error: No module named 'pandas.core.indexes'

If you see this error when you use automated machine learning:
1. Run this command to install two packages in your Azure Databricks cluster:
scikit-learn==0.19.1
pandas==0.22.0
2. Detach and then reattach the cluster to your notebook.

If these steps don't solve the issue, try restarting the cluster.
FailToSendFeather
If you see a FailToSendFeather error when reading data on Azure Databricks cluster, refer to the following
solutions:
Upgrade azureml-sdk[automl_databricks] package to the latest version.
Add azure-dataprep version 1.1.8 or above.
Add pyarrow version 0.11 or above.
Azure portal
If you go directly to view your workspace from a share link from the SDK or the portal, you will not be able to view
the normal Overview page with subscription information in the extension. You will also not be able to switch into
another workspace. If you need to view another workspace, the workaround is to go directly to the Azure portal
and search for the workspace name.
Diagnostic logs
Sometimes it can be helpful if you can provide diagnostic information when asking for help. To see some logs, visit
Azure portal and go to your workspace and select Workspace > Experiment > Run > Logs.
NOTE
Azure Machine Learning service logs information from a variety of sources during training, such as AutoML or the Docker
container that runs the training job. Many of these logs are not documented. If you encounter problems and contact
Microsoft support, they may be able to use these logs during troubleshooting.
Activity logs
Some actions within the Azure Machine Learning workspace do not log information to the Activity log. For
example, starting a training run or registering a model.
Some of these actions appear in the Activities area of your workspace, however they do not indicate who initiated
the activity.
Resource quotas
Learn about the resource quotas you might encounter when working with Azure Machine Learning.
Authentication errors
If you perform a management operation on a compute target from a remote job, you will receive one of the
following errors:
{"code":"Unauthorized","statusCode":401,"message":"Unauthorized","details":
[{"code":"InvalidOrExpiredToken","message":"The request token was either invalid or expired. Please try again
with a valid token."}]}
{"error":{"code":"AuthenticationFailed","message":"Authentication failed."}}
For example, you will receive an error if you try to create or attach a compute target from an ML Pipeline that is
submitted for remote execution.
Overloaded AzureFile storage

If you receive an error
Unable to upload project files to working directory in AzureFile because the storage is overloaded , apply
following workarounds.
If you are using file share for other workloads, such as data transfer, the recommendation is to use blobs so that
file share is free to be used for submitting runs. You may also split the workload between two different
workspaces.
What happened to Azure Machine Learning
Workbench?
The Azure Machine Learning Workbench application and some other early features were deprecated and replaced
in the September 2018 release to make way for an improved architecture.
To improve your experience, the release contains many significant updates prompted by customer feedback. The
core functionality from experiment runs to model deployment hasn't changed. But now, you can use the robust
SDK and the Azure CLI to accomplish your machine learning tasks and pipelines.
Most of the artifacts that were created in the earlier version of Azure Machine Learning service are stored in your
own local or cloud storage. These artifacts won't ever disappear.
In this article, you learn about what changed and how it affects your pre-existing work with the Azure Machine
Learning Workbench and its APIs.
WARNING
This article is not for Azure Machine Learning Studio users. It is for Azure Machine Learning service customers who have
installed the Workbench (preview) application and/or have experimentation and model management preview accounts.
What changed?
The latest release of Azure Machine Learning service includes the following features:
A simplified Azure resources model.
A new portal UI to manage your experiments and compute targets.
A new, more comprehensive Python SDK.
The new expanded Azure CLI extension for machine learning.
The architecture was redesigned for ease of use. Instead of multiple Azure resources and accounts, you only need
an Azure Machine Learning service Workspace. You can create workspaces quickly in the Azure portal. By using a
workspace, multiple users can store training and deployment compute targets, model experiments, Docker images,
deployed models, and so on.
Although there are new improved CLI and SDK clients in the current release, the desktop workbench application
itself has been retired. Experiments can be managed in the workspace dashboard in Azure portal. Use the
dashboard to get your experiment history, manage the compute targets attached to your workspace, manage your
models and Docker images, and even deploy web services.
Support timeline
On January 9th, 2019 support for Machine Learning Workbench, Azure Machine Learning Experimentation and
Model Management accounts, and their associated SDK and CLI has ended.
All the latest capabilities are available by using this SDK, the CLI, and the portal.
What about run histories?

Older run histories are no longer accessible, how you can still see your runs in the latest version.
Run histories are now called experiments. You can collect your model's experiments and explore them by using
the SDK, the CLI, or the Azure portal.
The portal's workspace dashboard is supported on Microsoft Edge, Chrome, and Firefox browsers only:
Start training your models and tracking the run histories using the new CLI and SDK. You can learn how with the
Tutorial: train models with Azure Machine Learning service.
Can I still prep data?

Your pre-existing data preparation files aren't portable to the latest release because we don't have Machine
Learning Workbench anymore. But you can still prepare any size data set for modeling.
With data sets of any size, you can use the data prep package for Azure Machine Learning to quickly prepare your
data prior to modeling by writing Python code.
Will projects persist?

You won't lose any code or work. In the older version, projects are cloud entities with a local directory. In the latest
version, you attach local directories to the Azure Machine Learning service Workspace by using a local config file.
See a diagram of the latest architecture.
Much of the project content was already on your local machine. So you just need to create a config file in that
directory and reference it in your code to connect to your workspace. To continue using the local directory
containing your files and scripts, specify the directory's name in the 'experiment.submit' Python command or using
the az ml project attach CLI command. For example:
run = exp.submit(source_directory=script_folder,
script='train.py', run_config=run_config_system_managed)
Create a workspace to get started.
What about my registered models and images?

The models that you registered in your old model registry must be migrated to your new workspace if you want to
continue to use them. To migrate your models, download the models and re-register them in your new workspace.
The images that you created in your old image registry cannot be directly migrated to the new workspace. In most
cases, the model can be deployed without having to create an image. If needed, you can create an image for the
model in the new workspace. For more information, see Manage, register, deploy, and monitor machine learning
models.
What about deployed web services?

Now that support for the old CLI has ended, you can no longer redeploy models or manage the web services you
originally deployed with your Model Management account. However, those web services will continue to work for
as long as Azure Container Service (ACS ) is still supported.
In the latest version, models are deployed as web services to Azure Container Instances (ACI) or Azure Kubernetes
Service (AKS ) clusters. You can also deploy to FPGAs and to Azure IoT Edge.
Learn more in these articles:
Where and how to deploy models
Tutorial: Deploy models with Azure Machine Learning service
What about the old SDK and CLI?

Yes, they'll continue to work until January. See the preceding timeline. We recommend that you start creating your
new experiments and models with the latest SDK or CLI.
By using the new Python SDK in the latest release, you can interact with Azure Machine Learning service in any
Python environment. Learn how to install the latest SDK. You can also use the updated Azure Machine Learning
CLI extension with the rich set of az ml commands to interact with the service in any command-line environment,
including Azure Cloud Shell.
What about Visual Studio Code Tools for AI?

In this latest release, the extension was renamed to Azure Machine Learning for Visual Studio Code and has been
expanded and improved to work with the preceding new features.
What about domain packages?
The domain packages for computer vision, text analytics, and forecasting can't be used with the latest version of
Azure Machine Learning. However, you can still build and train computer vision, text, and forecasting models with
the latest Azure Machine Learning Python SDK. To learn how to migrate pre-existing models built by using the
computer vision, text analytics, and forecasting packages, contact AML -Packages@microsoft.com.
Next steps
Learn about the latest architecture for Azure Machine Learning service.
For an overview of the service, read What is Azure Machine Learning service?.
Create your first experiment with the two-part tutorial to setup environment and workspace and train your first
model
For a more in-depth experience of this workflow, follow the full-length tutorial that contains detailed steps for
training and deploying models with Azure Machine Learning service.

Azure ML Documentation

Încărcat de

Informații document

Descriere originală:

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Azure ML Documentation

Încărcat de

Drepturi de autor:

Formate disponibile

Contents

Azure Machine Learning service Documentation

What is machine learning?

What is Azure Machine Learning service?

What can I do with Azure Machine Learning service?

How does Azure Machine Learning service differ from Studio?

AZURE MACHINE LEARNING SERVICE:

Generally available (GA) In preview

Modules for interface Many Initial set of popular modules

Automated model training and No Not yet in visual interface.

Tools for Azure Machine Learning

Workspace name Enter a unique name that identifies your workspace. In

Subscription Select the Azure subscription that you want to use.

Resource group Use an existing resource group in your subscription or

7. After you are finished configuring the workspace, select Create.

When the process is finished, a deployment success message appears.

4. Provide a name for your VM.

Connect workspace and create experiment

from azureml.core import Workspace

from azureml.core import Experiment

from sklearn.datasets import load_diabetes

from sklearn.linear_model import Ridge

for alpha in alphas:

model_name = "model_alpha_" + str(alpha) + ".pkl"

The above code accomplishes the following:

NAME WORKSPACE REPORT PAGE DOCS PAGE

diabetes-experiment your-workspace-name Link to Azure Portal Link to Documentation

View training results in portal

for run in experiment.get_runs():

print("Best run_id: " + minimum_rmse_runid)

Best run_id: 864f5ce7-6729-405d-b457-83250da99c80

2. From the list, select the VM.

Set up your development environment

# check core SDK version number

from azureml.core import Experiment

exp = Experiment(workspace=ws, name=experiment_name)

Create or attach an existing compute target

# choose a name for your cluster

# create the cluster

# For a more detailed view of current AmlCompute status, use get_status()

data_folder = os.path.join(os.getcwd(), 'data')

You will see output similar to this: ('./data/test-labels.gz', <http.client.HTTPMessage at 0x7f40864c77b8>)

Display some sample images

# make sure utils.py is in the same directory as this code

A random sample of images displays:

You now have everything you need to start training a model.

Train on a remote cluster

Create a training script

from sklearn.linear_model import LogisticRegression

from azureml.core import Run

# load train and test set into numpy arrays

# get hold of the current run

print('Train a logistic regression model with regularization rate of', args.reg)

print('Predict the test set')

# calculate accuracy on the prediction

run.log('regularization rate', np.float(args.reg))

Notice how the script gets data and saves models:

from azureml.train.sklearn import SKLearn

Submit the job to the cluster

Monitor a remote run

from azureml.widgets import RunDetails

If you need to cancel a run, you can follow these instructions.

run.wait_for_completion(show_output=False) # specify True for a verbose log