Sunteți pe pagina 1din 34

Machine Learning with MATLAB

U. M. Sundar
Senior Application Engineer
MathWorks India

© 2013 The MathWorks, Inc.1


Agenda

 Machine Learning Overview 0.9

0.8

0.7

 Machine Learning with MATLAB 0.6

0.5

– Unsupervised Learning
0.4

0.3

0.2

 Clustering 0.1

– Supervised Learning
-0.1 0 0.1 0.2 1 0.3 0.4 0.5 0.6
Group1
0.9
Group2
0.8 Group3

 Classification 0.7
Group4
Group5
0.6
Group6

 Regression 0.5 Group7


Group8
0.4

0.3

 Learn More 0.2

0.1

0
-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6

2
Machine Learning Overview
Types of Learning, Categories of Algorithms

Type of Learning Categories of Algorithms

Unsupervised
Clustering
Learning

Group and interpret


Machine data based only
Learning on input data

Classification

Supervised
Learning

Regression
Develop predictive
model based on both
input and output data

3
Machine Learning
When and where it is used?

 When to use it
– Predict a future outcome based on
 Historical data (many variables)
 Specific patterns
– Define a System that is
 Based on inputs and outputs from the system
 complex to define using governing equations
(e.g., black-box modeling)

 Examples
AAA 93.68% 5.55% 0.59% 0.18% 0.00% 0.00% 0.00% 0.00%

– Pattern recognition (speech, images) AA 2.44%

A 0.14%
92.60%

4.18%
4.03%

91.02%
0.73%

3.90%
0.15%

0.60%
0.00%

0.08%
0.00%

0.00%
0.06%

0.08%


BBB 0.03% 0.23% 7.49% 87.86% 3.78% 0.39% 0.06% 0.16%
Financial algorithms (credit scoring, algo trading) BB 0.03% 0.12% 0.73% 8.27% 86.74% 3.28% 0.18% 0.64%

B 0.00% 0.00% 0.11% 0.82% 9.64% 85.37% 2.41% 1.64%

– Energy forecasting (load, price) CCC 0.00%

D 0.00%
0.00%

0.00%
0.00%

0.00%
0.37%

0.00%
1.84%

0.00%
6.24%

0.00%
81.88%

0.00%
9.67%

100.00%


AAA AA A BBB BB B CCC D

Biology (tumor detection, drug discovery)

4
Basic Concepts in Machine Learning

 Start with an initial set of data

 “Learn” from this data


1
– “Train” your algorithm Group1
0.9
with this data Group2
0.8 Group3
Group4
0.7
Group5
 Use the resulting model 0.6
Group6

to predict outcomes 0.5 Group7


Group8
for new data sets 0.4

0.3

0.2

0.1

0
-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6

5
Machine Learning Process

Exploration Modeling Evaluation Deployment

6
Exploratory Data Analysis

 Gain insight from visual examination


– Identify trends and interactions
– Detect patterns
– Remove outliers
– Shrink data 40

MPG
– Select and pare predictors 20

Displacement Acceleration
– Feature transformation 20

10
400

200

Weight 4000

2000
Horsepower

200
150
100
50
20 40 10 20 200 400 2000 4000 50 100150200
MPG Acceleration Displacement Weight Horsepow er

7
Data Exploration
Interactions Between Variables
8
4
40 6 6
4
MPG

4
8
20
6
3 4
8
Displacement Acceleration

20
2 2
10

Coordinate Value
400

f(t)
1 0
200
0 -2
4000
Weight

-1 -4
2000
Horsepower

200
150 -2
-6
100
50
-3
20 40 10 20 200 400 2000 4000 50 100150200 MPG Acceleration Displacement Weight Horsepower -8
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
MPG Acceleration Displacement Weight Horsepow er
t

Plot Matrix by Group Parallel Coordinates Plot Andrews’ Plot

chevrolet chevelle malibu buick skylark 320 plymouth satellite

chevrolet chevelle malibu


buick skylark 320 plymouth satellite

amc rebel sst ford torino ford galaxie 500


amc rebel sst ford torino ford galaxie 500

chevrolet impala plymouth fury iii pontiac catalina


chevrolet impala plymouth fury iii pontiac catalina

Glyph Plot Chernoff Faces


8
Unsupervised Learning
Clustering
K-means,
Fuzzy K-means

Hierarchical
Unsupervised
Clustering
Learning
Neural Network

Group and interpret


Machine data based only Gaussian
Learning on input data Mixture

Classification

Supervised
Learning

Regression

9
Dataset We’ll Be Using

 Cloud of randomly generated points

– Each cluster center is


1
randomly chosen inside Group1
0.9
specified bounds Group2
0.8 Group3
Group4
0.7
– Each cluster contains 0.6
Group5

the specified number Group6


Group7
0.5
of points per cluster Group8
0.4

0.3
– Each cluster point 0.2
is sampled from a
0.1
Gaussian distribution
0
-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6

– Multi-dimensional dataset

10
Clustering
Overview
1

 What is clustering? 0.9

– Segment data into groups, 0.8

based on data similarity 0.7

0.6

0.5
 Why use clustering?
0.4
– Identify outliers
0.3
– Resulting groups may be 0.2
the matter of interest 0.1

0
-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6

 How is clustering done?


– Can be achieved by various algorithms
– It is an iterative process (involving trial and error)

11
Clustering: K – Means Clustering Statistics Toolbox

 K-means is a partitioning method

 Partitions data into k mutually


exclusive clusters

 Each cluster has a


centroid (or center)
– Sum of distances from
all objects to the center
is minimized

12
Clustering: Neural Networks

 Networks are comprised of one or more layers Transfer


Weights
function
 Outputs computed by applying a nonlinear
transfer function with weighted sum of Input Output
inputs variables Variable

 Trained by letting the network Bias


continually adjust itself
to new inputs (determines weights)

 Interactive apps for easily creating and training networks

 Multi-layered networks created by cascading


(provide better accuracy)

 Example architectures for clustering:


– Self-organizing maps
– Competitive layers

13
Clustering : Gaussian Mixture Models

Statistics Toolbox
 Good when clusters have different
sizes and are correlated

 Assume that data is drawn


from a fixed number K
of normal distributions
20

10

1
0 0.8
1
0.8 0.6
0.6 0.4
0.4
0.2
0.2
0 0

14
Cluster Analysis
Summary

 Segments data into groups, based on data similarity

 No method is perfect K-means,


Fuzzy K-means
(depends on data)
Hierarchical
 Process is iterative; Clustering
explore different algorithms Neural Network

Gaussian
 Beware of local minima Mixture
(global optimization can help)

15
Model Development Process

Exploration Modeling Evaluation Deployment

16
Supervised Learning
Classification for Predictive Modeling

Unsupervised
Learning
Decision Tree

Machine
Learning Ensemble
Method
Classification
Neural Network
Supervised
Learning
Support Vector
Machine
Develop predictive
model based on both
input and output data

17
Classification
Overview 1
Group1
0.9
Group2

 What is classification? 0.8 Group3


Group4
0.7
– Predicting the best group for each point 0.6
Group5
Group6

– “Learns” from labeled observations 0.5 Group7


Group8

– Uses input features


0.4

0.3

0.2

 Why use classification? 0.1

– Accurately group data never seen before


0
-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6

 How is classification done?


– Can use several algorithms to build a predictive model
– Good training data is critical

18
Classification - Decision Trees Statistics Toolbox

 Builds a tree from training data


– Model is a tree where each node is a
logical test on a predictor

 Traverse tree by comparing


features with threshold values

 The “leaf” of the tree


specifies the group

19
Classification - Ensemble Learners Statistics Toolbox
Overview
1.5
group1

 Decision trees are “weak” learners group2


group3
1

– Good to classify data used to train


group4
group5
group6

– Often not very good with new data group7

x2
0.5
group8

– Note rectangular groups


0

 What are ensemble learners? -0.5


-0.4 -0.2 0 0.2 0.4
x1
0.6 0.8 1 1.2 1.4 1.6

– Combine many decision trees to create a


“strong” learner
– Uses “bootstrapped aggregation”

 Why use ensemble methods?


– Classifier has better predictive power
– Note improvement in cluster shapes

20
Classification - Support Vector Machines
Overview
Statistics Toolbox
(as of R2013a)
 Good for modeling with complex
boundaries between groups 4
1

– Can be very accurate


2
3 Support Vectors

– No restrictions on the predictors 2

 What does it do? 0

– Uses non-linear “kernel” to -1

calculate the boundaries


-2

– Can be computationally intensive


-3 -2 -1 0 1 2 3

 Version in Statistics Toolbox only


classifies into two groups

21
K-Nearest Neighbor Classification Statistics Toolbox

 One of the simplest classifiers

 Takes the K nearest points 1.5


group1

from the training set, and 1


group2
group3
group4

chooses the majority class group5


group6

of those K points group7

x2
0.5
group8

 No training phase – all the 0

work is done during the -0.5


-0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

application of the model x1

22
Classification
Summary

Decision Tree
 No absolute best method
Ensemble
Method
Classification
 Simple does not
Neural Network
mean inefficient
Support Vector
Machine
 Watch for overfitting
– Decision trees and neural networks may overfit the noise
– Use ensemble learning and cross-validation

 Parallelize for speedup

23
Supervised Learning
Regression for Predictive Modeling

Unsupervised
Learning

Machine
Learning

Supervised Linear
Learning

Regression Non-linear
Develop predictive
model based on both
input and output data Non-parametric

24
Regression Statistics Toolbox
Curve Fitting Toolbox

 Why use regression?


– Predict the continuous response
for new observations

 Type of predictive modeling


– Specify a model that describes
Y as a function of X
– Estimate coefficients that
minimize the difference
between predicted and actual

 You can apply techniques from earlier sections with


regression as well
25
Linear Regression

 Y is a linear function of the regression coefficients

 Common examples:

Straight line 𝑌 = 𝐵0 + 𝐵1𝑋1

Plane 𝑌 = 𝐵0 + 𝐵1𝑋1 +𝐵2𝑋2

Polynomial 𝑌 = 𝐵0 + 𝐵1𝑋13 + 𝐵2𝑋12 +𝐵3𝑋1

Polynomial
𝑌 = 𝐵0 + 𝐵1𝑋12 + 𝐵2(𝑋1 ∗ 𝑋2) + 𝐵3 𝑋22
with cross terms

26
Nonlinear Regression

 Y is a nonlinear function of the regression coefficients

 Syntax for formulas:

y ~ b0 + b1*cos(x*b3) +
Fourier Series 𝑏0 + 𝑏1 cos 𝑏3𝑋 + 𝑏2 sin 𝑏3𝑋
b4*sin(x*b3)

Exponential Growth 𝑁 = 𝑁0𝑒


@(b,t)(b(1)*exp(b(2)*t)
𝑏0
@(b,t)(1/(b(1) + exp(-
Logistic Growth 𝑃 𝑡 =
1 + 𝑏1 𝑒 −𝑘𝑡
b(2)*x)))

27
Generalized Linear Models

 Extends the linear model


– Define relationship between model and response variable
– Model error distributions other than normal

 Logistic regression
– Response variable is binary (true / false)
– Results are typically expressed as an odd’s ratio

 Poisson regression
– Model count data (non-negative integers)
– Response variable comes from a Poisson distribution

28
Machine Learning with MATLAB

 Interactive environment
– Visual tools for exploratory data analysis
– Easy to evaluate and choose best algorithm
– Apps available to help you get started
(e.g,. neural network tool, curve fitting tool)

 Multiple algorithms to choose from


– Clustering
– Classification
– Regression

29
Learn More : Machine Learning with MATLAB
http://www.mathworks.com/discovery/
machine-learning.html

Data Driven Fitting Classification Regression


with MATLAB with MATLAB with MATLAB

Multivariate Classification in Electricity Load and Credit Risk Modeling with


the Life Sciences Price Forecasting MATLAB

30
MathWorks India – Services and Offerings

Local website:
www.mathworks.in
Technical Support India:
Consulting www.mathworks.in/
myservicerequests

Product Training: Customer Service for non-technical


www.mathworks.in/training questions: info@mathworks.in

Application
Engineering

31
Scheduled Public Training for Sep–Dec 2013
Course Name Location Training dates
Statistical Methods in MATLAB Bangalore 02- 03 Sep 2013
MATLAB based Optimization Techniques Bangalore 04 Sep 2013
Physical Modeling of Multi-Domain Systems using Simscape Bangalore 05 Sep 2013
Delhi 23-25 Sep 2013
Pune 07-09 Oct 2013
MATLAB Fundamentals Bangalore 21-23 Oct 2013
Web based 05- 07 Nov 2013
Chennai 09-11 Dec 2013
Delhi 26-27 Sep 2013
Pune 10-11 Oct 2013
Simulink for System and Algorithm Modeling Bangalore 24-25 Oct 2013
Web based 12-13 Nov 2013
Chennai 12-13 Dec 2013
MATLAB Programming Techniques Bangalore 18-19 Nov 2013
MATLAB for Data Processing and Visualization Bangalore 20 Nov 2013
MATLAB for Building Graphical User Interface Bangalore 21 Nov 2013
Generating HDL Code from Simulink Bangalore 28-29 Nov 2013
Email: training@mathworks.in URL: http://www.mathworks.in/services/training Phone: 080-6632-600032
MathWorks Certification Program- for the first
time in India!

MathWorks Certified MATLAB Associate Exam

Why certification?
 Validates proficiency with MATLAB
 Can help accelerate professional growth
 Can help increase productivity and project success and thereby
prove to be a strategic investment

 Certification exam administered in English at MathWorks facilities


in Bangalore on Nov 27,2013

Email: training@mathworks.in URL: http://www.mathworks.in/services/training Phone: 080-6632-600033


MathWorks India Contact Details MathWorks India Private Limited
Salarpuria Windsor Building
Third Floor,
No.3 Ulsoor Road
Bangalore - 560042, Karnataka
India

URL: http://www.mathworks.in

E-mail: info@mathworks.in

Technical Support: www.mathworks.in/myservicerequests

Tel: +91-80-6632 6000

Fax: +91-80-6632 6010

Thank You for Attending


Talk to Us – We are Happy to Support You 34

S-ar putea să vă placă și