Machine Learning

20/10/2017
Introduction to ML

 The study of algorithms and systems that improve their

performance with experience
 Experience = data / measurements / observations
 Deals with the construction and study of systems that can
learn from data
Types of Learning Paradigms

 Supervised Learning
 Unsupervised Learning
Supervised Learning

 A training set of examples with the correct responses (targets)

are provided and, based on this training set, the algorithm
generalizes to respond correctly to all possible inputs.
 This is also called learning from exemplars.
 Example:
 Would like to be able to predict an outcome of interest y for an object x
 Learn function y=f(x)
 We are given data with pairs {<xi,yi>: i=1,…,n}, xi: representation of an
object, yi: representation of a known outcome
 Learn the function y=f(x) that generalizes from the data the “best” (has
min avg error)
Supervised Learning Tasks

 Regression
 Classification
Regression
 Fit a mathematical function describing a curve, so that the
curve passes as close as possible to all of the data points
 A problem of function approximation or interpolation, working
out the value between values that we know
 A measure of the relation between the mean value of one
variable and corresponding values of other variables
 Is a statistical process for estimating the relationships among
variables
 Regression means to predict the output value using training
data
 If it is a real number/continuous, then it is regression problem
 E.g. regression to predict the house price from training data
Classification

 The classification problem consists of taking input vectors and

deciding which of N classes they belong to, based on training
from exemplars of each class
 Predict class from observations
 Classification means to group the output into a class
 If it is discrete/categorical variable, then it is classification
problem
 E.g. classification to predict the type of tumor i.e. benign or
malignant using training data
Neural Networks

 An Artificial Neural Network (ANN) is an information

processing paradigm that is inspired by the way biological
nervous systems, such as the brain, process information
 ANNs learn by example
 Provide a robust approach to approximating real-valued,
discrete-valued, and vector-valued target functions
 Can be used to extract patterns and detect trends that are
too complex to be noticed by either humans or other
computer techniques
ANN Representation – Feed-forward
Networks
 Many neurons can be joined together by communication
links to carry out complex computations
 Can be described as a graph whose nodes are the neurons
and each (directed) edge in the graph links the output of
some neurons to the input of another neuron
 Feed-forward ANNs allow signals to travel one way only, from
input to output
 There is no feedback (loops) in feed-forward ANNs
Network Layers

 The commonest type of ANN consists of 3 groups/layers/units

 A layer of input units is connected to a layer of hidden, which
is connected to a layer of output units
 The activity of the input units represents the raw information
that is fed into the network
 The activity of each hidden unit is determined by the activities
of the input units and the weights on the connections
between the input and the hidden units
 The behavior of the output units depends on the activity of
the hidden units and the weights between the hidden and
output units
Transfer Function

 The behavior of an ANN depends on both the weights and

the input-output function that is specified for the units
 3 categories of transfer function:
 Linear: the output activity is proportional to the total weighted
output
 Threshold: the output is set at one of two levels, depending on
whether the total input is greater than or less than some
threshold value
 Sigmoid: the output varies continuously but not linearly as the
input changes
Continue with..
 Neural Networks Learning
 Model Selection
 Support Vector Machines (SVMs)
 Kernels
 Unsupervised Learning - Clustering (K-Means algorithm)
 Dimensionality Reduction (Principal Component Analysis)
References

 https://www.coursera.org/learn/machine-learning
 https://books.google.gr/books/about/Pattern_Recognition_and_Machine_
Learning.html?id=HL4HrgEACAAJ&redir_esc=y
 http://www.cs.cmu.edu/~tom/mlbook.html
 https://books.google.gr/books/about/Understanding_Machine_Learning.ht
ml?id=ttJkAwAAQBAJ&redir_esc=y