Documente Academic
Documente Profesional
Documente Cultură
Chapter 1
Rosenblatt's Perceptron
Dr. Vincent A. Cassella
Catholic University of America
Material Acknowledgement
Neural Networks and Learning Machines, Third Edition
Simon Haykin
Perceptron
Figure 1.1 Signal-flow graph of the perceptron.
0, ()=1
<0, ()=-1
Decision Threshold
Perceptron (Hyperplane)
Figure 1.2 Illustration of the hyperplane (in this example,
a straight line) as decision boundary for a
two-dimensional, two-class pattern-classification problem.
Decision Threshold
Class 2
Class 1
BREAK
Neural Networks and Learning Machines, Third Edition
Simon Haykin
Perceptron
works
Perceptron
doesn't
work
Correct Classifications
Wrong Classifications
c11=cost of bagging a good apple, c22=cost of trashing a rotten apple, c21=cost of
trashing a good apple, c12=cost of bagging a rotten apple Note: c21>c11 and c12>c22
Neural Networks and Learning Machines, Third Edition
Simon Haykin
Correct Classifications
Class 1 is defined to
minimize the overall
risk or cost.
Wrong Classifications
Fixed risk.
End of Lecture 2
Neural Networks and Learning Machines, Third Edition
Simon Haykin
Figure 1.5 Two equivalent implementations of the Bayes classifier: (a) Likelihood ratio test, (b) Log-likelihood ratio test.
Class 2
Decision Boundary
Define:
and
The perceptron provides perfect classification when the classes are linearly
separable. However, Gaussian distributions overlap and are not linearly
separable. Since, classification error must occur, the goal is to minimize the risk.
Rosenblatt's perceptron is capable of doing that when the covariance matrices
are the same.
Neural Networks and Learning Machines, Third Edition
Simon Haykin
Study Material
What is the structure of a Rosenblatt perceptron and what is its goal? Is the
decision threshold of a Rosenblatt perceptron always a hyperplane or line? What
type of data is the Rosenblatt perceptron capable of perfectly classifying? How
are perceptron weights adapted using the Perceptron Convergence Theorem
(PCT)? Why does the PCT work? How are the weights initialized with the PCT?
What does the Bayes classifier minimize? What are the four costs considered
when forming the Bayes classifier? What is the ultimate test that is formed from
the Bayes classifier? How do you construct the Likelihood Ratio Test (LRT) and
what are all the variables involved? Can log(LRT) give a different result than
LRT? What is the special class of Gaussian data that enables a Rosenblatt
perceptron to duplicate Bayes classification? Can a Rosenblatt perceptron
perfectly classify Gaussian data? Can any classifier perfectly classify Gaussian
data?