Documente Academic
Documente Profesional
Documente Cultură
Chapter 5
Artificial Neural
Networks (ANNs)
Introduction to ANN
ANN models and examples of
application
Simple neural networks: McCulloch-
Pitts Neuron, Hebb, Perceptrons
Objectives
The sub-topic has been organized to have the following
objectives:
To understand the broad concept of artificial
intelligence and artificial neural networks.
To know the possible applications of artificial neural
networks (ANN).
To understand the capabilities and limitations of ANN.
To understand the underlying concepts of several ANN
paradigms.
To be familiar with at least one ANN software for the
development of ANN applications.
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global 3
Programming language
literature: x1 a1 O1
connectionist models x2 a2 O2
parallel distributed
processors
xj
1
Wj k
ak
Wkm
n
Om
Another Definition
Neuroscience
Psychology
Philosophy
Computer Science
Mathematics
Electrical/Electronics
Dendrites
Soma
Nucleus Axon
Synapse
AN ARTIFICIAL NEURON
oi wji
wji
oi f(netj) oj
wji
oi
Biological Neuron
Artificial Neuron
x1
W11 sum = W 11x + W 12x +...+ W1NxN
1 2
x2
W12
x3 f()
S
W13
W1jxj O
1
W1N
xN
Neural Networks
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
Pattern recognition
Pattern matching
Pattern classification
Pattern completion
Prediction
Clustering
Decision making
Control
??
Dr. Hafis SKEM4173 innovative entrepreneurial global 12
14/11/2017
Bintangor
Durian
Nyatoh
Ramin
Plant Optimization
INPUTS LAB
PLANT
F,T,P
CA,CB
On-Line Outputs RESULTS
TIME (2-4 Hours)
MODEL
DELAY
REAL-TIME PREDICTION
ON-LINE ANALYSIS
0?
Pen
Pen D? j8?2
Image
trajectory
trajectory
Recognition technologies
six
x2 a2 O2
xj ak Om
1 n
Wj k Wkm
Feedforward
All signals flow in one direction only, i.e. from lower
layers (input) to upper layers (output).
Feedback
Signals from neurons in upper layers are fed back to
either its own or to neurons in lower layers.
Cellular
Connected in a cellular manner .
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global 21
In the end when the error becomes very small, the ANN is
assumed to have learned the task and training is stopped.
Output
Input - +
Target
Patterns
Learning configuration
Competitive
layer Winner
Neuron
Adaptation of weights
Input layer is measured by an
auxilliary cost function
Input Patterns
ANN Classifications
NN Model
Feedforward Feedback
Least Mean Square
Backpropagation
Supervised
Reinforcement Recurrent
Learning Backpropagation
Fuzzy ARTMAP
GRNN
Adaptive Resonance
Unsupervised
ANN Performance
ANN PERFORMANCE
000 = MOUSE
001 = RABBIT
010 = COW
S b xi wi
1 b
W1
S
x1
y
W2
x2
34
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
o
q
b x1w1 x2 w2 q
Dr. Hafis SKEM4173 innovative entrepreneurial global 35
Input
Pattern Output
YES
NO
36
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
38
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
1 if s >= q
f(s) =
0 if s < q
S
f( )
W13 1
x3 W1jxj Oj
o
q
W1N N
xN
Oj = f ( SW
i =1
ij xj )
39
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
x1 x2 y
q, threshold for y is 2
1 1 1
1 0 0
0 1 0
0 0 0
x1 W=1
S Wij
1
o y
W=1 2
x2
y= f ( SW ij xj )
x1 W=2
S Wij
1
2
o
y
W=2
x2
y= f ( SW ij xj )
x1 x2 y
1 1 1
1 0 0
0 1 0
0 0 1
43
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
wi 0 i 1 to n
Step 1. For each input training vector and target output pair:
ui (u1, u2, u3,); t
do Steps 2-4.
Step 2. Set activations for input units :
xi ui i 1 to n
Step 3. Set activation for output unit :
yt
44
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
wnew wold xy
This is often written in terms of the weight change, w as
w xy
and
wnew wold w
0 0 0
x2
Using the Hebb network, solve the OR problem above and show the
results of the weights adaptation for each pattern over 1 epoch.
Fill up the following table.
Iter# x1 x2 t y w1 w2 b
0 0 0
46
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
wi ( new ) wi ( old ) Tx i
47
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
xi ui .
Step 4. Compute response of output unit:
s b xi wi
i
1 if s q
q is a threshold value
y 0 if -q s q
assigned between 0 and 1
1 if s < - q
48
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
Step 5 Update weights and bias if an error occurred for this pattern:
If y t
w i new wi old txi .
b new b old t.
else
w i new wi old ,
b new b old .
Exercise 5.2a
1
x1 x2 t b
1 1 1 x1 W1
1
-1
-1
1
-1
-1
W2
S y
-1 -1 -1
x2
Using the Perceptron network, solve the AND problem above (using
bipolar activations) and show the results of the weights adaptation for
each pattern over 1 epoch.
Choose all initial weights to be -1, =0.5, and q=0.2
Fill up the following table.
Iter# x1 x2 s y t w1 w2 b
Exercise 5.2b
x1 x2 x3 t
-1 -1 -1 -1 1
b
-1 -1 1 1
-1 1 -1 1 x1 W1
-1 1 1 1 y
x2
f
W2
1 -1 -1 1 W3
1 -1 1 1 x3
1 1 -1 1
1 1 1 1
Using the Perceptron network (3-input), solve the OR problem above
(using bipolar activations) and show the results of the weights
adaptation for each pattern over 1 epoch.
Choose all initial weights to be -0.5, =0.25, and q=0.1
Fill up the following table. 51
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
52
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
y = t, hence No Adaptation
w1(new) = w1(old) = -0.25
w2(new) = w2(old) = -0.25
w3(new) = w3(old) = -0.25
b(new) = b(old) = 0.75
y = t, hence No Adaptation
w1(new) = w1(old) = -0.25
w2(new) = w2(old) = -0.25
w3(new) = w3(old) = -0.25
b(new) = b(old) = 0.75
53
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
y = t, hence No Adaptation
w1(new) = w1(old) = -0.25
w2(new) = w2(old) = -0.25
w3(new) = w3(old) = -0.25
b(new) = b(old) = 0.75
y = t, hence No Adaptation
w1(new) = w1(old) = -0.25
w2(new) = w2(old) = -0.25
w3(new) = w3(old) = -0.25
b(new) = b(old) = 0.75
54
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
y = t, hence No Adaptation
w1(new) = w1(old) = -0.25
w2(new) = w2(old) = -0.25
w3(new) = w3(old) = -0.25
b(new) = b(old) = 0.75
y = t, hence No Adaptation
w1(new) = w1(old) = -0.25
w2(new) = w2(old) = -0.25
w3(new) = w3(old) = -0.25
b(new) = b(old) = 0.75
55
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
56
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
Epoch #1
x1 x2 x3 s y t w1 w2 w3 b
57
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
Epoch #2
x1 x2 x3 s y t w1 w2 w3 b
- - - 0 0 0 1
-1 -1 -1 1 1 -1 0.25 0.25 0.25 0.75
-1 -1 1 0.5 1 1 0.25 0.25 0.25 0.75
-1 1 -1 0.5 1 1 0.25 0.25 0.25 0.75
-1 1 1 1 1 1 0.25 0.25 0.25 0.75
1 -1 -1 0.5 1 1 0.25 0.25 0.25 0.75
1 -1 1 1 1 1 0.25 0.25 0.25 0.75
1 1 -1 1 1 1 0.25 0.25 0.25 0.75
1 1 1 1.5 1 1 0.25 0.25 0.25 0.75
58
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
Epoch #3
x1 x2 x3 s y t w1 w2 w3 b
59
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
Epoch #4
x1 x2 x3 s y t w1 w2 w3 b
60
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global
14/11/2017
Epoch #5
x1 x2 x3 s y t w1 w2 w3 b
61
Dr. Hafis SKEM4173
SKEM4173 innovative entrepreneurial global