During first stage, which is the initialization of weights ,some small random values are assigned .

During feed
forward stage each input unit receives an input signal and transmits to each of the hidden units
z1..zp.Then each hidden unit calculates the activation function and sends its signals Zj to each output
unit. The output unit calculates the activation function to form the response of the net for the given input
pattern
During back propagation of errors, each output compares its computed activation Yk with its target value T k to
determine the associated error for that pattern with that unit . Based on error ,the factor k (k=1,2,.m)
is computed and is used to distribute, the error at output unit Y, back for all units, in the previous layer.
Similarly the factor j(j=1,2.m)is computed for each hidden unit Zj.
During final stage ,the weight and bias are updated using the factor delta and activation.
INITIALIZATION OF WEIGHTS:

Step1: initialize the weights as small random values.

Step2: if stopping condition fail, do as steps 3-10.
Step3: for each training pair, do steps 4-9.

FEED FORWARD:
Step4:each input unit receives the input signal x, and transmits to all units in the hidden layer.
Step5: each hidden unit sums its weighted input signals

step6: Zinj=Voj+ xivij

Applying activation function ZJ=f(Zin) and to all units in the output.
Back propagation of errors:
Step7:Each output unit (YK,K=1,2.) receives target pattern corresponding an input pattern, error information term
is calculated as
k=(TK-YK).f(Y-jnk)
Step8:each hidden unit (Zij=1,2n) sums its delta inputs from output layer.
-inj= jwjk
The error information term is calculated as
j= -inj.f(Z-inj)

Updates of weights and biases :

Step9:each output unit (YK=K=1,20..) updates its bias and weights
The weight correction term is given by W jk= k zj
The bias correction term is given by wok= k
Wjk(new) = Wjk(old)+wjk
W0k(new) = Wok(old)+wok

Each hidden unit (Zj;j=1,2.p) updates its bias and weights (i=0,1,2n)
.the weight correction term is vij = j xi
the bias correction term is Voj= j
VJJ(new) = VJJ(old) + Vjj ; Voj (new) = Voj (old) + Voj
Step 10: Test the stopping condition
If the total error reduces to an acceptable value, is the stopping condition

MERITS OF BPA:
1.The mathematical formula present here, can be applied to any network
2.The computation is reduced if the weight chosen ate small at the beginning.

DEMERTIS OF BPA:
1.Slow convergence
2.Local minima problem
3.Scaling
4.Need for teacher

APPLICATIONS:
Image compression
Data compression
Control problems
Nor linear simulation
Fault detection problems
Face recognition
Load forecasting problems
 LEARNING RATE:

Learning rate parameter N is a small positive constant

It is used to adjust the weights in such away that all units learn nearly at some rate.
The order of the values for N depends on the variance of the input data
If N is large ,the speed of learning is high. But it results large random fluctuations in weight space which
in turn causes unstable situations:i.e new weights may not converge
So the learning rate parameter should be different for different weights
N should be small at O /P layer ,large at hidden layer.so that we can see the net change in weights remains
same for all layers
The change in N is made proportional to VE gradient of the instantaneous error with respect to N
njj(max) = -. e^2(m)/njj(m) ; where = proportionality constant.

MOMENTUM:

It is a small positive constant.

It is used to increase learning rate
It lies between 0 to 1
It helps in reducing the effects of local minimum of the error surface
It is also used to accelerate the convergence of error propogation algorithm. i.e. this method is useful
where both the weight used to determine the weight change for the current step
The change in weights using the momentum is :
wjj(m) = wjj(m-1)+n j(m).i(m)
 LIMITATIONS:
Learning rate places the upper limit on the amount by which the weight can be changed
It causes weight changes to be in a direction that could increase the error
 LMS:
The variables and parameters used in LMS algorithm are
X[N]=i/p vector
W[n]=weight vector
B[n]=bias
Y[n]=Actual response
D[n]=desired response
N=learning rate parameter
1.Initialization :set w[0]=0
2.activation:Activate perception by applying input
3.Find actual response of perceptron: y[n]=sgn [ w^t(n).x(n)]
4.adjust weight vector :If d[n] and y[n] are different then
W(n+1)=w(n)+n[d(n)-y[n])x[n] where d(n)= +1; x(n)
Continuation: Increment step by 1 and go to activation step

CHARACTERISTICS:
1.Can operate in unknown environment
2.can operate in stationary and non stationary environment
3.minimize instantaneous square error
4.stochastic
5.Approximate
Advantages:
1.simplicity in implementation
2.stable and robust performance against different signal condition
 Disadvantages:
1.slow convergence

The estimated output in computed form a linear combination of the input signal ,x(t) and weight vector w(k).The
estimated output y(k) is then compared to the desired output ,to find the error. If there is any difference, the error
signal will be used as feedback mechanism to adjust the weight vector w(k) on the other hand, if there is no
difference between these signals, no adjustment is needed, since estimated output is desirable
 XOR PROBLEM:
XOR problem can be solved by introducing a single hidden layer between input and output layers. consider two
neurons in hidden layer because the input to input layer are two
The Architecture for an XOR problem is drawn below:
 FOR N1:
The neuron labeled as N1 in hidden layer has the features as w11=w12=1

For an input considerations (0,0),(0,1),(1,0) the output of neuron in hidden layer

But for an input combination (1,1) the output of neuron in hidden layer
So graphically it can be represented as
FOR N2:
The neuron labeled as N2 in the hidden layer has the following features
w21=w22=1
slop wb2=-0.5
For an input combinations (0,0),the output of neuron in hidden layer=0
For an input combinations (0,1),(1,0)&(1,1) of neuron in hidden layer=1
So graphically it can be represented as
 FOR N3;
The neuron labeled as N3 in a output layer has following features
For an input combinations (0,0) &(1,1) the output of neuron in input layer=0
Similarly for an input combinations of (1,0) (0,1) the output of a neuron in output layer=1
So graphically it can represented as

By including one hidden layer we can solve an XOR problem i.e. linearly separable
A set is defined as collection of objects