ADALine Neural Net

WHY NEURAL NETWORKS AND WHYNOW?

As modern computers become ever more powerful, scientists continue to be challenged to use machines effectively for tasks that are relatively simple for humans. Based on examples, together with some feedback from a teacher, we learn easily to recognize the letter A or distinguish a cat from a bird.

Traditional, sequential, logic-based digital computing excels in many areas, but has been less successful for other types of problems. The development of artificial neural networks began approximately 50 years ago, motivated by a desire to try both to understand the brain and to emulate some of its strengths.

What is neural net

A type of artificial intelligence that attempts to initiate the way a human brain works. Rather than using a digital model, in which all computations manipulate zeros and ones, a neural network works by creating connections between processing elements, the computer equivalent of neurons. The organization and weights of the connections determine the output.

Artificial neural net

Artificial Neural Networks are a connectionist processing model inspired by the architecture of real brains. Artificial neural networks are composed of simple nodes called artificial neurons or Processing Elements. They can be implemented via hardware (i.e., electronic devices) or software (i.e., computer simulations).

Components of neural net

Its pattern of connections between the neurons (called its architecture) Its method of determining the weights on the connections (called its training) Its activation function

Architecture of neural net

The arrangement of neurons into layers and the connection pattern within and between layers is called the net architecture Neural nets are often classified as single layer or multilayer

Each neuron has an internal state, called its activation or activity level, which is a function of the inputs it has received. Typically, a neuron sends its activation as a signal to several other neurons. It is important to note that a neuron can send only one signal at a time, although that signal is broadcast to several other neurons. For example, consider a neuron Y, illustrated in Figure 1.1, that receives inputs from neurons XI, X2 and X3 The activations(output signals) of these neurons are XI, X2 and X3, respectively. The weights on the connections from XI, X2 and X3 to neuron Y are w to W2 and W3, respectively. The net input, y_in, to neuron Y is the sum of the weighted signals from neurons XI,X2,andX3,i.e. Y_in = w1x1 + w2x2 + w3x3

Neural Net

Application of neural net

Signal processing Control Pattern recognition Medicine (instant physician) Speech Production Speech Recognition Business

ADALine
Adaline is a single layer neural network with multiple nodes where each node accepts multiple inputs and generates one output. Given the following variables: x is the input vector w is the weight vector n is the number of inputs some constant y is the output

Architecture of ADALine

An ADALine is a single unit neuron that receives input from several units The ADALine (Adaptive Linear Neuron) typically uses bipolar (1or-1) An ADALine can be trained using the delta rule its also known as Least Mean Squares (LMS) or widrowHoff rule. After training if the net is used for pattern classification in which the desired output is either a 1 or a -1, a threshold function is applied to the net input to obtain the activation

Applications

Adaline has better convergence properties than Perceptron Useful in noise correction Adaline in every modem.

Algorithm
Step 0: Initialize weights (small random values) Set learning rate Step1: while stopping condition false do step 2-6 Step2: for each bipolar training pair s:t, do step 3-5 Step3: set activation of input unit,i=1..n Step4: compute net input to output unit Y-in=b+xiwi Set activation function Y= 1 if y-in0; -1 if y-in<0;

Step5: Update bias and weigths,i=1n b(new)=b(old)+(t-y-in) Wi(new)=wi(old)+ (t-y-in)xi Step6: test for stopping condition if the largest weight changed that occurred in step 2 is smaller than a specific tolerance, then stop; otherwise continue

Example
An ADALine for the AND function bipolar inputs and target output X1 x2 b t 1 1 1 1 1 -1 1 -1 -1 1 1 -1 -1 -1 1 -1 Firstly calculate the weights new weights are: W1=1/2 W2=1/2 W0=-1/2