S. Mandayam/ ANN/ECE Dept.

/Rowan University

Artificial Neural Networks

0909.560.01/0909.454.01

Fall 2004 Lecture 6 October 18, 2004 Shreekanth Mandayam

ECE Department Rowan University
http://engineering.rowan.edu/~shreek/spring04/ann/

S. Mandayam/ ANN/ECE Dept./Rowan University

Plan
Radial Basis Function Networks
RBF Formulation Network Implementation Matlab Implementation

Design Issues
Center Selection: K-means Clustering Algorithm Input data processing
Selection of training and test data - cross-validation Pre-processing: Feature Extraction

Lab Project 3

S. Mandayam/ ANN/ECE Dept./Rowan University

RBF Principle
Transform to
higher-dimensional vector space

Non-linearly separable classes

Linearly separable classes

S. Mandayam/ ANN/ECE Dept./Rowan University

Example: X-OR Problem

x1 0 0 1 1 x2 0 1 0 1 y 0 1 1 0 j 1 (x) 0.13 0.36 0.36 1 j 2(x) 1 0.36 0.36 0.13 y' 0 1 1 0

x2

j2(x)

Decision Boundary

x1

j1(x)

S. Mandayam/ ANN/ECE Dept./Rowan University

RBF Formulation
Problem Statement
Given a set of N distinct real data vectors (xj; j=1,2,,N) and a set of N real numbers (dj; j=1,2,,N), find a function that satisfies the interpolating condition F(xj) = dj; j=1,2,,N

S. Mandayam/ ANN/ECE Dept./Rowan University

RBF Network
Input Layer x1 Inputs x2 1 Hidden Layer Output j Layer j j 1 1 wij
1
0.5 0 -5 t 5

y1
Outputs y2 j(t)

1 1
1 j

x3
x i c j 2 2
2

j ij e

S. Mandayam/ ANN/ECE Dept./Rowan University

Matlab Implementation
%Radial Basis Function Network %S. Mandayam/ECE Dept./Rowan University %Neural Nets/Fall 04 clear;close all; %generate training data (input and target) p = [0:0.25:4]; t = sin(p*pi); %Define and train RBF Network net = newrb(p,t); plot(p,t,'*r');hold; %generate test data p1 = [0:0.1:4]; %test network y = sim(net,p1); plot(p1,y,'ob'); legend('Training','Test'); xlabel('input, p'); ylabel('target, t')

Matlab Demos demorb1 demorb3 demorb4

S. Mandayam/ ANN/ECE Dept./Rowan University

RBF - Center Selection

x2

x1
Data points

Centers

S. Mandayam/ ANN/ECE Dept./Rowan University

K-means Clustering Algorithm

N data points, xi; i = 1, 2, , N At time-index, n, define K clusters with cluster centers cj(n) ; j = 1, 2, , K Initialization: At n=0, let cj(n) = xj; j = 1, 2, , K (i.e. choose the first K data points as cluster centers) Compute the Euclidean distance of each data point from the cluster center, d(xj , cj(n)) = dij Assign xj to cluster cj(n) if dij = mini,j {dij};
i = 1, 2, , N, j = 1, 2, , K

For each cluster j = 1, 2, , K, update the cluster center cj(n+1) = mean {xj cj(n)} Repeat until ||cj(n+1) - cj(n)|| < e

S. Mandayam/ ANN/ECE Dept./Rowan University

Selection of Training and Test Data: Method of Cross-Validation

Trial 1
Trial 2 Train Train Train Test Train Train Test Train Test Train

Trial 3
Trial 4

Train Train

Test

Train Train Train

Vary network parameters until total mean squared error is minimum for all trials Find network with the least mean squared output error

S. Mandayam/ ANN/ECE Dept./Rowan University

Feature Extraction
Objective:
Increase information content Decrease vector length Parametric invariance
Invariance by structure Invariance by training Invariance by transformation

S. Mandayam/ ANN/ECE Dept./Rowan University

Lab Project 3: Radial Basis Function Neural Networks

http://engineering.rowan.edu/~shreek/fall04/ann/lab3.html

S. Mandayam/ ANN/ECE Dept./Rowan University

Summary