Documente Academic
Documente Profesional
Documente Cultură
com
Expert Systems
with Applications
Expert Systems with Applications 36 (2009) 13291336
www.elsevier.com/locate/eswa
Abstract
Over the last few decades pattern classication has been one of the most challenging area of research. In the present-age pattern classication problems, the support vector machines (SVMs) have been extensively adopted as machine learning tools. SVM achieves higher
generalization performance, as it utilizes an induction principle called structural risk minimization (SRM) principle. The SRM principle
seeks to minimize the upper bound of the generalization error consisting of the sum of the training error and a condence interval. SVMs
are basically designed for binary classication problems and employs supervised learning to nd the optimal separating hyperplane
between the two classes of data. The main objective of this paper is to introduce a most promising pattern recognition technique called
cross-correlation aided SVM based classier. The idea of using cross-correlation for feature extraction is relatively new in the domain of
pattern recognition. In this paper, the proposed technique has been utilized for binary classication of EEG signals. The binary classiers
employ suitable features extracted from crosscorrelograms of EEG signals. These cross-correlation aided SVM classiers have been
employed for some benchmark EEG signals and the proposed method could achieve classication accuracy as high as 95.96% compared
to a recently proposed method where the reported accuracy was 94.5%.
2007 Elsevier Ltd. All rights reserved.
Keywords: Support vector machines; Cross-correlation; Electroencephalogram signals; Optimal separating hyperplane
1. Introduction
Epileptic seizures are due to temporary electrical disturbances of the brain. Unfortunately, the occurrence of epileptic seizure is unpredictable (Subasi, 2007). The
neuroscientists and biological psychiatrists prefer to study
the electrical activity of the brain with the help of electroencephalographic records for diagnosis of neurological disorders (Gular, Ubeyli, & Guler, 2005). Basically, the
electroencephalogram (EEG) is a complex and aperiodic
time series containing information of the electrical activity
generated by cerebral cortex nerve cells. Earlier the analysis
of the electroencephalograph (EEG) records was restricted
to visual inspection. It has been shown that such visual
inspection is insucient to closely study the minute obser-
Corresponding author.
E-mail address: cha_ami@yahoo.co.in (A. Chatterjee).
0957-4174/$ - see front matter 2007 Elsevier Ltd. All rights reserved.
doi:10.1016/j.eswa.2007.11.017
1330
to traditional neural networks SVM achieves higher generalization performance, due to the utilization of structural
risk minimization principle. The SRM principle seeks to
minimize the upper bound of the generalization error consisting of the sum of the training error and a condence
interval, where as incase of traditional neural networks
the empirical risk minimization principle only minimizes
the training error (Cao & Tay, 2003).
Several works on the classication of EEG signals have
been reported. In Guler et al. (2005), the researchers evaluated the classication accuracy of the recurrent neural
networks (RNNs) using Lyapunov exponents trained with
LevenbergMarquardt algorithm on the electroencephalogram (EEG) signals. The database used for their work is
EEG database (EEG time series), which is developed by
department of Epileptology, University of Bonn, California. In this work three classes of EEG signals have been
considered. They are from healthy volunteers with eyes
open, epilepsy patients in the epileptogenic zone during a
seizure-free interval, and epilepsy patients during epileptic
seizures. An overall classication accuracy of 96.79% was
obtained in this work. In Subasi (2007), mixture of experts
model (ME) was utilized for classication of EEG signals.
A double-loop expectation-maximization (EM) algorithm
was introduced to the ME network structure for detection
of epileptic seizure and wavelet feature extraction was
employed. The database used in this work was with University of Bonn EEG time series, California. Healthy volunteers with eyes open and epilepsy patients during
epileptic seizures were used for classication and overall
accuracy was 94.5%. Up to now no study has been reported
in literature related to cross-correlation based feature
extraction in the analysis of EEG signals.
Correlation is a mathematical operation that is very similar to convolution. Cross-correlation of two signals measures the extent of similarity between these signals. In the
present work, a healthy subject is considered as reference
and the EEG signal of the reference subject is correlated
with the EEG signal of each of the other subjects. From
each of resulting cross-correlation sequences a set of ve
features is extracted. These feature vectors called patterns
are segregated into training and testing dataset and the
training set is utilized for training SVM classier. The
LS-SVM package, available in svm toolbox has been used
for this purpose. The generalization capability of the classier is then tested by utilizing it for testing dataset. The
structure of the paper is organized as follows. Section 2
briey reviews the sets of the EEG signals used in our
work. Section 3 reviews the cross-correlation of discretetime signals and the computation of various features of
crosscorrelograms utilized in this work. Section 4 is
devoted to provide necessary background and basic idea
behind support vector machine. Section 5 presents how
the proposed method is applied for classication of EEG
signals and how the cross-correlation is performed and features are extracted for each EEG signals is explained in
detail. Finally how the SVM is utilized for classication
1331
Set A
100
0
Set B
-100
0
500
500
1000
1500
2000
2500
3000
3500
4000
4500
500
1000
1500
2000
2500
3000
3500
4000
4500
500
1000
1500
2000
2500
3000
3500
4000
4500
500
1000
1500
2000
2500
3000
3500
4000
4500
500
1000
1500
2000
2500
3000
3500
4000
4500
Set C
-500
-5
0
200
0
Set D
-200
0
500
0
-500
Set E
1000
0
-1
-1000
Fig. 2. Exemplary EEG time series from each of the ve sets. Amplitude of surface EEG recordings are typically in the order of micro volts.
xy
(m)
-M
1332
minimize the upper bound of the generalization error consisting of the sum of the training error and a condence
interval (Cao & Tay, 2003).
To develop the SVM based classiers for linearly separable patterns, let us consider a training set represented
N
by fxi ; d i gi1 where xi is the n-dimensional input feature
vector and di represents the target output. The input patterns represented by the target output di = 1 constitute
the positive group and the target output di = 1 constitute
the negative group. For linearly separable case it is necessary to nd a hyperplane to distinguish the two classes of
input vectors (Haykin, 1999; Foody & Mathur, 2004).
There may be so many hyperplanes to separate the input
patterns, but there will be one and only one optimal separating hyperplane (OSH). The optimal separating hyperplane is a hyperplane that passes between two classes in
such a way that the distance between the closest training
points and itself is as maximum as possible.
The equation of decision surface in the form of hyperplane is written as w x + b = 0, where x is the input feature vector, w is the adjustable weight vector and b is the
bias. For linearly separable case, a separating hyperplane
can be found for two classes such that w x + b P +1 for
positive group of data and w x + b 6 1 for negative
group of data. These two equations can be combined
(Osuna, Freund, & Girosi, 1997; Kumar, 2004) as
d i w x b 1 P 0
1
2
uw kwk
2
d i w x b P 1;
Class 1
Class 1
Support
vector
Origin
i 1; 2 . . . N
Support
vector
Origin
x1
x2
Margin of
Separation
Class 2
Support
vector
Fig. 4. The SVM for the linearly separable case. The middle line indicates
optimal separating hyper plane.
Margin of
Separation
Class 2
Support
vector
-1-- 2
Fig. 5. The SVM for the linearly non-separable case. The middle line
indicates optimal separating hyper plane.
i1
kxyk2
2
2r
150
100
50
--100
-150
-200
0
200
150
100
50
0
-50
-100
-150
-200
-250
Amplitude 2 (V)2
200
-50
1333
Amplitude (V)
Amplitude (V)
x10
1.5
1
0.5
0
-0.5
-1
Table 1
Various types of kernels
Type of classier
Kernel function
Polynomial of degree m
Multi layer perceptron
Radial basis function
K(x, y) = (1 + x y)m
K(x, y) = tan h(x y h)
K(x, y) = exp(kx yk2)
(50 signals from each class, each of 4096 samples) the classier. In the second case study, a rectangular-window was
formed by 512 samples, such that each EEG segment is further divided into eight small segments. Hence in each set
there are 800 segments.
1334
Amplitude (V)
200
150
100
50
0
-50
-100
-150
-200
Amplitude (V)
600
400
200
0
represent the reference signal. This reference data is crosscorrelated with the data of the rest of healthy volunteers
and all the epilepsy patients. So in this process we create
99 feature vectors for healthy set, and 100 feature vectors
for epileptic seizure set. Figs. 6c and 7c shows typical
results of cross-correlation for of healthy volunteer and
epilepsy patient. In the second case study there are eight
reference data sub segments and each such sub segment is
crosscorrelated with the corresponding segment of the
data, belonging to either healthy dataset or epileptic dataset. So there are 792 feature vectors for healthy set and
there are 800 feature vectors formed for the epileptic seizure set. For both the two experiments the extracted ve
features were peak value (PV), instant at which peak occurs
(IP), equivalent width (EW), centroid (C) and mean square
abscissa (MSA). Hence for each crosscorrelogram we have
created ve-dimensional feature vector [PV IP EW C
MSA]T. These feature vectors are segregated into training
and testing dataset as described before, and the training
set is utilized for training SVM classier. We utilized the
LS-SVM package, available in svm toolbox for this purpose. The generalization capability of the classier is then
tested by utilizing it for testing dataset.
Tables 2 and 3 describe the segregation of signal segments into training and testing datasets for the two studies.
-200
Table 2
Segregation of training and testing sets for case study 1
-400
-600
Class
Training set
Testing set
Total
Healthy
Epileptic
Total
50
50
100
49
50
99
99
100
199
Amplitude 2 ((V)
V) 2
x 10
2
1
Table 3
Segregation of training and testing sets for case study 2
Class
Training set
Testing set
Total
Healthy
Epileptic
Total
492
500
992
300
300
600
792
800
1592
0
-1
Table 4
Confusion matrix for experiment 1
Predicted output
-2
-3
Actual output
-5000 -4000-3000-2000-1000 0 1000 2000 3000 4000 5000
samples
Epileptic
Healthy
Epileptic
Healthy
46
0
4
49
Table 5
Confusion Matrix for experiment 2
Predicted output
Actual output
Epileptic
Healthy
Epileptic
Healthy
277
4
23
296
ME
(Subasi,
2007)
model (%)
MLPNN
(Subasi,
2007) (%)
Crosscorrelation
SVM
(Experiment1)
(%)
Crosscorrelation
SVM
(Experiment2)
(%)
Sensitivity
Specicity
Accuracy
95.0
94.0
94.5
93.6
92.6
93.2
92.00
100.00
95.96
92.4
98.6
95.5
1335
1336