MEDICAL IMAGE CLASSIFICATION USING SUPPORT VECTOR MACHINE

A PROJECT REPORT

Submitted by

N.N.ABIMANYU S.GUGAPRIYA D.LALITHA

In partial fulfillment for the award of the degree of

BACHELOR OF ENGINEERING
in

ELECTRONICS & COMMUNICATION ENGINEERING

SCHOOL OF COMMUNICATION & COMPUTER SCIENCES

KONGU ENGINEERING COLLEGE, PERUNDURAI

ANNA UNIVERSITY::CHENNAI 600 025

APRIL 2010

ANNA UNIVERSITY::CHENNAI 600 025

BONAFIDE CERTIFICATE

Certified

that

this

project

report

MEDICAL

IMAGE

CLASSIFICATION USING SUPPORT VECTOR MACHINE is the bonafide work of N.N.Abimanyu (71106106001), S.Gugapriya

(71106106029), and D.Lalitha (71106106047), who carried out the project work under my supervision.

SIGNATURE

SIGNATURE

Dr. G. Murugesan, B.E., M.S., M.E., Ph.D., HEAD OF THE DEPARTMENT Professor, Department of Electronics & Communication Engineering, School of Communication & Computer Sciences, Kongu Engineering College, Perundurai, Erode 638 052.

Mr. K. Venkateswaran, M.E., SUPERVISOR Lecturer, Department of Electronics & Communication Engineering, School of Communication & Computer Sciences, Kongu Engineering College, Perundurai, Erode 638 052.

ii

CERTIFICATE OF EVALUATION

College Name Branch Semester

: Kongu Engineering College : Electronics & Communication Engineering : VIII

S.No.

Name of the student N.N. Abimanyu (71106106001) S. Gugapriya (71106106029) D. Lalitha (71106106047)

Title of the project MEDICAL IMAGE CLASSIFICATION USING SUPPORT VECTOR MACHINE

Name of the supervisor with designation

1.

Mr. K. Venkateswaran, M.E. Lecturer ECE Department

2.

3.

The report of the project work submitted by the above students in partial fulfillment for the award of Bachelor of Engineering degree in Electronics & Communication Engineering, Anna University was evaluated and confirmed to be the report of the work done by the above students.

Submitted for the University examination held on ..

Internal Examiner

External Examiner

iii

ACKNOWLEDGEMENT We express our sincere thanks to our beloved Correspondent Thiru. V. R. Sivasubramanian, B.Com., B.L., and all the members of Kongu Vellalar Institute of Technology Trust at this high time for providing all the necessary facilities to complete the course successfully. We wish to express our profuse thanks and gratitude to our beloved Principal Prof. S. Kuppusami, B.E., M.Sc (Engg), Dr.Ing (France) for his constant encouragement in the successful completion of this project work. We wish to thank our Dean Prof.S.Balamurugan B.E., M.Sc (Engg), School of Communication And Computer Sciences for his timely advice and help in the completion of the project. We express our gratitude to Dr. G. Murugesan, B.E., M.S., M.E., Ph.D., Head of the Department, Electronics and Communication Engineering, who has been the major source of inspiration to us throughout the duration of our work. We are thankful to our project co-ordinator Ms. D. Malathi, M.E. for her valuable suggestions. We intend to thank our guides Mr. K. Venkateswaran, M.E. and Mr. S. Raja, M.E. for their valuable suggestions in the improvement of the project. We wish to express our sincere thanks to all the faculties of Electronics and Communication Department and our friends for their constant support towards the successful completion of this project.
iv

ABSTRACT

The project presented in this report is aimed at developing an automated machine learning algorithm for the classification of medical images obtained from Magnetic Resonance Imaging (MRI) scans. This also implies the necessity for applying transforms on images. The algorithm consists of two steps: automatic extraction of features from the MRI images through two different image transforms namely-Discrete Wavelet

Transform (DWT) and Gabor transform and their classification by creating a classifier trained on the extracted results. The classifier used is the Support Vector Machine (SVM) with linear kernel function. Further classification and comparison of obtained results showed that the features extracted by applying Gabor transform on MR images provide an accuracy of 100% by using SVM with linear kernel function.

TABLE OF CONTENTS

CHAPTER NO.

TITLE

PAGE NO.

ABSTRACT LIST OF TABLES LIST OF FIGURES LIST OF ABBREVIATIONS

v viii ix x

1.

INTRODUCTION

2.

LITERATURE REVIEW

3.

MEDICAL IMAGE CLASSIFICATION 3.1 Medical Images 3.2 Applying Image Transforms 3.2.1 Image Transformations 3.2.2 Discrete Wavelet Transform 3.2.3 Gabor Transform 3.3 Feature Extraction 3.3.1 Gray Scale Feature 3.3.2 Shape Features 3.3.3 Texture Features 3.4 Data Set Formation

5 5 7 7 7 11 14 14 16 16 18

vi

3.5 Classification by SVM 3.5.1 Statistical Learning Theory 3.5.2 Support Vector Machine 3.5.3 Kernel Classifiers 3.5.4 Linear SVM 3.6 Performance of SVM

19 19 20 23 24 25

4.

RESULTS 4.1 Results for transforms 4.1.1 Discrete Wavelet Transform 4.1.2 Gabor Transform 4.2 Results for Feature Extraction 4.3 Results for SVM Performance Measurement 4.3.1 Wavelet Performance measures 4.3.2 Gabor Performance measures

27 27 27 29 32 37 37 38

5.

COMPARISON OF RESULTS AND ANALYSIS CONCLUSION

40 42

6.

REFERENCES

43

vii

LIST OF TABLES

TABLE NO. 4.1 4.2 4.3 4.4 4.5 4.6 4.7

TITLE Normal Images Mean Normal Images Variance Normal Images Entropy Abnormal Images Mean Abnormal Images Variance Abnormal Images Entropy Performance Measures without applying transforms

PAGE NO. 32 33 33 34 35 36 37

4.8

Performance Measures of DWT based SVM

37

4.9

Performance Measures of Gabor based SVM

38

5.1

Performance Measures Comparison

40

viii

LIST OF FIGURES

FIGURE NO. 3.1 3.2 3.3

TITLE Normal Image Abnormal Image Three level wavelet decomposition Tree

PAGE NO. 6 6

10 11 22 25 27 28 29 30 31

3.4 3.5 3.6 4.1 4.2 4.3 4.4 4.5

Wavelet Families SVM principles Linear SVM Wavelet transform of abnormal image Wavelet transform of normal image Gabor sub-bands Gabor Transform of normal image Gabor Transform of abnormal image

ix

LIST OF ABBREVIATIONS

MRI CADe CADx t2fs FLAIR ANN WT CWT DWT GT COM IDM SVM RBF SLT SRM

Magnetic Resonance Imaging Computer Aide Detection Computer Aided Diagnosis T2 weighted FLAIR image Fluid Attenuated Inversion Recovery Artificial Neural Network Wavelet Transform Continuous Wavelet Transform Discrete Wavelet Transform Gabor Transform Co-Occurrence Matrix Inverse Difference Moment Support Vector Machine Radial Basis Function Statistical Learning Theory Structural Risk Minimization

CHAPTER 1 INTRODUCTION There are many factors in the process of the physicians diagnosis. Firstly, the physicians diagnosis is subjective, because the diagnosis result is affected by the doctors experience and ability; Secondly, the physicians diagnosis tends to omit some tiny changes that the human eyes cant find; thirdly, different physicians would get the different diagnosis conclusions for the same medical image. Comparing to the physicians, the computer has tremendous predominance in the aspect of avoiding the incorrect results. Image Classification plays an important role in the fields of Medical diagnosis, Remote Sensing, Image analysis and Pattern Recognition. Digital image classification is the process of sorting of images into a finite number of individual classes. In the case of main fields like medical diagnosis, images have to be classified with maximum accuracy, or else it will lead to incomplete treatment of the corresponding disease. The main advantage of applying transform to a medical image before extracting its features is to reduce redundant data present in the image. Applying transforms can improve the accuracy and accordance of the diagnosis result. First and second order statistics of the wavelet detail coefficients provide descriptors that can discriminate intensity properties spatially distributed throughout the image, according to various levels of resolution. Then, using SVM classifier, these features are classified and performance measures of SVM classifier such as accuracy, precision, sensitivity, specificity are evaluated and compared with those from the nontransformed images.

CHAPTER 2 LITERATURE REVIEW Image Classification plays an important role in the fields of Medical diagnosis, Remote sensing, Image analysis and Pattern Recognition.. In case of main fields like medical diagnosis, images have to be classified with maximum accuracy, or else it will lead to incomplete treatment to the corresponding disease. In order to achieve high classification accuracy in image classification, numerous methods for feature extraction and image classification were introduced. For feature extraction, different transforms like Wavelet, Ridgelet, Curvelet, Contourlet transforms were used. For classification different statistical classifiers like Feed forward Neural Networks(FNN) ,Feedback Neural Networks, Back propagation Neural Networks(BPNN),Multi-Layer Perceptron(MLP),Hybrid Hopfield Neural Networks(HHNN),Radial Basis Function Neural Network(RBF-NN)[1],SelfOrganizing Map(SOM) and kernel classifiers like Support Vector Machine(SVM) are used. One method is by using Ridgelet transform for feature extraction and classification as explained in the paper Ridgelet based Texture Classification of Tissues in Computed tomography [3] by Lindsay Semlera, Lucia Dettoria and Brandon Kerrb of DePaul University, Chicago. This article focuses on using Ridgelet-based multi-resolution texture analysis and also on investigating the discriminating power of several Ridgelet based texture descriptors. Even though Ridgelet transform provided good performance, but

it gives good results only in case of texture descriptors, which are somewhat difficult to calculate. A paper titled A Comparison of Daubechies and Gabor Wavelets for Classification of MR Images [2] by Ulas Bagci, Li Bai of CMAIG proposes a machine learning algorithm for classifying MR image using wavelet transform and Gabor transform using SVM.But,results of 100% classification accuracy in case of classifying normal and abnormal images were obtained using SVM classifier of Sigmoid and RBF kernels only,which are more time consuming and less effective than SVM classifier of linear kernel. An article named A Comparison of Wavelet-based and Ridgelet based texture Classification of tissues in Computed Tomography[3] by Lindsay Semler, Lucia Dettori of DePaul University, Chicago presents a comparison of wavelet-based and Ridgelet-based algorithms for medical image classification. Tests on a large set of chest and abdomen CT images indicate that, the one using texture features derived from the Haar wavelet transform clearly outperforms the one based on Daubechies and Coiflet transform. The tests also show that the Ridgelet-based algorithm is significantly more effective, but it fails to recognize point coordinates in image which represents the tumour effectively and does not provide 100% accuracy. Another problem lies in the case of classification. Before SVM, Artificial Neural Networks (ANN) were widely used for classification of images and other applications. Many types of Artificial Neural Networks(ANN) namely Feed forward Neural Networks(FNN) ,Feedback Neural Networks, Back propagation Neural Networks(BPNN) ,Multi-Layer Perceptron(MLP), etc were developed. But all of them have their own

advantages and disadvantages and no one of them provided 100% classification accuracy. With the introduction of kernel classifiers, SVM classifier becomes more common. But, kernel classifiers called Support Vector Machine (SVM) with linear kernel provided 100% classification accuracy. A comparison on SVM and FNN was provided in the paper titled Research on Comparison and Application of SVM and FNN Algorithm [5] written by Shaomei Yang and Qian Zhu. This paper concluded that high recognition accuracy and high speed convergence are achieved by using SVM rather than FNN. Another study titled Improved Classification of Pollen texture Images using SVM and MLP [4] by M. Fernandez-Delgado, P. Carrion, E. Cernadas, J.F. Galvez, Pilar Sa-Otero, explored the use of more sophisticated classifiers to improve the classification stage. It explains that SVM achieved high accuracy when compared to k-Nearest Neighbours (KNN) and MultiLayer Perceptron (MLP). Also KNN requires storing of whole training set and hence it is time consuming.

CHAPTER 3 MEDICAL IMAGE CLASSIFICATION

3.1 MEDICAL IMAGES As the projects main objective is to classify medical images, the first and foremost need is the input to the algorithm, i.e., medical images. Here, classification is done on medical images obtained from Magnetic Resonance imaging (MRI) scans. Among different types of MRI scans like Diffusion MRI, Functional MRI, FLAIR (Fluid Attenuated Inversion Recovery) MRI, Interventional MRI, images taken here are FLAIR images of t2fs type. Fluid attenuated inversion recovery (FLAIR) is a pulse sequence used in magnetic resonance imaging which was invented by Dr. Graeme Bydder. FLAIR can be used with both three dimensional imaging (3D FLAIR) or two dimensional imaging (2D FLAIR). The pulse sequence is an inversion recovery technique that nulls fluids. For example, it can be used in brain imaging to suppress cerebrospinal fluid (CSF) effects on the image, so as to bring out the periventricular hyperintense lesions, such as multiple sclerosis (MS) plaques. By carefully choosing the TI, the signal from any particular tissue can be nullified. The appropriate TI depends on the tissue via the formula: TI = ln2 * T1 One should typically yield a TI of 70% of T1. In the case of CSF suppression, one aims for T2 weighted images. T2-weighted scans use a spin echo (SE) sequence, with long TE and long TR. They have long been the clinical workhorse as the spin echo
5

sequence is less susceptible to inhomogeneities in the magnetic field. They are particularly well suited to edema as they are sensitive to water content (edema is characterized by increased water content). Also, this project mainly focuses on creating an algorithm for classifying brain images especially and detecting any abnormalities in brain like tumor. Therefore, the images which are given as input to the algorithm are of two types 1. Normal brain images (without tumor) (eg. Fig. 3.1) and 2. Abnormal brain images (with tumor) (eg. Fig. 3.2)

Fig. 3.1 Normal Image

Fig.3.2 Abnormal Image

3.2 APPLYING IMAGE TRANSFORMS 3.2.1 IMAGE TRANSFORMATIONS In different image processing applications, the main step considered is applying transforms on images. This step is usually carried out for the following reasons: 1. To reduce the redundant data in image 2. To reduce the actual size of image content. 3. Easy analysis. The transform of a signal is a way to represent the signal. It does not alter the information content present in the signal. There are different image transformations available such as: 1. Cosine Transform 3. Fourier Transform 5. Ridgelet Transform 7. Contourlet Transform 9. Gabor Transform, etc. Among these most promising are - 1.Wavelet Transform and 2.Gabor transform. 2.Sine Transform 4.Wavelet Transform 6. Curvelet Transform 8.Riplet Transform

3.2.2 DISCRETE WAVELET TRANSFORM The Wavelet Transform provides a time-frequency representation of the signal. It was developed by Morlet to overcome the short comings of the Short Time Fourier Transform (STFT) and Fourier Transform (FT), which

can also be used to analyze non-stationary signals where all frequencies have an infinite coherence time. While FT and STFT give a constant resolution at all frequencies, the Wavelet Transform uses multi-resolution technique by which different frequencies are analyzed with different resolutions and a coherence time proportional to the period of the signal. Also, Fourier transform (FT) does not provide any information to show where within single certain frequencies occur, i.e., information about the time domain is lost. Since, most of the signals in the real-world change with time, it is especially useful to characterize signals in both time and frequency domain simultaneously. Wavelet transform is used for this reason. The Wavelet Transform (WT) decomposes a signal into a linear sum of basis-functions which are dilated and translated wavelets. A wavelet is a function L2 () with zero average, normalized to one and centered in the neighbourhood of t=0. A family of time-frequency components is obtained by translating it by u and scaling it by .

Convolution of this wavelet function with the image will give us the Wavelet transform of the image f(x,y). The wavelet transform of the image can be analysed with respect to both time and frequency domains. Hence, wavelet transform provides information about both time and frequency domains at various resolutions. Discrete Wavelet transform converts a discrete time signal into wavelets. Discrete Wavelet transform is expressed as:
8

where dm(n) = <x, mn> and

are basis for L2() satisfying the biorthogonality condition ,which is defined as(mn ,lk)=ml nk. In orthogonal case, mn can be used for both synthesis and analysis of wavelets. The two-dimensional Wavelet transform is carried out by applying one-dimensional wavelet transform to the rows and columns of the twodimensional data consecutively. It is computed by successive lowpass and highpass filtering of the discrete time-domain signal .This is called the Mallat algorithm or Mallat-tree decomposition. In the figure 3.3, the signal is denoted by the sequence x[n], where n is an integer. The low pass filter is denoted by G0 while the high pass filter is denoted by H0. At each level, the high pass filter produces detailed information; d[n], while the low pass filter associated with scaling function produces coarse approximations,a[n]. Generally, there are three detailed components in images namely: 1.Horizontal, 2.Vertical and 3.Diagonal components. These three components contain all the detailed information about high frequency contents present in the image and hence they are used to extract image features rather than from approximation components.

Fig 3.3 Three level wavelet decomposition tree The advantages of DWT are: 1. It is easy to implement 2. It reduces the computation time 3. Resources required for the computation of DWT co-efficients are less. 4. It needs only O(N) operations to get computed and hence, it is also referred to as the fast wavelet transform In Wavelet Families there are a number of basis functions that can be used as the mother wavelet for Wavelet Transformation. Since the mother wavelet produces all wavelet functions used in the transformation through translation and scaling, it determines the characteristics of the resulting Wavelet Transform. They include Haar wavelet, Daubechies wavelet, Symlet, Coiflet, Biorthogonal wavelet, Morlet, Meyer wavelet and Mexican Hat wavelet as given in fig 3.4

10

Fig 3.4 Wavelet Families (a)Haar (b)Daubechies (c)Coiflet1 (d)Symlet (e)Meyer (f)Morlet (g)Mexican Hat Haar wavelet is the oldest and simplest wavelet available which is used as mother wavelet for classifying medical images. Since Haar wavelet is suitable for representing the image in time domain, it is used here.

3.2.3 GABOR TRANSFORM Gabor transforms are widely used in image analysis and computer vision. The Gabor transform provides an effective way to analyze images and has been elaborated as a frame for understanding the orientation and spatial frequency selective properties of signals. It seems to be a good approximation to the sensitivity profiles. The important advantages are infinite smoothness and exponential decay in frequency. Gabor transform represents an approach to characterise a time function in terms of time and frequency simultaneously.

11

A joint space-frequency analysis of any signal cannot be performed by the Fourier transform. This can be easily performed by Short time Fourier Transform (STFT).The STFT of a signal s(t) can be given asSTFT (,) = s(t) g(t-) exp (-jt) dt Thus, STFT of a signal is defined as Fourier Transform of the signal s(t) windowed by the function g(t-).The STFT with a Gaussian window is called a Gabor transform. The Gabor transform can be regarded as a signal being convoluted with a filter bank, whose impulse response in the time domain is Gaussian signal modulated by sine and cosine waves. The characteristics of the Gabor wavelets, especially for frequency and orientation representations, are similar to those of human visual system, and they have been found to be particularly appropriate for texture representation and discrimination. In the spatial domain, the 2D Gabor filter is a Gaussian kernel function modulated by a sinusoidal plane wave.

where f is the central frequency of the sinusoidal plane wave, is the anticlockwise rotation of the Gaussian and the plane wave, is the sharpness of the Gaussian along the major axis parallel to the wave, and is the sharpness

12

of the Gaussian minor axis perpendicular to the wave. = f / and = f / are defined to keep the ratio between frequency and sharpness constant.

The Gabor filters are self-similar; all filters can be generated from one mother wavelet by dilation and rotation. Each filter is in the shape of plane waves with frequency f, restricted by a Gaussian envelope function with relative width and . To extract useful features from an image, normally a set of Gabor filters with different frequencies and orientations are required. The Gabor representation of a MR brain image M(~x) can be obtained by convolving the image with the family of Gabor filters is given by:

where LHS denotes resultant Gabor representation of image M at orientation u and scale v. Thus, the resultant Gabor feature set consists of the convolution results of an input image M(x) with all of the Gabor wavelets:

13

The number of Gabor wavelets used varies with different applications. Better results can be obtained, if 5 scales and 8 orientations are used for classification of MR brain images. Extracted feature vectors are then concatenated together to construct a new feature vector to be used for classification purposes. In Figure 4.3, a MR brain image is convolved with 40 Gabor filters (8 orientations, 5 scales) and each row and column shows a different scale and orientation respectively. 3.3 FEATURE EXTRACTION Feature Extraction is an important step in image classification. It refers to extraction of various characteristics of the image, either transformed or untransformed. There are numerous features available in the image to get extracted. The feature extraction includes the 1. Gray scale (intensity) Extraction 2. Shape extraction 3. Texture extraction

3.3.1 GRAY SCALE FEATURES In digital picture process, the two-dimensional digitized gray scale image (MN) can be seen as MN pixels in two-dimensional surface XOY, each pixel f(x,y) can be expressed as its gray value.

Mean Mean or average value represents the arithmetic mean or average of all pixels in the image f(x,y).

14

Variance Variance is the first moment about the mean. It reflects the separate degree of gray scale value.

Skewness Skewness is the second moment about the mean. It takes the mean value as the central data distribution not as symmetrical degree.

Kurtosis Kurtosis is the third moment about the mean. It reflects the normal distribution sharpness or smoothness of compared the data.

Histogram The gray histogram is the gray scale value function; it describes the rate of the pixels that have the same gray scale value in the picture.

where nb is the number of the pixels that the gray scale value is b, n is the total number of the pictures pixels.

15

The features aiming at the gray scale histogram is mainly the gray scale average value, gray scale variance, gray Skewness, gray Kurtosis, gray energy and gray entropy. 3.3.2 SHAPE FEATURES The shape features have the characters that are scaling invariability, rotation invariability, translation invariability, so they can become the object recognition features. The features include the Normalized Moment of Inertia (NMI), moment invariants and sphericity. The shape features can highly identify the figure. As the exterior shapes of brain image are all nearly ellipse, it cant distinguish the image by extracting the features to the whole image. Considering the place and character of the certain pathological changes we select the ROI and extract its features.

3.3.3 TEXTURE FEATURES A co-occurrence matrix (COM) is a square matrix whose elements correspond to the relative frequency of occurrence p(i,j,d,h) of two pixel values (one with intensity i and the other with intensity j ), separated by a certain distance d in a given direction h . A COM is therefore a square matrix that has the size of the largest pixel value in the image. The COM is scale invariant. The matrices present the relative frequency distributions of gray levels and describe how often one gray level will appear in a specified spatial relationship to another gray level within each image.

16

The matrix was normalized by the following function:

where, R is the normalized function, which is usually set as the sum of the matrix.

Energy Energy is also called Angular Second Moment. It is a measure of the homogeneousness of the image and can be calculated from the normalized COM. It is a suitable measure for detection of disorder in texture image. Higher values for this feature mean that the amplitude or intensity changes less in the image, resulting in a much sparser COM.

Contrast Contrast is a measure of amount of the local variation in the image. It presents the degree of the legibility of the image. A higher contrast value indicates a high amount of local variation, so the higher contrast is, the clearer the image is.

IDM The Inverse Difference Moment (IDM) reflects the local texture changes. It is another feature of image contrast.

17

Entropy Entropy is a statistical measure of randomness that can be used to characterize the texture of the input image. The entropy of the image reflects the gray asymmetrical extent and complicated extent.

We have considered three important features-two gray scale features namely mean and variance and one texture feature namely entropy are calculated and used to create datasets for SVM classification.

3.4 DATASET FORMATION As prior to classification by SVM the extracted features must be arranged in dataset. All the images used for classification are got from hospital. There are totally 38 images, of which 7 images are got from healthy persons and the remaining 31 images are got from persons who are suffering from brain tumour. As classification by SVM consists of two phases namely: training and testing phases, all images are used for these two phases in the following manner: For training phase: 11 abnormal images and 3 normal images are used. For testing phase: 20 abnormal images and 4 normal images are used.

18

3.5 CLASSIFICATION BY SVM 3.5.1 STATISTICAL LEARNING THEORY Learning can be thought of as inferring regularities from a set of training examples. Various learning algorithms allow the extraction of regularities. If the learning has been successful, these intrinsic regularities will be captured in the values of some parameters of a learning machine. Geometry provides a very intuitive background for the understanding and the solution of many problems in Machine Learning. Statistical learning theory addresses a key question that arises when constructing predictive models from data-how to decide whether a particular model is adequate or whether a different model would produce better predictions. Whereas classical statistics typically assumes that the form of the correct model is known and the objective is to estimate the model parameters, statistical learning theory presumes that the correct form is completely unknown and the goal is to identify mathematical form and none of them need be correct. The theory provides a sound statistical basis for assessing model adequacy under these circumstances, which are precisely the circumstances encountered in machine learning, pattern recognition, and exploratory data analysis. Estimating the performance of competing models is the central issue in statistical learning theory. Performance is measured through the use of loss functions. The loss Q(z,) between a data vector z and a specific model (one with values assigned to all parameters) is a score that indicates how well performs on z, with lower scores indicating better performance. Statistical Learning Theory forms the basis of classification by Support Vector Machine (SVM).

19

3.5.2 SUPPORT VECTOR MACHINE Support Vector Machines (SVMs) are a new supervised classification technique that has its basis in Statistical Learning Theory .Based on machine vision fields such as character, handwriting digit and text recognition there has been increased interest in their application to image classification. SVMs are non-parametric hence they boost the robustness associated with Artificial Neural Networks and other nonparametric classifiers. The purpose of using SVM is to obtain the acceptable results fast and easily. The Support Vector Machine is a theoretically superior machine learning methodology with great results in classification of high dimensional datasets .A classication task usually involves with training and testing data which consist of some data instances. Each instance in the training set contains one target value (class labels) and several attributes (features). The goal of SVM is to produce a model which predicts target value of data instances in the testing set which are given only the attributes. SVM functions by nonlinear projection of the training data in the input space to a feature space of higher (infinite) dimension by use of a kernel function . Then SVM identifies a linear separating hyperplane with the maximal margin in this higher dimensional space. This process enables the classification of remote sensing datasets which are usually nonlinearly separable in the input space. In many instances, classification in high dimension feature spaces results in overfitting of the input space. However, in SVMs, overfitting is controlled through the principle of structural risk minimization. The empirical risk of misclassification is minimised by maximizing the margin between the data points and the decision boundary. In

20

practice this criterion is softened to the minimisation of a cost factor involving both the complexity of the classifier and the degree to which marginal points are misclassified. The tradeoff between these factors is managed through a margin of error parameter which is tuned through crossvalidation procedures. The SVM paradigm in Machine Learning presents a lot of advantages over other approaches such as: 1) Uniqueness of the solution (as it is guaranteed to be the global minimum of the corresponding optimization problem), 2) Good generalization properties of the solution, 3) Rigid theoretical foundation based on SLT and optimization theory, 4) Common formulation for the class separable and the class non-separable problems as well as for linear and non-linear problems (through kernel trick). 5) Clear geometric intuition of the classification problem. Due to these very attractive properties, SVM have been successfully used in a number of applications. SVM allows the construction of various learning machines by choice of different dot products. Thus the influence of the set of function that can be implemented by a specific learning machine can be studied in a unified frame work. SVM builds support vectors on results of statistical learning theory, namely on the Structural Risk Minimisation principle guaranteeing high generalization ability. There is reason to believe that decision rules constructed by the support vector algorithm do not reflect incapabilities of learning machine but rather the regularities of data.

21

Fig 3.5 SVM principle A SVM finds the best separating (maximal margin) hyperplane between two classes of training samples in the feature space, which is in line with optimizing bounds concerning the generalization error. The playground for SVM is the feature space H, which is a Reproducing Kernel Hilbert Space (RKHS), where the mapped patterns reside (: X->H). It is not necessary to know the mapping itself analytically, but only its kernel, i.e., the value of the inner products of the mappings of all the samples (K (x1, x2) = <(x1),(x2)> for all x1, x2 X) .Through the kernel trick, it is possible to transform a nonlinear classification problem to a linear one, but in a higher (maybe infinite) dimensional space H2. Once the patterns are mapped in the feature space, provided that the problem for the given model (kernel) is separable, the target of the classification task is to find the maximal margin hyperplane.
22

Principle of Structural Risk Minimisation Technique (SRM) is an inductive principle of use in machine learning. Commonly in machine learning a generalized model must be selected from a finite data set, with the consequent problem of overfitting the model becoming too strongly tailored to the particularities of the training set and generalizing poorly to new data. The SRM principle addresses this problem by balancing the Modds complexity against its success at fitting the training data. The functions used to project the data from input space to feature space are called kernels. K (xi, xj) (xi) (xj) is called the kernel function.
T

3.5.3 KERNEL CLASSIFIERS Depending on the kernel function used, SVM classifier is classified into different types namely: 1. Linear SVM 2. Polynomial SVM 3. Radial Basis Function (RBF) SVM 4. Sigmoid SVM Four basic kernels functions used are: Linear: K(xi, xj) = x i j. Polynomial: K(xi, xj) = (xiTxj+r) , > 0. Radial basis function (RBF): K(xi, xj) = exp(kxi xjk ), > 0.
2 d T x

23

Sigmoid: K(xi, xj) = tanh(xiTxj+ r)

Here, , r, and d are kernel parameters. In this project linear SVM classifier is used for the classification of MR brain images.

3.5.4 LINEAR SUPPORT VECTOR MACHINE The task of image classification is similar to the following problem: Consider the problem of classifying m points in the n-dimensional real space R n, represented by the m*n matrix A, according to membership of each point Ai in the class A+ or A- as specified by a given m*m diagonal matrix D with plus ones or minus ones along its diagonal. For this problem the standard support vector machine with a linear kernel is given by the following quadratic program with parameter >0

Here is the normal to the bounding planes and determines their location relative to the origin. The plane *x+b>1 bounds the class A+ points, possibly with some error, and the plane *x+b>1 bounds the class Apoints, also possibly with some error. The linear separating surface is the plane *x+b=0 which is the midway between the bounding planes.

24

Fig.3.6 Linear SVM

3.6 PERFORMANCE OF SVM The performance of Linear Support Vector Machine (SVM) classifier is to be measured to find the extent of classification of MR images. It is done by calculating the performance measures of the classifier. The performance measures of SVM classifier are: 1. Accuracy/Classification rate/Efficiency 2. Precision 3. Sensitivity/Recall 4. Specificity
25

Accuracy Accuracy is the probability that a diagnostic test is correctly performed. Accuracy = (TP + TN)/(TP + TN + FP + FN) Precision Precision is the probability that a diagnostic test is correctly performed, when certain classes of images come consecutively. It is the degree of accuracy. Precision=TP/(TP+FP) Sensitivity Sensitivity (true positive fraction) is the probability that a diagnostic test is positive, given that the person has the disease. Sensitivity = TP/(TP + FN) Specificity Specificity (true negative fraction) is the probability that a diagnostic test is negative, given that the person does not have the disease. Specificity = TN/(TN + FP) Where: TP (True Positives) TN (True Negative) FP (False Positives) FN (False Negative) : Correctly classified positive cases, : Correctly classified negative cases, : Incorrectly classified negative cases, and : Incorrectly classified positive cases

26

CHAPTER 4 RESULTS

4.1 RESULTS FOR TRANSFORMS The Discrete Wavelet transform with Haar function and Gabor transform were applied to the MR images, both normal and abnormal. The following results were obtained.

4.1.1 DISCRETE WAVELET TRANSFORM

Fig.4.1 Wavelet transform of abnormal image

27

Fig.4.2 Wavelet transform of normal image

28

4.1.2 GABOR TRANSFORM

Fig.4.3 Gabor sub-bands

29

Fig.4.4 Gabor Transform of normal image

30

Fig.4.5 Gabor Transform of abnormal image

31

4.2 RESULTS FOR FEATURE EXTRACTION In this the feature extraction results such as mean, variance and entropy values for both normal and abnormal images, without and with applying wavelet and Gabor transformations are given as follows,

Mean Image
Without Transform Horizontal Image 1 Image 2 Image 3 Image 4 Image 5 Image 6 Image 7 60.74703 56.40564 56.32641 58.95343 68.64449 70.68972 56.61366 0.037695 0.046558 0.051931 0.049337 0.056753 0.062788 0.046875 Vertical 0.093213 0.091089 0.059011 0.066793 0.104955 0.102326 0.094604 Diagonal 0.000317 0.001172 0.005503 0.001343 0.00174 0.000313 0.000342 Wavelet Gabor (Sub-band 1) 0.251645 0.235427 0.286015 0.287435 0.339758 0.323999 0.271618

Table 4.1 Normal Images Mean

32

Variance Image
Without Transform Horizontal Image 1 Image 2 Image 3 Image 4 Image 5 Image 6 Image 7 2891.725 2369.369 2861.059 3059.756 4193.412 4173.711 2005.429 62.92573 57.73664 69.58309 69.86778 81.61674 80.50006 54.35772 Vertical 50.4192 45.22733 67.54331 71.42149 73.89571 73.31604 40.09178 Diagonal 2.032886 1.953733 3.643807 3.549834 3.67788 3.333723 2.547767 Wavelet Gabor (Sub-band 1) 0.044661 0.040062 0.068496 0.061923 0.087613 0.074757 0.049764

Table 4.2 Normal Images Variance

Entropy Image
Without Transform Horizontal Image 1 Image 2 Image 3 Image 4 Image 5 Image 6 Image 7 0.04601 0.046103 0.042084 0.042005 0.046667 0.046762 0.04601 1.22917 1.243307 1.235241 1.224813 1.190325 1.191948 1.216871 Vertical 1.348253 1.3554 1.299876 1.29799 1.259527 1.25965 1.287355 Diagonal 1.525145 1.528097 1.497687 1.492563 1.465836 1.474173 1.49454 Wavelet Gabor (Sub-band 1)

7.067558 6.928274 7.117588 7.132964 7.210374 7.186631 7.067558

Table 4.3 Normal Images Entropy

33

Mean Image
Without Transform Horizontal Image 1 Image 2 Image 3 Image 4 Image 5 Image 6 Image 7 Image 8 Image 9 Image 10 Image 11 Image 12 Image 13 Image 14 Image 15 Image 16 Image 17 Image 18 Image 19 Image 20 Image 21 Image 22 Image 23 Image 24 Image 25 Image 26 Image 27 Image 28 Image 29 Image 30 Image 31 44.6699 36.6242 57.9026 55.8079 51.7139 55.1247 54.1993 53.3235 49.8209 56.3098 57.4865 76.994 75.3437 75.7451 72.9491 72.0666 67.9041 63.5098 67.2230 65.9541 66.4657 62.9119 66.3750 68.7174 36.8495 36.5772 36.4418 35.7769 34.1989 35.6062 36.5106 0.043616 0.053906 0.045923 0.038306 0.044006 0.046 0.052643 0.047933 0.052948 0.04066 0.046356 0.057016 0.052534 0.042781 0.048465 0.045297 0.050343 0.0573 0.047266 0.045044 0.054529 0.055029 0.05575 0.057751 0.001372 0.002877 0.003194 0.005754 0.00293 0.006202 0.003946 Vertical 0.071741 0.077075 0.083789 0.077734 0.077087 0.067322 0.061106 0.073222 0.073639 0.058034 0.065684 0.104016 0.099584 0.098295 0.09534 0.108248 0.088654 0.089111 0.088062 0.094971 0.095325 0.093604 0.097644 0.096448 0.008499 0.015546 0.010505 0.013725 0.004408 0.001874 0.004025 Diagonal 0.000916 7.32E-05 0.002222 0.001074 0.000183 0.000346 0.001516 0.000732 0.00294 0.001801 0.002879 0.002879 0.000225 0.001991 0.000563 0.001828 0.000814 0.000464 0.000537 0.004272 8.54E-05 0.001172 0.001379 0.004602 0.001082 0.002349 0.002323 0.001056 0.001795 0.00161 0.000699 Wavelet Gabor (Sub-band 1) 0.275158 0.28023 0.247304 0.263427 0.276598 0.267245 0.28876 0.312288 0.332287 0.285401 0.27992 0.325076 0.305669 0.30544 0.294071 0.292423 0.280234 0.288493 0.300335 0.294079 0.293742 0.282132 0.301768 0.319182 0.151476 0.146255 0.135219 0.117068 0.132999 0.13789 0.155908

Table 4.4 Abnormal Images Mean

34

Variance Image
Without Transform Horizontal Image 1 Image 2 Image 3 Image 4 Image 5 Image 6 Image 7 Image 8 Image 9 Image 10 Image 11 Image 12 Image 13 Image 14 Image 15 Image 16 Image 17 Image 18 Image 19 Image 20 Image 21 Image 22 Image 23 Image 24 Image 25 Image 26 Image 27 Image 28 Image 29 Image 30 Image 31 3074.957 2216.829 2919.207 3453.95 3469.815 3096.86 3280.115 3468.613 3396.621 3013.183 3262.737 4768.797 4533.509 4685.575 4513.721 4579.436 4306.119 2904.708 3307.889 3327.111 3560.139 3186.652 2802.463 3172.501 2641.164 2780.292 2851.483 2927.764 2283.569 2417.582 2511.656 45.2334 33.04906 58.86982 57.30016 53.62047 54.13341 59.26085 59.65723 60.73324 67.5673 62.39801 80.14766 71.62744 78.90971 72.547 69.05656 60.29156 70.198 76.85706 68.24504 61.61814 51.54247 66.34095 79.34072 75.10117 72.00747 63.2016 45.76717 57.25194 66.79382 83.11073 Vertical 23.89255 16.64941 44.82321 37.80715 27.9559 41.61895 40.41683 39.96407 39.6609 68.50105 56.1362 70.45372 60.21357 63.72544 62.16007 62.94815 54.93299 55.69006 64.99517 65.98338 62.5934 46.8791 48.72643 60.32123 52.95186 51.93181 46.25931 35.66286 41.55681 48.52081 52.94149 Diagonal 1.681724 1.577053 1.835159 1.713261 1.799635 2.36158 2.638464 3.090099 3.361088 3.282204 2.669302 2.669302 3.232478 3.317919 3.181601 3.01611 2.56416 2.925376 3.422935 3.216113 2.89766 2.492735 2.943655 3.514329 3.766885 3.444566 2.906308 2.058119 2.828104 3.271357 3.973229 Wavelet Gabor (Sub-band 1) 0.226409 0.22223 0.203457 0.216925 0.228751 0.221209 0.24328 0.267676 0.301476 0.249265 0.244138 0.275816 0.262391 0.268795 0.256495 0.24539 0.229173 0.242471 0.250287 0.239318 0.235636 0.222 0.249485 0.26923 0.323131 0.302872 0.278063 0.233298 0.265601 0.288454 0.33414

Table 4.5 Abnormal Images Variance

35

Entropy Image
Without Transform Horizontal Image 1 Image 2 Image 3 Image 4 Image 5 Image 6 Image 7 Image 8 Image 9 Image 10 Image 11 Image 12 Image 13 Image 14 Image 15 Image 16 Image 17 Image 18 Image 19 Image 20 Image 21 Image 22 Image 23 Image 24 Image 25 Image 26 Image 27 Image 28 Image 29 Image 30 Image 31 0.04601 0.046057 0.046057 0.046057 0.04601 0.042044 0.042044 0.042163 0.042084 0.042044 0.042005 0.046667 0.04662 0.04662 0.046667 0.046667 0.046715 0.04601 0.046057 0.04601 0.04601 0.04615 0.046103 0.04601 0.969726 0.962768 0.958183 0.947184 0.972916 0.971439 0.973666 1.22975 1.229486 1.234505 1.231195 1.222984 1.237614 1.224281 1.220224 1.217412 1.233267 1.233912 1.183855 1.196428 1.20201 1.214072 1.2072 1.215082 1.207481 1.198508 1.200482 1.198974 1.208821 1.198707 1.190959 0.798299 0.777796 0.769407 0.764631 0.802867 0.805303 0.812416 Vertical 1.360388 1.354789 1.358728 1.347056 1.343129 1.314423 1.3119 1.287376 1.29642 1.305855 1.308709 1.259021 1.283083 1.280835 1.288439 1.29658 1.306171 1.274256 1.272535 1.270475 1.27809 1.28574 1.271244 1.257168 0.804175 0.791559 0.783881 0.771277 0.822998 0.81776 0.813293 Diagonal 1.518533 1.522715 1.52438 1.531343 1.525853 1.506982 1.49547 1.482197 1.48085 1.500817 1.509876 1.474565 1.485851 1.488458 1.495205 1.498235 1.510551 1.485508 1.470825 1.489269 1.487759 1.49064 1.485709 1.466409 0.877173 0.859575 0.857869 0.837862 0.89641 0.886377 0.888803 Wavelet Gabor (Sub-band 1) 7.074133 7.102217 6.980089 7.038835 7.08164 7.080376 7.149141 7.21587 7.249478 7.13097 7.122963 7.178054 7.143073 7.133883 7.099794 7.120152 7.086348 7.113188 7.146436 7.136099 7.142772 7.142772 7.148212 7.185435 3.950262 3.90922 3.824544 3.692486 3.939474 3.95897 3.988039

Table 4.6 Abnormal Images Entropy

36

4.3 RESULTS FOR SVM PERFORMANCE MEASUREMENT After the feature extraction of normal and abnormal images, the extracted results are classified using the SVM classifier with linear kernel and the performance of SVM classifier is measured. The performance measured without applying transforms is shown below,

Accuracy 73.33

Precision 73.33

Sensitivity 100

Specificity 0

Table 4.7 Performance Measures without applying transforms

4.3.1 WAVELET PERFORMANCE MEASURES

Accuracy Horizontal Vertical Diagonal 73.3 73.3 73.3

Precision 73.3 76.9 73.3

Sensitivity 100 90.9 100

Specificity 0 25 0

Table 4.8 Performance Measures of DWT based SVM

37

4.3.2 GABOR PERFORMANCE MEASURES

Sub band Sub band 1 Sub band 2 Sub band 3 Sub band 4 Sub band 5 Sub band 6 Sub band 7 Sub band 8 Sub band 9 Sub band 10 Sub band 11 Sub band 12 Sub band 13 Sub band 14 Sub band 15 Sub band 16 Sub band 17 Sub band 18 Sub band 19 Sub band 20 Sub band 21 Sub band 22 Sub band 23 Sub band 24 Sub band 25 Sub band 26 Sub band 27 Sub band 28 Sub band 29

Accuracy 100 73.33 73.33 73.33 80 73.33 73.33 73.33 80 73.33 73.33 73.33 80 86.67 80 73.33 73.33 73.33 80 73.33 73.33 80 80 73.33 73.33 73.33 80 80 80

Precision 100 76.9 78.57 78.57 78.57 73.33 73.33 78.57 78.57 73.33 73.33 73.33 78.57 100 78.57 73.33 73.33 78.57 78.57 73.33 73.33 78.57 78.57 73.33 73.33 73.33 78.57 83.33 78.57

Sensitivity 100 90.91 100 100 100 100 100 100 100 100 100 100 100 81.81 100 100 100 90.9 90.9 100 100 100 100 100 100 100 100 90.9 100

Specificity 100 25 25 25 25 0 0 25 25 0 0 0 25 100 25 0 0 0 0 0 0 25 25 0 0 0 25 50 25

38

Sub band 30 Sub band 31 Sub band 32 Sub band 33 Sub band 34 Sub band 35 Sub band 36 Sub band 37 Sub band 38 Sub band 39 Sub band 40

73.33 73.33 80 80 73.33 73.33 73.33 73.33 80 73.33 73.33

73.33 73.33 78.57 78.57 73.33 73.33 73.33 73.33 78.57 73.33 73.33

100 100 100 100 100 100 100 100 100 100 100

0 0 25 25 0 0 0 0 25 0 0

Table 4.9 Performance Measures of Gabor based SVM

39

CHAPTER 5 COMPARISON OF RESULTS AND ANALYSIS

Performance measures Without transform With DWT Horizontal Vertical Diagonal With Gabor transform (Sub-band -1)

Accuracy 73.33 80 86.67 86.67 100

Precision 73.33 100 84.6 100 100

Sensitivity Specificity 100 100 100 100 100 0 100 50 100 100

Table 5.1 Performance Measures Comparison After obtaining the results, they are compared with one another to determine which provide good results. From the obtained results, the following things are inferred: 1. If images are classified directly without applying any transform, it provided a classification accuracy of 73.33%, but specificity becomes 0.This means that SVM classifier finds more difficulty in recognizing the abnormal images, when it is fed with features from untransformed images as inputs. 2. If images are classified by extracting features through wavelet transform, it provided a classification accuracy of 73.33%, precision of 76.8% and specificity of 25%.This means that SVM classifier finds more difficulty in recognizing about 3/4th of abnormal images available, when it is fed with features from wavelet transformed images as inputs.
40

3. If images are classified by extracting features through Gabor transform with 5 scales and 8 orientations, it provided a classification accuracy of 100%,precision of 100% and specificity of 100% even in the first sub band itself(scale-1,orientation-1).This means that SVM classifier finds more easiness when it is fed with simply first sub band of Gabor extracted features.

41

CHAPTER 6 CONCLUSION The real world data is generally imperfect for two reasons: one is that the data can be incomplete for the lack of the necessary information; the other is that data may be inaccurate, because it includes the noise and even the wrong information. With this incomplete real world data, how to extract the specific image feature is still a present major issue. Feature extraction and selection are essential in case of image classification. Once we have extracted the features, the feature selection aiming at improving classification accuracy and reducing the restless feature plays an important role in classification because the SVM has its influence on classification. In this project, the proposed approach based on linear SVM has demonstrated great potential and usefulness in MRI image classification. From the above analysis ,it is clear that medical images are perfectly classified with 100% classification accuracy when they are given to Gabor transform based Support Vector Machine (SVM) with linear kernel. Also, in addition to 100% classification accuracy, it provides high recognition efficiency and high speed convergence when comparing to discrete wavelet transform based Support Vector Machine (SVM) with linear kernel. Also, results show that Gabor extracted features provide greater and absolute resolution to images in their classification process. These entire factors make Gabor Transform based Support Vector Machine (SVM) with linear kernel more efficient than other classifiers in medical image classification process.

42

REFERENCES

[1] Antonio V. Netto, Cesar B. Castanon , C. P. De Almeida, Joo E. S. B. Neto, Maria Cristina F. de Oliveira Osvaldo, Ridgelet based Texture Classification of Tissues in Computed tomography, Machine Vision and Applications, Springer-Verlag, 2008 [2] Li Bai , Ulas Bagci, A Comparison of Daubechies and Gabor Wavelets for Classification of MR Images, Collaborative Medical Image Analysis Group (CMIAG), 2007 [3] Lindsay Semler, Lucia Dettori, A Comparison of Wavelet-based and Ridgelet based texture Classification of tissues in Computed Tomography, Springer-Verlag, 2008, p.no:21-29 [4] P. Carrion, E. Cernadas, M. Fernandez-Delgado, J.F. Galvez, Pilar SaOtero, Improved Classification of Pollen texture Images using SVM and MLP, Springer-Verlag, 2008, p.no:115-119 [5] Qian Zhu , Shaomei Yang, Comparison and Application of SVM and FNN Algorithm, Vol :1,2007 [6] Abdel-Badeeh, El-Dahshan, El-Sayed, M.Salem, Tamer.H.Younis, A Hybrid Technique for Automatic MRI Brain Images Classification, Studia Univ. Babesbolyai, Informtica,Vol:54,2009 [7]M. Fletcher-Heath, D. B. Goldgof, L. O. Hall, F. R. Murtagh, Automatic segmentation of non-enhancing brain tumors in magnetic resonance images, Artificial Intelligence in Medicine 21 (2001), pp. 43-63

43

[8] Wei-Li Zhang, Xi-Zhao Wang, Feature Extraction and Classification for Human Brain CT images, International Conference on Machine Learning and Cybernetics, Hong Kong, 2007 [9] Dinstein, Haralick, R. M., Shanmugam, D., Texture Features for Image Classification, IEEE Transactions on Systems and Cybernetics, Vol:3,1973, p.no: 610 621 [10] Vapnik, V. N., The Nature of Statistical Learning Theory,(New York: Springer-Verlag), 1995. [11] Chih-Chung Chang, Chih-Jen Lin, Chih-Wei Hsu, A Practical Guide to Support Vector classification, http://www.csie.ntu.edu.tw/~cjlin, 2009 [12] C. Chang, C. Lin, LIBSVM: a library for support vector machines, 2001. [13] Anna Rejani, Y.Ireaneus, Dr. S. Thamarai Selvi, Early Detection of Breast Cancer using SVM Classifier Technique, International Journal on Computer Science and Engineering , 2009, Vol.1,p.no:127-130. [14] S. Mika, K.R. Muller, An introduction to kernel-based learning algorithms, IEEE Transactions on Neural Network, 2001, Vol: 12, p.no:181201. [15] V. Vapnik, Statistical learning theory, Wiley, New York, 1998 [16] Ahmet Sertbas, Niyazi Kilic, Onur Osman, Osman N. Ucan, Pelin Gorgel, , Mammographic Mass Classification Using Wavelet based Support Vector Machine, Journal of Electrical & Electronics Engineering,Vol:9,2009,p.no:867-875

44