International Journal of Information Technology & Decision Making Vol. 10, No.

6 (2011) 11611174 c World Scientic Publishing Company DOI: 10.1142/S0219622011004750

A FUZZY LINEAR PROGRAMMING-BASED CLASSIFICATION METHOD

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

AIHUA LI School of Management Science and Engineering Central University of Finance and Economics Beijing 100080, P. R. China aihuali@cufe.edu.cn YONG SHI Fictitious Economics and Data Technology Research Centre Chinese Academy of Science Beijing 100081, P. R. China College of Information Science and Technology University of Nebraska NE 68182, USA yshi@gucas.ac.cn JING HE and YANCHUN ZHANG Centre for Applied Informatics Victoria University, Melbourne City MC VIC 8001, Australia Fictitious Economics and Data Technology Research Centre Chinese Academy of Science Beijing 100081, P. R. China Jinghe@gucas.ac.cn yzhangvu@gmail.com

Multiple criteria linear programming and multiple criteria quadratic programming classication models have been applied in some eld in nancial risk analysis and credit risk control such as credit cardholders behavior analysis. In this paper, a fuzzy linear programming classication method with soft constraints and criteria was proposed based on the previous ndings from other researchers. In this method, the satised result can be obtained through selecting constraint and criteria boundary variable di , respectively. A general framework of this method is also constructed. Two real-life datasets, one from a major USA bank and the other from a database of KDD 99, are used to test the accurate rate of the proposed method. And the result shows the feasibility of this method. Keywords : Classication; data mining; MCLP; fuzzy linear programming; membership function.

Corresponding

author. 1161

1162

A. Li et al.

1. Introduction Data mining becomes an important international technology with the development of database and internet, which can extract nontrivial, implicit, previously unknown and potential useful patterns, or knowledge from database. Classication is one of the functions in data mining, which is a kind of supervised learning. There are two steps in the classication process.1 First, hidden pattern or discriminant function can be derived from the training set. Second, the pattern or discriminant function is applied to classify the testing dataset. The training accurate rate and testing accurate rate are often used to evaluate the model. The term of classication methods initially employ articial intelligent (AI), traditional statistics and machine learning tools, such as decision tree,2 linear discriminant analysis (LDA),3 support vector machine (SVM),4 and so on. They have been applied in real-life medical, communication, and strategic management problems. For dierent datasets with dierent characters, classication methods show their dierent advantages and disadvantages. For example, SVM or neural network (NN) ts well for the output of some dataset, but it may result in overt problem sometimes. LDA shows its advantage when the datasets obey normal distribution, but not a good choice in other conditions. Linear programming (LP) classication method was rst proposed in 1980s,57 which showed its potential applications. In 1990s, multiple criteria linear programming (MCLP) and multiple criteria quadratic programming (MCQP) classication models were developed,810 which have been successfully used in credit cardholders behavior analysis1114 and network intrusion detection later.15 He et al.16 proposed a fuzzy linear programming (FLP) model only with soft criteria, in which a satised solution could be solved. In this paper, we proposed a FLP classication method with soft constraints and criteria based on the previous researchers work. This paper is presented as follows: Sec. 2 reviews LP, MCLP, and FLP method. Section 3 proposes FLP with soft constraints and criteria, which means decision maker can choose the reasonable bound for constraints in deriving a satised solution. Section 4 uses two examples, one from a major USA bank and the other from the database of KDD 99,17 to test the accurate rate of the proposed method. Some remarks are given in Sec. 5.

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

2. LP, MCLP, and FLP Classication Models In the LP classication method, the objectives of initial forms can be categorized as MMD and MSD.6 Here, MMD means maximize the minimum distance of observations from the critical value. MSD means minimize the sum of the distance of the observations from the critical value. For example, in the credit cardholder behavior analysis a basic framework of two-class problems can be presented as. Given a set of r variables (attributes) about a cardholder a = (a1 , a2 , . . . , ar ), let Ai = (Ai1 , Ai2 , . . . , Air ) be the development sample of data for the variables,

A Fuzzy Linear Programming-based Classication Method

1163

where i = 1, 2, . . . , n and n is the sample size. We want to determine the best coecients of the variables, denoted by X = (x1 , x2 , . . . , xr )T , and a boundary value b (a scalar) to separate two classes: G (Good for nonbankrupt accounts) and B (Bad for bankrupt accounts), that is as follows: Ai X b, Ai X b, Ai B (Bad), Ai G (Good).

To measure the separation of Good and Bad, we dene:

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

i = the overlapping of two-class boundary for case Ai (external measurement); = the maximum overlapping of two-class boundary for all cases Ai (i < ); i = the distance of case Ai from its adjusted boundary (internal measurement); = the minimum distance for all cases Ai from its adjusted boundary (i > ). A simple version of Freed and Glovers model which seeks MSD can be written as Minimize
i

i , (2.1)

Subject to: Ai X b + i , Ai B, Ai X b i , Ai G, where Ai are given, X and b are unrestricted, and i 0. The alternative of the above model is to nd MMD as follows: Maximize
i

i , (2.2) Ai B, Ai G,

Subject to: Ai X b i , Ai X b + i ,

where Ai are given, X and b are unrestricted, and i 0. A hybrid model7 that combines models (2.1) and (2.2) can be as follows: Minimize
i

i
i

i , (2.3) Ai B, Ai G,

Subject to: Ai X = b + i i , Ai X = b i + i ,

where Ai are given, X and b are unrestricted, and i , i 0, respectively. Shi et al.8 applied the compromise solution of MCLP to minimize the sum of i and maximize the sum of i simultaneously. A two-criteria LP model is stated

1164

A. Li et al.

as follows: Minimize
i

and Maximize
i

i , (2.4)

Subject to: Ai X = b + i i , Ai X = b i + i , Ai B, Ai G,

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

where Ai are given, X and b are unrestricted, and i , i 0, respectively. For this model MCLP, the system explanation and summary are presented in these papers.1820 In compromise solution approach,21 the best trade-o between i i and i i is identied for an optimal solution. To explain this, assume the ideal value of i i be > 0 and the ideal value of i i be > 0. Then, if i i > , the regret measure is dened as d+ = i i + . Otherwise, it is dened as 0. If i i < , the regret measure is dened as d = + i i ; otherwise, it + is 0. Thus, the relationship of these measures are (i) + i i = d d , (ii) + + | +i i | = d +d , and (iii) d , d 0. Similarly, we derive i i = d d+ , + + + d , and d , d 0. | i i | = d An MCLP model for two-class separation is presented as
+ + Minimize d + d + d + d

Subject to: +
i + i = d d , + i = d d ,

(2.5)

Ai X = b + i i , Ai X = b i + i ,

Ai B, Ai G,

+ where Ai , , and are given, X and b are unrestricted, and i , i , d , d , d , + d 0. In a FLP approach with soft criteria,16 membership functions for the criteria Minimize i i and Maximize i i were expressed respectively by if i y1U 1, i i i y1L , if y1L < i < y1U , F 1 (x) = y 1U y1L i if i y1L 0, i

A Fuzzy Linear Programming-based Classication Method

1165

1, if i y2U i i i y2L , if y2L < i < y2U . F2 (x) = y2U y2L i if i y2L 0,
i

Then, a fuzzy classication method for relaxing criteria given as

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

i and

i is

Maximize , Subject to:

i i y1L , y1U y1L i

i y2L , y2U y2L Ai X = b + i i , Ai X = b i + i , Ai G, Ai B,

(2.6)

where Ai , y1L , y1U , y2L , and y2U are known, X and b are unrestricted, and i , i , 0. 3. A FLP Classication Method with Soft Constraints and Criteria It has been recognized that in many decision-making problems, instead of nding the exist optimal solution (a goal value), decision makers often approach a satisfying solution between upper and lower aspiration levels that can be represented by the upper and lower bounds of acceptability for objective payos, respectively.22,18 This behavior, which has an important and pervasive impact on human decision making,23 is called the decision makers goal-seeking and compromise behavior. Zimmermann applied it as the basis of his pioneering work on FLP.24 FLP problem can be described as follows25 : B (x ) = max(D(X ) F (X )) = max{|D(x) , F (x) , 0}
x X

= max{|D1 (x) Dm (x) , F (x) , 0}, where fuzzy sets, D(x) and F (x), are transferred from the constraints and criteria of general programming problem with the membership function Di and F , respectively. x is the satisfying solution of original programming problem. And the fuzzy method has been used widely in some leds.26,27

1166

A. Li et al.

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

For decision makers, since the optimal solution is not necessary at most time, satisfying solution may be enough to solve real-life problems. In the model MSD and MMD, the crisp distance measurements (i and i ) of observations in criteria and constraints are used to evaluate the classication model in application. To consider the exibility of the choices for these measurements in obtaining a satisfying solution, the crisp criteria and constraints are relaxed to soft criteria and constraints. This means that we can allow the exible boundary b for classication scalar to derive the result what we expect in reality. Based on this idea, a FLP method with both soft criteria and constraints is constructed by the following steps. First, we dene the membership functions for the MSD problem with soft criterion and constraints as follows: 1, if i i y1L , i i y1U , if y1L < i < y1U , F1 (x) = y1L y1U i 0, if i y1U ,
i

1, D1 (x) = 1 1 [Ai X (b + i )], d1 0, 1,

if Ai X b + i , if b + i < Ai X < b + i + d1 , if Ai X b + i + d1 , if Ai X b i ,

D2 (x) = 1 + 1 [Ai X b + i )], if b i d2 < Ai X < b i , d2 0, if Ai X b i d2 . Then, dene y1L = min i i and y1U (min i i , max i i ), in which the former can be computed from (2.1); thus the fuzzy MSD problem with soft criterion and constraints for (2.1) is constructed as follows: Maximize Subject to: i y1U , y1L y1U Ai X (b + i ) 1 , d1 Ai X (b i ) 1+ , d2
i

Ai B, Ai G,

(3.1)

1 > 0, where Ai are given, X and b are unrestricted, i , d1 , d2 > 0, respectively.

A Fuzzy Linear Programming-based Classication Method

1167

For the model (2.2), we similarly dene the membership function as follows: 1, if i y2U , i i i y2L , if y2L < i < y2U , F2 (x) = y2U y2L i i y2L , if 0, 1, D3 (x) = 1 + 0, 1, D4 (x) = 1 0,
i

if Ai X b i , 1 [Ai X b + i ], d3 if b i d3 < Ai X < b i , if Ai X b i d3 , if Ai X b + i , 1 [Ai X b i ], d4 if b + i < Ai X < b + i + d4 , if Ai X b + i + d4 .

i

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

Then, with the denition y2U = max a FLP for model (2.2) is built as below: Maximize Subject to: ,

i and y2L (min

i , max

i ),

i i y2L , y2U y2L Ai X b + i 1+ , d3 Ai X b i 1 , d4

Ai B, Ai G,

(3.2)

1 > 0, where Ai are given, X and b are unrestricted, i , d3 , d4 > 0, respectively. In order to unify the sign of models in this research, we use the same membership function F1 in the model of (3.1), F2 in the model of (3.2) instead of them in model (2.6), so model (2.6) would be changed into the following format: Maximize , Subject to:
i i y1U , y1L y1U

i y2L , y2U y2L

i

(3.3)

Ai X = b + i i , Ai X = b i + i ,

Ai G, Ai B,

1168

A. Li et al.

where Ai is known, X and b are unrestricted, and i , i , 0, y1L , y1U , y2L , and y2U are the same in the models of (3.1) and (3.2). To identify a fuzzy model for the model (2.4), we rst relax the model (2.4)s constraints to inequality constraints. Then, suppose d1 = d2 = d 1 , d3 = d4 = d2 , a fuzzy model with the combinations (3.1) and (3.2) for the relaxed (M4) will be Maximize , Subject to: i y1U , y1L y1U
i i i y2L , y2U y2L

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

1+ 1 1+ 1

Ai X (b + i i ) , d 1 Ai X (b + i i ) , d 1 Ai X (b i + i ) , d 2

Ai B, Ai B, Ai G, Ai G,

(3.4)

Ai X (b i + i ) , d 2 1 > 0,

where Ai are given, X and b unrestricted, i , i > 0, respectively. d i > 0, i = 1, 2 are xed in the computation. The denitions of y1L , y1U , y2L , and y2U are the same as those in models (3.1) and (3.2), respectively. There are two pieces of dierence between the models (3.4) and (2.4). First, instead of optimal solution, a satisfying solution is obtained based on the membership function from the FLP. Second, with these soft constraints to the model (2.4), the boundary b can be exibly moved by the upper bound and the lower bound with the separated distance di , i = 1, 2, 3, 4 according to the characteristics of the data. 4. Experimental Studies There are two datasets used here to test the accuracy rate of the proposed fuzzy classication method with both soft criteria and constraints. The rst dataset came from a major US bank with 65 attributes which include the credit cardholders over limit fee, over charge fee, and other information in credit card using history, etc. There are in total 6000 records in the dataset. Here we compare the proposed FLP with both soft criteria and constrains with MSD, MMD, and MCLP model. We select 1400 records with 700 Good (nonbankrupt) and 700 Bad (bankrupt) randomly from the dataset for training, and the left 4600 are used to test the classier accuracy, which is based on the method of cross validation. In the experiment,

A Fuzzy Linear Programming-based Classication Method

1169

b is given as 0.5 for all models, d1 = d2 = d 1 = 1, d3 = d4 = d2 = 1.5, respectively for fuzzy-model (3.4). There are ve groups training results in Table 1 and testing results in Table 2 listed below. In Tables 13, we dene

Absolute accurate rate of Good = Sensitivity =

t Good , Good t Bad , Absolute accurate rate of Bad = Specicity = Bad Catch rate = Accuracy = Sensitivity + Specicity Good Good + Bad

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

Bad , Good + Bad

where t Good is the number of the Good (Good records that were correctly classied as much). Good is the number of Good; t Bad is the number of the Bad (Bad records that were correctly classied as much). Bad is the number of Bad. In this case, to catch a bad person is more important than to catch a good cardholder in order to avoid the defaulting. Tables 1 and 2 show us that model (2.5)-MCLP, fuzzy-model (3.3)-FLP1, and the proposed model (3.4)-FLP2 are better than (2.1)-MSD and (2.2)-MMD. Although model (2.2)-(MMD) is the best for Bad catching it cannot be selected due to its poor Good catching and instability in the experiment. MCLP shows its trade-o with the balanced Good and Bad accuracy rate. FLP1 works well for the overall catch rate and a little worse than FLP2 for Bad catching. Thus, among MSD, MCLP, and FLP if we give importance to catching Bad cardholder and keeping a satised absolute accuracy rate, fuzzy model (3.4) would be a good choice. Table 3 shows us that the choice of boundary value d i in the model (3.4) aects the result of classication. By adjusting the value of d i , we can get the satised result of classication in the training process. The second dataset came from KDD 99. Here a connection is a sequence of TCP packets starting and ending between which data ows from a source IP address to a target IP address under some well-dened protocol. Each connection is labeled as either normal or an attack, here dos is exactly one specic attack type. In this task, we select 38 characters needed. There are 1,060,078 records in the dataset we used in this example; 812,812 Normalrecords and 247,266 Dos records. First, 4000 records was selected randomly from the dataset for training, 2000 of which is labeled Normal, the other 2000 is labeled Dos. Second, the left records, 810,812 records for Normal and 245,266 records for Dos were used for testing. Tables 4 and 5 show us the training and testing results. In Tables 4 and 5, we use: Absolute accurate rate of Normal = Sensitivity = t Normal , Normal

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

1170

A. Li et al.

Table 1. Training results of 1400 records. Model 5 (MCLP) Absolute Accuracy Rate Fuzzy Model 3 (FLP1) Absolute Accuracy Rate Fuzzy Model 4 (FLP2) Absolute Accuracy Rate Catch Rate 0.62 0.68 0.67 0.62 0.58 0.82 0.83 0.79 0.84 0.84 0.72 0.75 0.73 0.73 0.71

Dierent Groups

Model 1 (MSD) Absolute Accuracy Rate

Model 2 (MMD) Absolute Accuracy Rate

Good Bad Catch Rate Good Bad Catch Rate Good Bad Catch Rate Good Bad Catch Rate Good Bad 0.06 0.04 0.04 0.06 0.28 0.93 0.90 0.93 0.91 0.60 0.50 0.47 0.49 0.48 0.44 0.74 0.79 0.77 0.76 0.73 0.74 0.79 0.77 0.75 0.78 0.74 0.79 0.77 0.76 0.75 0.77 0.75 0.74 0.75 0.28 0.80 0.78 0.78 0.79 0.60 0.78 0.76 0.76 0.77 0.44

Group1 Group2 Group3 Group4 Group5

0.67 0.70 0.69 0.69 0.72

0.68 0.70 0.70 0.70 0.69

0.68 0.70 0.70 0.69 0.71

Table 2. Testing results of 4600 records. Model 5 (MCLP) Absolute Accuracy Rate Fuzzy Model 3 (FLP1) Absolute Accuracy Rate Fuzzy Model 4 (FLP2) Absolute Accuracy Rate Catch Rate 0.72 0.74 0.76 0.74 0.72 0.78 0.74 0.75 0.70 0.63 0.73 0.74 0.76 0.74 0.72 0.61 0.63 0.67 0.65 0.61 0.84 0.79 0.78 0.82 0.78 0.62 0.64 0.67 0.66 0.62

Dierent Groups

Model 1 (MSD) Absolute Accuracy Rate

Model 2 (MMD) Absolute Accuracy Rate

Good Bad Catch Rate Good Bad Catch Rate Good Bad Catch Rate Good Bad Catch Rate Good Bad 0.03 0.03 0.03 0.04 0.28 0.92 0.91 0.90 0.88 0.63 0.08 0.08 0.08 0.09 0.30 0.75 0.73 0.75 0.77 0.75 0.74 0.79 0.72 0.68 0.72 0.75 0.74 0.75 0.76 0.75

Group1 Group2 Group3 Group4 Group5

0.70 0.69 0.72 0.69 0.70

0.77 0.71 0.73 0.70 0.68

0.70 0.69 0.72 0.69 0.70

A Fuzzy Linear Programming-based Classication Method Table 3. Training and testing result of 1400 records for fuzzy model 4. Dierent di d1 = d2 1 1 1 1 d3 = d4 3 2 1.5 1 Training: Absolute Accuracy Rate Good 0.547 0.574 0.619 0.673 Bad 0.897 0.863 0.823 0.746 Catch Rate 0.722 0.719 0.721 0.709

1171

Testing Absolute Accuracy Rate Good 0.538 0.570 0.607 0.670 Bad 0.896 0.877 0.838 0.777 Catch Rate 0.558 0.588 0.620 0.676

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

Absolute accurate rate of Dos = Specicity = Catch rate = Accuracy = Sensitivity

t Dos , Dos

Dos Normal + Specicity , Normal + Dos Normal + Dos

where t Normal is the number of the Normal (Normal records that were correctly classied as much). Normal is the number of Normal; t Dos is the number of the Dos (Dos records that were correctly classied as much). Dos is the number of Dos. In this experimental study, MMD shows the same character as the credit cardholder dataset analysis. But the result of comparison is not very clear from the separate group training and testing result, so we compute the average value to analyze the classication eciency. The average value tells that MCLP and FLP2 show better catch rate in testing. MSD works well for Dos catching and fuzzy model (3.4) FLP2 does a little worse than that. In this paper, we just compared the proposed FLP classication method with MMD, MSD, and MCLP in two real-life datasets. As references, the readers can nd the previous works comparing MCLP and FLP with soft criteria, decision tree, and neural network in Refs. 9, 10 and 17. Thus, we shall not elaborate the comparison of this FLP method with other classication methods.

5. Remarks In this paper, a FLP classication method with both soft criteria and constraints is proposed based on the previous researchers works. The relationship between this model and other related models was discussed. Two real-life datasets, one from the real bank in USA and the other from KDD 99, have been used to evaluate the accurate rate of classication. The result shows the feasibility of this method. Moreover, the general framework of FLP for classication have been described for the rst time systemically and evaluated. However, there is some new research work to be considered and continued in the line of research. For example, how does the value di aect the result of classication? How can we consider ensemble analysis to improve the selection of the best classier? We shall report the signicant results of these ongoing projects in the near future.

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

1172

A. Li et al.

Table 4. Training results of 4000 records. Model 2 (MMD) Absolute Accuracy Rate Normal 0.508 0.269 0.232 0.263 0.318 0.919 0.972 0.992 0.982 0.966 0.713 0.621 0.612 0.622 0.642 0.998 0.992 0.993 0.994 0.994 0.993 0.998 0.998 0.997 0.997 0.995 0.995 0.995 0.995 0.995 0.989 0.99 0.987 0.990 0.989 Dos Catch Rate Normal Dos Catch Rate Normal Model 5 (MCLP) Absolute Accuracy Rate Fuzzy Model 4 (FLP2) Absolute Accuracy Rate Dos 0.997 0.996 0.997 1.000 0.998 Catch Rate 0.993 0.993 0.992 0.995 0.993

Dierent Groups

Model 1 (MSD) Absolute Accuracy Rate

Normal

Dos

Catch Rate

Group1 Group2 Group3 Group4 Average

0.989 0.991 0.987 0.989 0.989

0.997 0.995 0.998 0.997 0.997

0.993 0.993 0.992 0.993 0.993

Table 5. Testing results of other records. Model 2 (MMD) Absolute Accuracy Rate Normal 0.499 0.323 0.254 0.290 0.342 0.916 0.980 0.988 0.982 0.967 0.595 0.476 0.424 0.451 0.487 Dos Catch Rate Normal 0.975 0.968 0.930 0.963 0.959 Model 5 (MCLP) Absolute Accuracy Rate Dos 0.983 0.989 0.987 0.985 0.986 Catch Rate 0.977 0.973 0.943 0.968 0.965 Fuzzy Model 4 (FLP2) Absolute Accuracy Rate Normal 0.953 0.959 0.966 0.954 0.958 Dos 0.988 0.988 0.988 0.988 0.988 Catch Rate 0.961 0.965 0.972 0.962 0.965

Dierent Groups

Model 1 (MSD) Absolute Accuracy Rate

Normal

Dos

Catch Rate

Group1 Group2 Group3 Group4 Average

0.918 0.971 0.925 0.914 0.932

0.990 0.986 0.989 0.989 0.989

0.935 0.975 0.940 0.931 0.945

A Fuzzy Linear Programming-based Classication Method

1173

Acknowledgments The authors would like to thank Professor S. Cheng for his patience and encouragements on this work. They also express their thanks to Mr. G. Kou and P. Zhang for their constructive comments in preparing this paper. This research is partially supported by the grants (70531040, 70472074 and 70921061) from the National NSFC, the third 211 construction funding and Program for Innovation Research in CUFE. References
1. J. Han and M. Kamber, Data Mining: Concepts and Techniques (Academic Press, Beijing, 2001), p. 28. 2. J. R. Quinlan, Induction of decision tree, Machine Learning 1 (1986) 81106. 3. R. A. Fisher, The use of multiple measurements in taxonomic problems, Annals of Eugenics 7 (1936) 179188. 4. V. Vapnik, Statistical Learning Theory (Wiley, New York, 1998). 5. N. Freed and F. Glover, Simple but powerful goal programming models for discriminant problems, European Journal of Operational Research 7 (1981) 4460. 6. N. Freed and F. Glover, Evaluating alternative linear programming models to solve the two-group discriminant problem, Decision Sciences 17 (1986) 151162. 7. F. Glover, Improve linear programming models for discriminant analysis, Decision Sciences 21 (1990) 771785. 8. Y. Shi, M. Wise, M. Luo and Y. Lin, Data mining in credit card portfolio management: A multiple criteria decision making approach, in Multiple Criteria Decision Making in the New Millennium (Springer, Berlin, 2001), pp. 427436. 9. Y. Shi, Y. Peng, X. Xu and X. Tang, Data mining via multiple criteria linear programming: Applications in credit card portfolio management, International Journal of Information Technology and Decision Making 1 (2002) 145166. 10. G. Kou, X. Liu, Y. Peng, Y. Shi, M. Wise and W. Xu, Multiple criteria linear programming approach to data mining: Models, algorithm designs and software development, Optimization Mathods and Software 18 (2003) 453473. 11. Y. Peng, G. Kou, Y. Shi and Z. Chen, A descriptive framework for the eld of data mining and knowledge discovery, International Journal of Information Technology and Decision Making 7 (2008) 639682. 12. A. Li, Y. Shi and J. He, MCLP-based methods for improving Bad catching rate in credit cardholder behavior analysis, Applied Soft Computing 8 (2008) 12591265. 13. Y. Peng, G. Kou, Y. Shi and Z. Chen, A multi-criteria convex quadratic programming model for credit data analysis, Decision Supply System 44 (2008) 10161030. 14. J. He, Y. Zhang, Y. Shi and G. Huang, Domain-driven classication based on multiple criteria and multiple constraint-level programming for intelligent credit scoring, IEEE Transactions on Knowledge and Data Engineering 22 (2010) 826838. 15. G. Kou, Y. Peng, Z. Chen and Y. Shi, Multiple criteria mathematical programming for multi-class classication and application in network intrusion detection, Information Sciences 179 (2009) 371381. 16. J. He, X. Liu, Y. Shi, W. Xu and N. Yan, Classications of credit cardholder behavior by using fuzzy linear programming, International Journal of Information Technology and Decision Making 3 (2004) 633650. 17. http:/ /kdd.ics.uci.edu/databases/kddcup99/kddcup99.html. 18. Y. Shi, The research trend of information technology and decision making in 2009, International Journal of Information Technology and Decision Making 9 (2010) 18.
Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

1174

A. Li et al.

Int. J. Info. Tech. Dec. Mak. 2011.10:1161-1174. Downloaded from www.worldscientific.com by 85.74.84.134 on 10/23/12. For personal use only.

19. Y. Shi, Multiple criteria optimization-based data mining methods and applications: A systematic survey. Knowledge and Information Systems 24 (2010) 369391. 20. Y. Shi, Current research trend: Information technology and decision making in 2008, International Journal of Information Technology and Decision Making 8 (2009) 15. 21. P. L. Yu, Multiple Criteria Decision Making: Concepts, Techniques and Extensions (Plenum Press, New York, 1985). 22. A. Charnes and W. W. Cooper, Management Models and Industrial Applications of Linear Programming (Wiley, New York, 1961). 23. P. H. Lindsay and D. A. Norman, Human Information Processing: An Introduction to Psychology (Academic Press, New York, 1972). 24. H. J. Zimmermann, Fuzzy programming and linear programming with several objective functions, Fuzzy Sets and Systems 1 (1978) 4555. 25. D. Dubois and H. Prade, Fuzzy Sets and Systems: Theory and Application (Academic Press, New York, 1980), pp. 242248. 26. S. Wang and C. Lee, A fuzzy real option valuation approach to capital budgeting under uncertainty environment, International Journal of Information Technology and Decision Making 5 (2010) 695713. 27. A. Nachev, S. Hill, C. Barry and B. Stoyanov, Fuzzy, distributed, instance counting, and default artmap neural networks for nancial diagnosis, International Journal of Information Technology and Decision Making 9 (2010) 959978.