2010 International Conference on Measuring Technology and Mechatronics Automation

Electromagnetism-like Method For Constrained Optimization Problems

Lixia Han
School of Computer Science and Technology China University of Mining and Technology Xuzhou, China e-mail: lxhan@cumt.edu.cn
AbstractElectromagnetism-like method (EM) has proved to be a very robust algorithm for solving non-differentiable and non-convex unconstrained optimization problems. In this paper, we extend EM to solve constrained optimization problems. Firstly, a new constraint handling approach is introduced which transforms constrained optimization problem into a bi-objective optimization problem, and a preference selection strategy is proposed in order to find the global optimality for the original constrained problem. Secondly, new formula for computing charges is designed for the bi-objective optimization problem. It can assign larger charges to the feasible particles while smaller ones to unfeasible particles. Thirdly, a particular formula for computing the force between two particles is proposed that encourage the particles to move towards the attractive valleys, and discourage further away from steeper hills. Finally, an adaptive movement is presented to create new particle. Simulation results on benchmark test functions indicate the potential of this new approach. Keywords-Electromagnetism-like method; optimizatio,; penalty function; attraction-repulsion constrained

Zhanli Han
Accountancy Department, Management School Jinan University Guangzhou, China e-mail: han_zhan_li@yahoo.cn function evaluations with any first or second order derivative information, and it was proved in [2] that the EM converges rapidly to the vicinity of global optimum with probability one. Ana and Edite [3] presented a modified electromagnetism-like algorithm, incorporating exploratory moves, for global optimization problem. Debels et al. [4] proposed a hybrid scatter search EM for the solution of resource constraint project scheduling problem. The computational experiments on standard benchmark dataset show that the hybrid scatter search EM is capable of producing consistently good results for challenging instance of the resource-constrained project scheduling problem and outperforms state-of-the-art existing heuristics. It is the first paper that includes an EM type methodology for the solution of a combinatorial optimization problem. Then, Wu et. al [5] extended the EM to solve the Traveling Salesman Problem (TSP) and the results show the potential of the new heuristic. In this paper, we extend the EM to solve constrained optimization problems. II. ELECTROMAGNETISM-LIKE METHOD Electromagnetism-like method (EM) is a relatively recent heuristic inspired by the attraction-repulsion mechanism of electromagnetism theory which is based on Coulombs law. In the standard EM, each sample point is treated as a charged particle that is released to an electromagnetic space. The charge of each particle relates to the objective function value, and determines the magnitude of attraction or repulsion of the particle over the other particles in the population-the better the objective function value, the higher the magnitude of attraction. Then, a direction for each particle to move is found and new particle is generated by moving along this direction. Thus, the population is updated and the method approaches the vicinity of the global optimality iteration by iteration. Generally, the EM consists of four phases: initialization, local search, calculation the total force vector and movement in the direction of the total force. III. NOVEL BI-OBJECTIVEMODEL Without loss of generality, we consider a minimization constrained problem with decision space S , which is described as follows:
f ( x) min xS s.t. g i ( x) 0 i = 1,2, , p h j ( x) = 0 j = 1, , q

I.

INTRODUCTION

In recent years, constrained optimization problems become an active field of study for its diverse applications in engineering design, molecular biology, financial planning, environmental management, and other scientific aspects. When the problems involve nonlinear or non-differential functions of many variables with complex attributes, they become difficult and hard to solve. Hence, the traditional mathematical tools, such as Quasi-Newton method, modified steepest descent method, conjugate gradient method et al., cannot be applied for solving such constrained problems. As a result, many researchers have devoted themselves in finding some reliable stochastic global optimization methods, such as electromagnetism-like method. Electromagnetism-like method is initially proposed and developed by Birbil and Fang [1] in 2003, called the standard EM, which simulates the attraction-repulsion mechanism of electromagnetism theory for solving unconstrained optimization problems. The ease implementation and flexibility of the EM gains more attentions from a lot of researchers, and it has been extended to solve a number of practical problems successfully. Birbil and Fang proposed originally EM for unconstrained global optimization. The computational results on standard test functions showed that EM can converge to the optimal solution in less number of
978-0-7695-3962-1/10 $26.00 2010 IEEE DOI 10.1109/ICMTMA.2010.783 87

(1)

Where x = ( x1 , x 2 , , x n ) is the decision vector, S is an n dimensional rectangle space defined by the parametric constraints l k x k u k , k = 1,2, n . Denote the feasible region by
I = {x S g i ( x) 0, h j ( x ) = 0, i = 1,2, , p, j = 1, , q} .

x I is called feasible solution for constrained optimization problem (1). Thus, the aim of constrained optimization problem is to find the global minimum among many local minima under constraints. For constrained optimization problem (1), the violation degree function P( x) can usually be constructed as follows.

p( x) = max{g i ( x),0} + h j ( x)
i =1 j =1

(2)

Where max{g i ( x),0}(i = 1,2,

, p) represents the violation , q)

strategy is designed which is different from that for usual multi-objective optimization problem as given any two particles of the current population. The detail is as follows. If the second objective values of two particles are both zero, we prefer to select the one with the smaller first objective value; If the second objective value of one particle is zero, and that of the other is nonzero, the one with the zero second objective value is preferred. If the second objective values of two particles are both nonzero, we prefer to select the one with the smaller second objective value. It follows from this selection strategy that feasible particles gain more opportunities than unfeasible ones to be chosen to the next generation. The particles in the population tend to become feasible iteration by iteration B. The Charge of Particles The particle charge has an important effect on the performance of the EM. It relatives to the objective function value and determines the magnitude of attraction or repulsion of the particle on other particles. A new computational formal of particles charge is presented for the bi-objective optimization model (3). The charge of particle x i equals as follows. f f (xi ) q ( x i ) = (1 + max ) sign1 ( p ( x i )) f max f min (4) 1 + (1 sign1 ( p ( x i ))) 1 + p( x i ) Where f max , f min and f ( x i ) denote the objective value for the worst particle, the best particle and particle x i according to the first objective in the current population, and sign1 () is a signal function as follows. 1 c = 0 (5) sign1 (c) = 0 c > 0 Obviously, the charge defined above has the follows properties: 1)The magnitude of feasible particles charge is range from 1 to 2; 2)The magnitude of unfeasible particle charge is located in [0,1]; 3)For feasible particles, the better the first objective function value, the higher the magnitude of attraction; 4)For unfeasible particles, the smaller the violation degree, the higher the magnitude of attraction; 5)The magnitude of feasible particles charge is always larger than that of unfeasible particles

degree of the ith inequality constraint, h j ( x) ( j = 1,

represents the violation degree of the jth equality constraint, and function p (x) means the violation degree of constraints at x . Therefore, the constrained optimization problem (1) can be transformed into the following bi-objective optimization problem. min F ( x ) = ( f ( x), p ( x)) (3)
xS

where f (x) equals the objective function as in problem (1) and p ( x) is the violation degree function. In order to find the optimal solution for problem (1), the definition of preference optimality is introduced for problem (3). Definition 1 x 0 S is called preference better than x S (denoted as x 0 P x ), if it satisfied one of the following conditions: a) If x 0 and x have the same second objective values, the first objective value of solution x 0 is smaller than that of x ; b) Otherwise, the second objective value (violation degree) of solution x 0 is smaller than that of x . Definition 2 x* S is called preference optimal to problem (3) if there exists no x S that x P x* , or, f ( x) < f ( x * ) and p ( x) = 0 . It is follows from the above definition that feasible solution is always preference better than unfeasible solution in decision space S . In fact, the preference optimal solution x * is the global optimal solution for constrained problem (1). In other words, the bi-objective problem is to find the preference optimal solution x * for problem (3) IV. THE MODIFED EM

A. Selection Strategy with Preference In order to drive the particles in the EM population toward the preference optimality x * , a preference selection

C .Calculation of the Total Force Contrary to the force in the EM, a particular force between two particles is designed for constrained optimization problems. For any two particles, the force exerted on particle x i by particle x j is computed by the following equation:

88

F ij =

x j xi x j xi

(q j qi )

(6)

As seen from equation (6), particles with relatively larger charges will attract the other particles in the population while particles smaller charges will repel the others. Generally, the preference better particle attracts the other one. In this way, EM encourages the particles to converge to the highly attractive valleys, and discourages the particles to move further away from steeper hills in the objective space. Subsequently, the total force F i exerted on particle x i by all other particles in the population is combined by means of vector summation as follows.
Fi =

(Update) Selection the N particle from set P (t ) O1 O1 as the next generation population according to the preference selections strategy. Let t = t + 1 ; (Termination) If the termination criteria is 6) satisfied, then stop, and output the best solution obtained as the approximate global optimal solution of the problem; otherwise, go to 2).
. NUMERICAL RESULTS AND DISCUSSION

5)

F
j =1

i j

(7)

D .Movement to New Position After evaluating the total force F i , the particle x i is moved to a new position in the direction of the force by a random step length as below.
x i' = x i + Fi Fi V , i = 1,2, ,N

In order to verify the effectiveness of the proposed algorithm, we choose five benchmark functions from literature [6]. The following data are recorded: Best objective value (denoted by Best), mean best objective value (denoted by Mean) and the worst objective value (denoted by Worst). In addition to reporting ours results, we compare these results with the existing other methods, and found by algorithms CHEA in [6], TCEA in [7], and IS-PAES in [8] respectively. The results obtained by each method are shown in Table 1. Note that for each of the approaches compared, 30 independent runs were performed, of 350000 fitness function evaluations each.
TABLE I.
Function (Optimal) g02 (0.803619) g06 (-6961.814) g08 (0.095825) g09 (680.630) g11 (0.750)

(8)

COMPARISIONS OF FOUR ALGORITHMS

Methods CHEA TCEA IS-PAES EMCOP CHEA TCEA IS-PAES EMCOP CHEA TCEA IS-PAES EMCOP CHEA TCEA IS-PAES EMCOP CHEA TCEA IS-PAES EMCOP Best 0.068087 0.000212 0.803376 0.803547 -6957.9506 -6941.9321 -6961.814 -6961.814 0.095825 0.095825 0.095825 0.09582504 681.3240 693.2523 680.630 680.630 0.7490 0.74904 0.750 0.7499 Mean 0.58474 0.000077 0.793281 0.7986714 -6903.774 -6873.139 -6961.81 -6961.81 0.095825 0.095825 0.095825 0.0958250 681.1016 717.0234 680.631 681.0321 0.7491 0.7601 0.750 0.750 Worst 0.04992 0.000008 0.768291 0.78364 -6945.432 -6845.432 -4859.331 -6961.814 0.095825 0.095825 0.095825 0.0958249 700.5506 744.0844 680.634 680.9528 0.7493 0.8119 0.751 0.7501

Where is the random step length uniformly distributed between 0 and 1, and V = (v1 , v 2 , v n ) is the allowed feasible movement toward the lower bound l i , or the upper 1 bound u i for the corresponding dimension, = sin 3 is
2 t

an adaptive function relating of the evolution number t . Clearly, the parameter becomes smaller and smaller with the iteration. It is reasonable that algorithm can exploit the neighborhood of the optimum at the later stage of the search. According to the movement mechanism, EM ensures that the new particle can locate in the decision space for the constrained optimization problem and particles have a nonzero probability to the unvisited regions along the total force based on the movement scheme. EMCOP: Electromagnetism-like Method for Constrained Optimization Problems (Initialization) Randomly generate N particles 1) from the decision region S as the initial population P(0) . Let the generation number t = 0 ; 2) (Local search) For each particle in population P(t ) , the linear local search scheme is used to generate an improve particle. All these new particles constitute a set denoted by O1 ; (Calculation of the total force) For each particle 3) generated by local search, the total force exerted by other particles is computed; 4) (Movement to new position) Move the particle to new position along the direction of total force with a feasible stepwise. All these new particles constitute a set denoted by O 2 ;

It can be seen from table 1 that for each test function the performance of IS_PAES is best among the five algorithms for most problems, then EMCOP follow, CHEA and TCEA compete for the worst performer. For five test functions, the proposed algorithm (EMCOP) can find the known optimal solution for all test function except function g02. Thus, the results shows the EMCOP can converge to vicinity of the optimal solution for all test function and is competitive with the existing algorithms. It is notable that the EMCOP find a better solution than the known optimal solution for test function g08 and g11. The best objective value is 0.09582504 and 0.7499 for the two functions, respectively, while the known optimal objective value is 0.095825 and 0.750. It indicates the potential of EM for constrained optimization problem. It also

89

follows from table 1 that the error between mean best objective value and the known optimal objective value is less than 0.6% for five test function. Thus, it shows the good stability of EMCOP for constrained optimization problem. Figure 1 shows the particles distribution in evolution process of EMCOP on test function F11. Figure 1 gives the particle distribution in the evolution process when generation number is 0, 10, 20 and 50, respectively. For test function g11, the global minimum is
x* = (
1 1 , ) where f ( x* ) = 0.75 . It can be seen in figure 2 2

. CONCLUSION

In this paper, electromagnetism-like method is extended for solving constrained optimization problem. It can effectively encourage the particles to converge to the attractive valleys and discourage the particles to move further away from steeper hills in the neighborhood. Compared with the existing algorithm, the proposed algorithm is superior to these algorithms. REFERENCES
[1] Rocha A. M. A. C. and Fernandes E. M. G. P..: A Modified Electromagnetism-like Algorithm Based on Exploratory Moves. In: OMS Meeting- 2nd Conference on Optimization Methods & Software, Prague, Czech Republic, July 4-7, 2007. Debels D, and Dereyck B.: A Hybrid Scatter Search/Electromagnetism Meta-Heuristic for Project Scheduling. European Journal of Operation Research, Vol. 169, 2006, pp. 638653. P. Wu H. and Chiang C.: An EM +K-opt Methods for the TSPs. In: Proceeding of 2005 Taiwan Operation Research Conference Annual Meeting, Taiwan, R.O.C., 2005 A. H. Aguirre, S. B. Rionda, Carlos A. Coello Coello, et al.: Handing Constraints Using Multiobjective Optimization Concepts. International Journal for Numerical Methods in Engineering, Vol. 59, 2004, pp.1989-2017. Carlos A. Coello Coello: Constraint Handling Using an Evolutionary Multiobjective Optimization Technique. Civil Engineering System, Vol. 17, 2000, pp. 319-346. Carlos A. Coello Coello.: Treating Constraints as Objective for Single Objective Evolutionary Optimization. Engineering Optimization, Vol.32, 2000, pp. 275-308

1(a) that the starting position of the particles is evenly distributed after the initialize in decision space. According to the attraction-repulsion mechanism, the particles in population move towards the feasible region
2 I = {x x2 = x1 ,1 xi xi , i = 1,2}

[2]

in

figure

1(b)

when
[3]

generation number equals 5. In twentieth generation, all 2 particles are located in the curve with x1 = x2 . Finally, the most attractive region is observed, and global optimum is found in figure 1(d). The particles in the current population are collected near the global optimal solution at
x* = (
1 1 , ) . It can be seen from figure 1 that the EMCOP 2 2

[4]

[5]

can converge to the vicinity of the optimal solution rapidly and is effective for constrained optimization problem.

[6]

Figure 1 Particle distribution with evolution process in decision space for g11

90