Documente Academic
Documente Profesional
Documente Cultură
DOI 10.1007/s00158-006-0021-2
R E S E A R C H PA P E R
Received: 5 September 2005 / Revised manuscript received: 12 January 2006 / Published online: 8 July 2006
Springer-Verlag 2006
Abstract In this paper, the basic characteristics of particle why many metaheuristic methods utilize multiple-point
swarm optimization (PSO) for the global search are discussed search.
at first, and then the PSO for the mixed discrete nonlinear Few researches on the PSO for the discrete design vari-
problems (MDNLP) is suggested. The penalty function ap- ables problems have been reported (Venter and Sobieski
proach to handle the discrete design variables is employed, in 2003; Venter and Sobieski 2004). However, these reports
which the discrete design variables are handled as the contin- handled the discrete design variables as the continuous ones,
uous ones by penalizing at the intervals. As a result, a useful directly. That is, firstly, all design variables may be handled
method to determine the penalty parameter of penalty term as the continuous design variables, and optimized. Secondly,
for the discrete design variables is proposed. Through typical the round-off or cut-off procedure can be applied to get a
mathematical and structural optimization problems, the va- discrete optimum. These approaches to handling the discrete
lidity of the proposed approach for the MDNLP is examined. design variables may be valid in a sense, but some problems
are included. Let us consider the following two cases shown
Keywords Global optimization Particle swarm in Fig. 1a,b.
optimization Mixed discrete nonlinear problems Figure 1a shows a case where an optimum xL is obtained
Penalty function approach when the discrete design variables are handled as the con-
tinuous ones. Points A and B represent the discrete design
variables close to xL . In this case, point B is selected as
1 Introduction the neighborhood of xL by the round-off procedure. How-
ever, the objective function at point B makes a change of
The particle swarm optimization (PSO), which mimics social the function value worse in comparison with the objective
behavior, is an optimization technique developed by Kennedy function at point A. This case shows that the better discrete
and Eberhart (2001). It has been reported that the PSO is design point is not always selected by the round-off procedure
suitable for the minimization of nonconvex and continuous (Rao 1996). Another case shown in Fig. 1b is well known.
function through many numerical examples, and it is often That is, the discrete optimum obtained by the round-
said that the PSO can find a global minimum of nonconvex off or cut-off procedure does not satisfy all feasibilities
function with high certainty. Many methods, called meta- (Fu et al. 1991).
heuristics, have some common characteristics as follows: The PSO is suitable for the optimization of nonconvex and
(1) Metaheuristic methods are applicable to the nondifferen- continuous function. Therefore, all design variables should
tiable optimization problems because these methods do be handled as the continuous ones whenever the PSO is
not require the gradient information of functions. applied to the mixed or discrete design variables problems.
(2) Compared with the one-point search method, it is possi- That is, the discrete design variables should be transformed
ble to search the design space widely. This is the reason into the continuous ones by introducing some approaches.
The optimization methods for the discrete design variables
S. Kitayama (B) K. Yamazaki are summarized by Arora et al. (1994).
Kanazawa University, Kakuma-machi, Kanazawa, 920-1192, Japan On the other hand, the penalty function approach for the
Tel.: +81-76-2344758
Fax: +81-76-2344668 discrete design variables has been proposed by Shin et al.
e-mail: kitagon@t.kanazawa-u.ac.jp (1990). The algorithm of their approach is described briefly
e-mail: yamazaki@t.kanazawa-u.ac.jp as follows:
M. Arakawa
Kagawa Univeristy, Hayashi-cho, Takamatsu, Kagawa, 761-0396, (STEP1) It is supposed that all design variables are the con-
Japan tinuous ones. In this stage the penalty function for
e-mail: arakawa@eng.kagawa-u.ac.jp the discrete design variables is not introduced.
192 S. Kitayama et al.
The following equation can be obtained from (1) and (2) after 2.4 Best position-keeping model
some replacements.
The g-best pkg in (2) represents the best particle in the swarm
at the k-th iteration, and then it may be considered that pkg
x k+1 = x kd + wv kd + q x kd
d (4) is updated at each iteration. As a result, there is no certainty
that the following relationship between pkg and pk+1 g may be
where and q are given by (5) and (6), respectively. established.
f pk+1 f pkg
g (9)
= c1r1 + c2 r2 (5) It is preferable to have the relationship represented by
(9) because of the descent property of swarm. Consequently,
the model keeping the best particle is also considered. This
c1r1 pkd + c2 r2 pkg model, called as the best position-keeping model, is expressed
q= (6)
c1 r 1 + c2 r 2 as follows:
v k+1 = wv kd + c1r1 pkd x kd + c2 r2 pg x kd
d (10)
The neighborhood of PSO is generated from (4), as shown in
Fig. 2, where
the magnitude of neighborhood is represented where pg represents the best position among pkd until now.
by q x kd . Because pg is selected among pkd , pg is not updated until the
Additionally, it is possible to interpret that q x kd repre- better pkd appears. As a result, it is expected that the (local)
sents the search direction when we imagine the similarity to search ability will be enhanced, and fast convergence will be
the gradient methods. in (5) may also be regarded as the expected.
stochastic step-size, in which its lower and upper bounds are Figure 3a shows the relationship of f pkd , f ( pg ) and
0 and c1 +c2 , and the mean value is (c1 + c2 )/2. From these f (q) when pg and pkd locate in the same convex space. From
relationships, it is possible to consider that the PSO has the Fig. 3a, the following relationship can be obtained:
search direction vector and stochastic step-size, even though
f (q) a f pkd + (1 a) f pg
the PSO does not utilize the gradient information of function. (11)
where a represents a real number in an interval [0,1]. Indeed, 3.2 Penalty function
(11) shows the condition for the convex space. Additionally,
the following relationship is established between pkd and pg . In this paper, the following penalty function suggested by
Shin et al. (1990) is adopted.
f pg f pkd
(12) n
" c #
X 1 2 xm+i 0.25 di, j+1 + 3di, j
(x) = sin +1
From (12), the following equation can be obtained: 2 di, j+1 di, j
i=1
f (q) f pg
(13) (19)
where di,j and di,j+1 represent the discrete design variables.
The above equation is always satisfied when pkd and pg locate c
xm+i is the continuous design variables between di,j and di,j+1 .
in the same convex space. Additionally, pkd represents the Therefore, the augmented objective function F(x) is con-
best position of particle d till the k-th iteration. As a result, structed by using the above penalty function as follows:
the following equation can be obtained.
ncon
X
f (q) f (x) (14) F(x) = f (x) + s(x) + r max [0, gk (x)] (20)
k=1
Hence, the search direction vector always turns to the direc- where s and r denote the penalty parameters of penalty terms
tion which improves the objective function when pkd and pg for the discrete design variables and the behavior constraints,
locate in the same convex space. respectively. Finally, the MDNLP is transformed into the fol-
On the other hand, there is no certainty that (11) is satisfied lowing problem of continuous design variables.
in the case of the situation shown in Fig. 3b. In this case,
the search direction vector turns to the direction which does F(x) min (21)
not improve the objective function. This means that some
particles can escape from a local minimum. xiL xi 6 xiU i = 1, 2, , m (22)
c
di,1 xm+i di,q i = 1, 2, , n (23)
For simplicity, the design variables are supposed as the
3 Penalty function approach for the discrete design discrete ones in the following discussion. The case of mixed
variables design variables will be discussed in Section 3.8 separately.
sd = 1 + (x d ) d = 1, 2, , agent (28)
where sd represents the penalty parameter of particle d. The
number agent in (28) is the total number of particles, and the
initial penalty parameter sinitial is determined as follows:
F pk f pk
The following equation is used to update the penalty para-
g
g
(27) meter s.
F pk
g
s = s exp 1 + pkg
(30)
In case of F pkg , the numerator in (27) is used as
The behavior of F(x) by updating the penalty parameter
the terminal criteria (that is, F pkg f pkg ).
s is shown schematically in Fig. 6, in which the solid line
As mentioned above, the PSO does not use the gradient shows F(x) at the k-th iteration, and the dotted line shows
information of function; therefore, it is difficult to satisfy (24) F(x) at the k+1-th iteration.
strictly. Therefore, (27) is used instead. Behaviors of F(x) for As shown in Fig. 6, F(x) at the k+1-th iteration becomes
the various penalty parameter s are shown in Fig. 5. It is found a highly nonconvex and continuous function in comparison
from this figure that to determine the penalty parameter s in with F(x) at the k-th iteration. For example, a point A in Fig. 6
advance is very difficult. corresponds to the point pkg at the k-th iteration. By updating
the penalty parameter s, pkg corresponds to the point A0 on the Fig. 7 Escape from a local minimum by relaxation of augmented
dotted line, which represents F(x) at the k+1-th iteration. It objective function
is apparent that F(x) at the k+1-th iteration becomes a highly
nonconvex function in comparison with F(x) at the k-th itera-
tion. As discussed in Section 2.3, the PSO has a similar struc- tinuous ones was found, the penalty function for the discrete
ture to the gradient methods. For this reason, it is expected design variables was introduced to avoid the search proce-
that pkg moves to the direction in Fig. 6. Finally, it is also dure of a global minimum amongst many local minima of
expected that pkg will satisfy (27). The proposed approach the augmented objective function. Additionally, Shins ap-
need not select an appropriate constant positive number to proach cannot overcome the case shown in Fig. 1b.
update the penalty parameter s in advance. By using (30), the On the other hand, the penalty parameter s is actively
penalty parameter s is determined automatically. introduced at the initial stage in the proposed approach.
Obviously, the augmented objective function becomes a
nonconvex and continuous function. However, this is not
3.6 Initialization of penalty parameter a serious problem because the PSO is applied to this non-
convex and continuous function. The proposed approach ob-
When (27) is satisfied, the discrete value resides around the viously overcomes the case shown in Fig. 1b because of its
neighborhood of pkg . Then, the initial penalty parameter ob- flexible and strict search ability. The new update scheme of
tained by (29) is utilized to find another discrete value because penalty parameter s is proposed by (30). In past works (Fu
F(x) becomes a highly nonconvex and continuous function et al. 1991; Shin et al. 1990), a constant positive number
by updating the penalty parameter s. It may be assumed that was used to update the penalty parameter. This means that
pkg fails to escape from a local minimum. In such occasions, the update scheme of the penalty parameter depends on the
F(x) is relaxed by using the initial penalty parameter when problem. On the other hand, the penalty parameter s may al-
(27) is satisfied. As a result, it may be expected that pkg can ways change in the proposed approach because the value of
penalty function pkg is utilized. It may be expected that the
escape from a local minimum. The concept of relaxation of
F(x) to escape from a local minimum is shown in Fig. 7. penalty parameter s is determined automatically, and flexible
applications are possible. Finally, the initialization of penalty
parameter s is also proposed to relax the augmented objective
3.7 Difference between the traditional and proposed function. As a result, it is expected that pkg can escape from
methods a local minimum.
With the binary PSO, it is also easy to handle the dis-
The penalty function for the discrete design variables adopted crete design variables (He et al. 2004; Kennedy and Eberhart
in this paper is basically the same as that of Shin et al. (1990). 1997); however, the search process of binary PSO is stochas-
However, the approaches are very different from each other. tic. Additionally, it is not sure that the objective function or
Shin et al. searched the optimum by handling all design vari- design space is not always continuous. On the other hand, the
ables as the continuous ones at the initial stage, where the characteristics of PSO and penalty function for the discrete
penalty parameter s in (20) has been set as zero. After the op- design variables have been clarified through Sections 2 and 3.
timum obtained by handling all design variables as the con- The approach adopted here utilizes the characteristics of PSO
Penalty function approach for the mixed discrete nonlinear problems by particle swarm optimization 197
4 Algorithm examine the search ability between the g-best and the best
position-keeping model. Then, typical benchmark problems
The basic algorithm of PSO for the MDNLP is described in for the MDNLP are treated.
the following:
(STEP1) Number of particles and maximum search itera- 5.1 Continuous design variables problem
tions k max are determined. Set iteration counter
k=1, initial position, and velocity of each particle Let us consider the following problem:
at random. m n
(STEP2) Calculate the value of (19) for each particle. X p o
(STEP3) Calculate the penalty parameter s for each particle f (x) = 418.9829 m + xi sin |xi | min (32)
i=1
by (28).
(STEP4) From STEP3, determine the initial penalty para- 500 x 500 (33)
meter sinitial by (29).
(STEP5) Calculate the augmented objective function. The global minimum of this benchmark problem is well
(STEP6) Calculate of the g-best and the p-best. known as follows:
(STEP7) Check the terminal criteria by (27). If (27) is satis- xi,G = 420.9687 i = 1, 2, , m (34)
fied, the penalty parameter s is initialized. Other-
wise, the penalty parameter s is updated by (30). The behavior of this function in two dimensions and the
(STEP8) Update the inertia term by (7). global optimum are shown in Fig. 9.
(STEP9) Update the position and velocity by (1) and (2). The number of particles is set as 10, and the maximum
(STEP10) Increase iteration counter as k=k+1. number of search iterations k max is set as 200. Examples of
(STEP11) Check the iteration counter. If kk max , go to typical histories of the g-best are shown in Fig. 10 for the
STEP5. Otherwise, the algorithm will terminate g-best model, and Fig. 11 for the best position-keeping model.
by taking pkg as optimum. From Fig. 10, the g-best of the g-best model moves at
each iteration because the g-best is selected among the best
The algorithm mentioned above is shown in Fig. 8. When particles at the k-th iteration. As considered that each particle
the best position-keeping model is used, (11) is used to update goes to the g-best, it is apparent that the g-best model may
the velocity. show slow convergence. On the other hand, the g-best of best
position-keeping model shows fast convergence because the
g-best is selected among the p-best. Additionally, the g-best of
5 Numerical examples the best position-keeping model can find some local minima
during the search process, as shown in Fig. 11, where points
In this section, the validity of the proposed approach is exam- a, b, c, and d represent local minima. The search process of
ined through numerical examples. At first, benchmark prob- g-best in the best position-keeping model is also shown in
lems of the continuous design variables are considered to Fig. 9.
198 S. Kitayama et al.
m
X
f (A) = Ai L i min (35)
i=1
Fig. 12 Ground structure of topology optimization
g j (A) = u k (A) u a (A) 1 0 j = 1, 2, , ncon
(36)
and Yamazaki (2005) [in the simple GA, the population size
is set as 100, the maximum number of search iterations is set
as 3,000, the crossover ratio is set as 0.6, and the mutation
ratio is set as 0.01. In the distributed GA, the population size
is set as 100, the maximum number of search iterations is
set as 10,000, the crossover and mutation ratios are the same
as those of simple GA, The number of islands in distributed
GA is four. In the SA, initial temperature is set as 1.0. TcT
(c=0.85) is utilized as the cooling schedule. Minimum tem-
perature is Tmin = 1.00 105 . The number of search itera-
tions for each temperature step is 1,000].
The final results are shown in Table 1. In the topology il-
lustration some members that have the cross-sectional areas
on the lower bound are omitted. Additionally, two behavior
constraints of vertical displacements at nodes 4 and 7 are
active at the global minimum. The result of PSO is consid-
ered that the active constraints can be practically obtained.
From Table 1, the PSO is superior to the GA in the case of
continuous design variables.
Fig. 13 The discrete and continuous optima Fig. 15 Optimum design problem of pressure vessel
Penalty function approach for the mixed discrete nonlinear problems by particle swarm optimization 201
Sandgren (1990) Qian et al. (1993) Kannan and Kramer (1994) Hsu et al. (1995) He et al. (2004) This research
R (inch) 47.00 58.31 58.29 N/A 42.10 42.37
L (inch) 117.70 44.52 43.69 N/A 176.64 173.42
Ts (inch) 1.125 1.125 1.125 N/A 0.8125 0.8125
Th (inch) 0.625 0.625 0.625 N/A 0.4375 0.4375
g1 (x) 0.19 0.00 0.00 N/A 0.00 0.00
g2 (x) 0.28 0.11 1.11 N/A 0.08 0.08
g3 (x) 0.51 0.81 0.82 N/A 0.26 0.28
g4 (x) 0.05 0.02 1.11 N/A 0.00 0.00
Objective ($) 8, 129.80 7, 238.83 7, 198.20 7,021.67 6, 059.71 6, 029.87
Trial no. R (inch) L (inch) Ts (inch) Th (inch) g1 (x) g2 (x) g3 (x) g4 (x) Objective
1 42.37 173.42 0.8125 0.4375 0.00 0.08 0.28 0.0 6, 029.87
2 45.79 135.72 0.875 0.4375 0.00 0.00 0.43 0.0 6, 039.84
3 45.78 135.81 0.875 0.4375 0.00 0.00 0.43 0.0 6, 040.78
4 45.73 136.27 0.875 0.4375 0.00 0.00 0.43 0.0 6, 046.33
5 45.70 136.27 0.875 0.4375 0.00 0.00 0.43 0.0 6, 049.72
6 45.70 136.59 0.875 0.4375 0.00 0.00 0.43 0.0 6, 049.97
7 45.41 136.62 0.875 0.4375 0.00 0.00 0.42 0.0 6, 081.93
8 45.31 139.48 0.875 0.4375 0.00 0.01 0.41 0.0 6, 094.76
9 45.30 140.59 0.875 0.4375 0.00 0.01 0.41 1.0 6, 094.30
10 45.10 142.68 0.875 0.4375 0.00 0.02 0.41 0.0 6, 117.81
202 S. Kitayama et al.
References