PSO Mini Tutorial

Particle Swarm optimisation
2002-04-24 Maurice.Clerc@WriteMe.com
Particle Swarm optimisation: A mini
tutorial
The inventors (1)
Russell
Eberhart
eberhart@engr.iupui.edu
The inventors (2)
James
Kennedy
Kennedy_Jim@bls.gov
Part 1: United we stand
Cooperation example
Initialization. Positions and velocities
Neighbourhoods
geographical
social
The circular neighbourhood
Virtual circle
1
5
7
6
4
3
8
2
Particle 1s 3-
neighbourhood
Psychosocial compromise
Here I am! The best perf. of
my neighbours
My best
perf.
x
p
g

p
i

v
The historical algorithm
( )
( )
( ) ( ) ( )
( ) ( ) ( )
+
+
= +
t x p rand
t x p rand
t v
t v
d d g
d d i
d
d
, 2
, 1
, 0
, 0
1
|
|
o
for each particle
update
the
velocity
( ) ( ) 1 ) 1 ( + + = + t v t x t x then move
for each component d
At each time step t
Random proximity
x
p
g

p
i

v
Hyperparallelepiped => Biased
Animated illustration
Global
optimum
Part 2: How to choose parameters
Type 1 form
2 1 2 1
' ' ) , 0 ( ) , 0 ( + = + = rand rand
2 1
2 1
' '
' '

+
+
=
g i
p p
p
+ + = +
+ = +
) ( ) 1 ( ) 1 (
))) ( ( ) ( ( ) 1 (
t x t v t x
t x p t v t v _
with
>
+ =

4 for
4 2
2
2
k

k
_
else
Usual values:
k=1
=4.1
=> _=0.73
swarm size=20
hood size=3
5D complex space
4
2
0
-2
-4
2
1
0
-1
-2
6
5
4
3
2
1
0
}
}
Convergence
Non
divergence
A 3D section

Re(y)
Re(v)
Move in a 2D section (attractor)
1
0.5 0 -0.5 -1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
Im(v)
Re(v)
Some functions ...
Rosenbrock
Griewank
Rastrigin
... and some results
30D function PSO Type 1" Evolutionary
algo.(Angeline 98)
Griewank [300] 0.003944 0.4033
Rastrigin [5] 82.95618 46.4689
Rosenbrock [10] 50.193877 1610.359
Optimum=0, dimension=30
Best result after 40 000 evaluations
Beat the swarm!
Part 3: Beyond real numbers
0
1
2
3
4
0
1
2
3
0
1
2
3
4
1
2
3
4
5
6 0
1
2
0
1
2
3
4
8
8
Bingo!
Minimun requirements
( ) ( ) ( ) ( ) ( ) ( ) ' ' , ) ' , ( x f x f x f x f H H x x > v < e
( )
( )
( )
( ) position velocity position
velocity position position
velocity velocity velocity
velocity velocity t coefficien

,
,
,
,
Comparing positions in the

search space H
Algebraic operators
velocity = pos_minus_pos(position
1
, position
2
)
velocity = linear_combin(o,velocity
1
,|,velocity
2
)
position = pos_plus_vel(position, velocity)
(position,velocity) = confinement(position
t+1
,position
t
)
Pseudo code form
}
algebraic
operators
=>
Fifty-fifty

x
i
e 1...N
{ }
i = j x
i
= x
j
x
i
= x
i
D/ 2+1
D
1
D / 2

N=100, D=20. Search space: [1,N]
D

105 evaluations:
63+90+16+54+71+20+23+60+38+15
=
12+48+13+51+36+42+86+26+57+79 (=450)
Knapsack

x
i
e 1...N
{ }
i = j x
i
= x
j
x
i
= S
ieI , I = D, I c 1, N
{ }

N=100, D=10, S=100,
870 evaluations:
run 1 => (9, 14, 18, 1, 16, 5, 6, 2, 12, 17)
run 2 => (29, 3, 16, 4, 1, 2, 6, 8, 26, 5)
Graph Coloring Problem
1
4 5
2
1
1
5
5
3
2
2
1
1
5
5
5
0
2
0
-
1
4
-3
-1
-1
+
=
The Tireless Traveller
Example of position: X=(5,3,4,1,2,6)
Example of velocity: v=((5,3),(2,5),(3,1))
n
1

n
3

n
2

Apple trees
( ) ( )
2
3 2
2
2 1
n n n n f + =
Evaluation n1 n2 n3
0
1
2
3
Swarm size=3
Best position
7 7 6
3 11 6
6 4 10
3 0 17
Part 4: Some variants
0
0.2
0.4
0.6
0.8
1
1.2
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
Unbiased random proximity
x
p
g

p
i

v
Hyperparallelepiped => Biased
Dimension
V
o
l
u
m
e

Hypersphere vs hypercube
Clusters and queens
Each particle is weighted by its
perf.
Dynamic clustering
Centroids = queens =
temporary new
particles
Think locally, act locally (Adaptive
versions)
Adaptive swarm size
There has been enough
improvement
but there has been not enough
improvement
although I'm the worst
I'm the best
I try to kill myself
I try to generate a
new particle
Adaptive coefficients
The better I
am, the more I
follow my own
way
The better is my best
neighbour, the more
I tend to go towards
him
ov
rand(0b)(p-x)
Energies: classical process
Rosenbrock 2D. Swarm size=20, constant coefficients
Energies: adaptive process
Rosenbrock 2D. Adaptive swarm size, adaptive coefficients
Part 5: Real applications (hybrid)
Medical diagnosis
Industrial mixer
Electrical generator Electrical vehicle
Real applications (stand alone)
Cockshott A. R., Hartman B. E., "Improving the fermentation medium for
Echinocandin B production. Part II: Particle swarm optimization", Process
biochemistry, vol. 36, 2001, p. 661-669.

He Z., Wei C., Yang L., Gao X., Yao S., Eberhart R. C., Shi Y., "Extracting
Rules from Fuzzy Neural Network by Particle Swarm Optimization", IEEE
International Conference on Evolutionary Computation, Anchorage, Alaska,
USA, 1998.

Secrest B. R., Traveling Salesman Problem for Surveillance Mission using
Particle Swarm Optimization, AFIT/GCE/ENG/01M-03, Air Force Institute of
Technology, 2001.

Yoshida H., Kawata K., Fukuyama Y., "A Particle Swarm Optimization for
Reactive Power and Voltage Control considering Voltage Security
Assessment", IEEE Trans. on Power Systems, vol. 15, 2001, p. 1232-1239.
To know more
Clerc M., Kennedy J., "The Particle Swarm-Explosion,
Stability, and Convergence in a Multidimensional
Somplex space", IEEE Transaction on Evolutionary
Computation, 2002,vol. 6, p. 58-73.
Clerc M., "L'optimisation par essaim particulaire.
Principes et pratique", Herms, Techniques et
Science de l'Informatique, 2002.
Particle Swarm Central, http://www.particleswarm.net
THE site:
Self advert
Appendix
Canonical form
( ) ( ) ( ) ( )
( ) ( ) ( )
+ + = +
+ = +
1 1
1
t v t x t x
t x p t v t v
( ) ( ) t x p t y =
v t +1 ( )= ov t ( )+|y t ( )
y t +1 ( ) = v t ( )+ o q ( )y t ( )

o
|
oq
( )
( )
( )
( )
(

=
(
+
+
t y
t v
t y
t v
1 1
1
1
1
M
Eigen values e
1
and e
2

Constriction
( ) ( ) ( )
( ) ( ) ( )
+
+ + +
=
+
+ + + +
=

o o | oq oq q q o o
_

o o | oq oq q q o o
_
4 2
2 2
4 2
2 2
2
2 2
2
2
2 2
1
Constriction coefficients
Convergence criterion

6 5 4 3 2 1
3.5
3
2.5
2
1.5
1
0.5
_
1
e
1
<1
_
2
e
2
<1

:
_
1
<1
_
2
e
2
<1

k = _
2
e
2
Magic Square (1)
(
(
(
D D D
j i
D
m m
m
m m
, 1 ,
,
, 1
1 , 1
( )
{ }
=
e
=
+
l k j i
j i
D i j
j i
m m
N m
x m
, ,
,
1
,
1
( )
( )
0
1
1
2
1
1 , ,
1
1
2
1
, 1 ,
=
|
|
.
|
\
|
+
|
|
.
|
\
|
= =
+
= =
+
D
j
D
i
j i j i
D
i
D
j
j i j i
m m
m m
Magic Square (2)
D=3x3, N=100
10 runs
13430 evaluations
10 solutions
55 30 68
42 49 62
56 74 23
30 61 53
89 32 23
25 51 68
27 96 39
73 40 49
62 26 74
22 70 58
40 75 35
88 5 57
50 43 67
58 55 47
52 62 4
43 51 78
75 33 64
54 88 30
50 65 68
69 42 72
64 76 43
18 25 59
32 53 17
52 24 26
80 3 30
22 72 19
11 38 64
65 28 64
63 55 39
29 74 54
Non linear system
( )
=
= +
0 10 sin
0 1
2 1
2
2
2
1
x x
x x
Search space
[0,1]
2

1 run
143 evaluations
1 solution
10 runs
1430 evaluations
3 solutions

PSO Mini Tutorial

Încărcat de

Informații document

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

PSO Mini Tutorial

Încărcat de

Drepturi de autor:

Formate disponibile

Particle Swarm optimisation

Comparing positions in the

S-ar putea să vă placă și