Pheromone models in ant colony optimization (ACO)

E. Foundas

A. Vlachos

Department of Informatics
University of Piraeus
Piraeus 185 34
Greece
Abstract
Ant Colony Optimization is a constructive meta-heuristic that uses an analogue of ant
trail pheromones to learn about good features of solutions. In this paper, using the difference
equations as a tool of research, we propose the mathematical model of the distribution of
pheromone at the classic double bridge experiment, explain the mathematical model of the
pheromone function in the arcs of a connected graph and specify the mathematical models
of pheromone update in Ant Colony System (ACS) and Max-Min Ant System (MMAS)
algorithms.
Keywords and phrases : Ant colony optimization, pheromone, difference equations, stochastic
model, ant colony system (ACS), max-min ant system (MMAS).
1. Introduction
A category of problems that appear in nature are the Hard Combina-
torial Optimization Problems and their solution, using classic determinis-
tic methods, is difcult, if not impossible. These difculties were exceeded
by the evolution of the stochastic heuristic algorithms and especially
by those algorithms which are inspired by natural, biological or social
phenomena [1]. The basic property of these algorithms is that they are

E-mail: efountas@unipi.gr

E-mail: AVlachos@unipi.gr

Journal of Interdisciplinary Mathematics

Vol. 9 (2006), No. 1, pp. 157168
c Taru Publications
158 E. FOUNDAS AND A. VLACHO
not constrained to local optimal solutions, whereas there is the ability of
localization of the area which provides the total optimal solution through
a large number of estimations of the objective function which describes the
problem.
There is a strong argument which determines the gravity of the
heuristic algorithms: in action, a model of the actual problem is the one
which is optimized. The heuristic algorithms are usually exible. They
can encounter, in an efcient way, complicated objective functions and
their constraints comparing to the classic algorithms.
There are two competitive mechanisms in the heuristic algorithms:
exploration, which regards to the total possible space of solutions for
the effective search of new solution areas, and exploitation of the area of
solutions in which local search, for optimal solutions in spaces that have
already given good results, takes place. A right heuristic algorithm should
run at the convergence of the best results of the preceding competitive
mechanisms.
One of the most well known heuristic methods, inspired by the
nature, is the Ant Colony Optimization [2]. Dorigo and his partners, based
on Deneubourgs work on social insects and especially on ants, create the
Ant System algorithm (AS) [4]. The key of success of this algorithm is the
indirect communication among the ants through the chemical substance
pheromone which is deposited on earth while ants forage and collect
food. The ants of a colony sense the pheromone and decide, based on a
probabilistic law, to pheromone this pheromone trail.
The paper is organized as follows: a stochastic model, concerning
the pheromone distribution in the double bridge experiment, is presented
in section 2, section 3 explains a mathematical model of the pheromone
objective function, F(
i j
), in the arcs of a connected graph and section 4
gives the conclusions of this work.
2. A stochastic model through difference equations
Deneubourg and his partners, [5], proposed a stochastic model,
which describes the dynamics of the ant society as it is observed at
the double bridge experiment. Figure 1 shows the experimental setup
of a bridge that has branches with different length. Supposing that
ants cover per second the bridge, to every direction with stable velocity
PHEROMONE MODELS 159
cm/s, and deposit a unit of pheromone in each branch. In this paper the
phenomenon of the evaporation of the pheromone is not considered.
Nest Source food
s

1
i
2
j
Figure 1
Experimental setup of a bridge with two branches with different
length. Ants move from their nest to the food source and back-
wards
If
s
and

are the lengths, in cm, if the shorter (s) and longer ()

branch of the bridge respectively, an ant will choose to cover the shorter
branch (s) in a time of t
s
=

s

seconds. The demanded time for an ant to

cover the longer branch () is t
1
= r t
s
seconds, with r =

s
.
Proposition 2.1. If the demanded time, t
1
, for an ant to cover the longer branch,

, with stable velocity cm/s, is t

1
=

, then t
1
= r t
s
Proof. Indeed, it is t
1
=

, and then t
1
=

, that is t
1
= r t
s
, where
r =

s
and r > 1.
The probability P
ia
(t) for an ant to reach the decision point i {1, 2}
and choose the branch a {s, } is [6]:
P
ia
=
(t
s
+
is
(t))
a
(t
s
+
is
(t))
a
+ (t
s
+
i
(t))
a
(2.1)
160 E. FOUNDAS AND A. VLACHO
where
ia
(t) is the total amount of pheromone on the bridges branch,
which is proportional to the number of ants that use the branch in time
t . Experimentally, it comes up that a = 2. It has been proofed that
P
is
(t) + P
i
(t) = 1. The dynamic system:
d
is
dt
= P
is
(t t
s
) +P
is
(t), i, j {1, 2} , i = j (2.2)
d
i
dt
= P
i
(t rt
s
) +P
i
(t), i, j {1, 2} , i = j (2.3)
results in the following proposition.
Proposition 2.2. The amount of pheromone
is
(t) (respectively
i
(t)) which
is deposited on the s branches (respectively ) is given by the solution of the
following difference equation system:

is
(t + 1)
is
(t) = [P
js
(t t
s
) + P
is
(t)] (2.4)

i
(t + 1)
i
(t) = [P
j
(t t

) + P
i
(t)] . (2.5)
Proof. The difference equation (2.4) (respectively (2.5)) is of rst degree,
non homogenous. We set
is
(t 1) y
t+1
,
is
(t) y
t
and [P
js
(t t
s
)
+P
is
(t)] g
t
, and so the equation (2.4) becomes:
y
t+1
y
t
= g
t
. (2.6)
The respective homogenous difference equation is:
y
t+1
y
t
= 0 . (2.7)
which is successionally written as:
y
2
= y
1
y
3
= y
2
(2.8)
.
.
.
.
.
.
.
.
.
y
t
= y
t1
.
Multiplying by members the equations (2.8), we have that:
y
t
= t
1
k = constant . (2.9)
The difference equation (2.6) is written as:
1
(y
t
) = g
t
therefore y
t
=

1
(g
t
),
PHEROMONE MODELS 161
and
y
t
=
t1

=1
g

. (2.10)
The general solution is:
y
t
= k +
t1

=1
g

or
y
t
= k +
t1

=1
[P
js
(t t
s
) + P
is
(t)]
or

is
(t) = k +
t1

=1
[P
js
(t t
s
) + P
is
(t)] . (2.11)
Similarly the solution of the equation (2.5) is:

i
(t) = k +
t1

=1
[P
j
(t t

) + P
i
(t)] . (2.12)
The equation (2.11) expresses the alteration if the pheromone, in time
t , of the branch s at the decision point i . It is given by the constant number
of the ants, , multiplied with the probability of ants to choose the shorter
branch at the decision point j in time t t
s
and with the probability of ants
to choose the shorter branch at the decision point i in time t . Finally the
constant k is added. The constant t
s
, is necessary demanded time for the
ants to cover the shorter branch. The equation (2.12) shows the alteration
of pheromone on the longer branch, when the demanded time, for the ants
to cover it, is t t

.
3. Origin of the F(
i j
) function
Consider the minimization of a problem (S, f , ), where S is the set
of feasible solutions, f is the objective function and supposing f (s) is an
objective function of cost, for every feasible solution s S, and is the
set of constraints. The objective of the problems minimization is to nd
an optimal solution S which will be a feasible solution of minimum cost.
The combinatorial optimization problem (S, f , ) has the following
characteristics:
162 E. FOUNDAS AND A. VLACHO
A nite set of elements, C = {c
1
, c
2
, . . . , c
n
}, is given which repre-
sents the components of the problem.
A nite set of X conditions of the problem is given which is dened
by the terms of the sequence, s = {c
i
, c
j
, . . . , c
n
}, elements of set C.
The length of the sequence s, which is the number of its elements, is
declared as |s| .
A set of solutions S is a subset of the X conditions (S X).
A set of feasible conditions

X, with

X X.
A non empty set, S

, of optimal solutions, with S

X and S

S.
A function of cost, f (s), that corresponds to every feasible solution
s S.
When a problemhas been dened like previously, articial ants create
feasible solutions by following possible routes paths on a full connected
graph, G (C, L, T), where the nodes are the components C and the set T
is the vector of the pheromone trails. The probability for an ant to choose
the node j is:
P(c
h+1
= j | X
h
) =
_

_
F
i j
(
i j
)

(i, j)N
k
i
F
il
(
il
)
, if (i, l) N
k
i
0 , otherwise
(3.1)
where (i, j) N
k
i
only if the sequence X
h+1
= {c
1
, c
2
, . . . , c
h
, j} is
created by the ant k, satisfying the constraints, and only if f
i j
(
i j
) is an
ascending monotonous function that represents the pheromone function
in every arc [6]. If we set F
i j
(
i j
) f
i j
(
i j
), then we have the following
proposition.
Proposition 3.1. The function f (
i j
) is given by the determinant of the matrix:
_

_
a
n
a
n1
a
n2
. . . a
1
a
0
1
i j
0 . . . 0 0
0 1
i j
. . . 0 0
.
.
.
.
.
. 1 . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. . . .
.
.
.
.
.
.
0 0 0 . . . 1
i j
_

_
(3.2)
with a
n
0, n = 0, 1, . . . , n.
PHEROMONE MODELS 163
Proof. Indeed, the determinant of the previous matrix is:
We set:
f
n
(
i j
)
_

_
a
n
a
n1
a
n2
. . . a
1
a
0
1
i j
0 . . . 0 0
0 1
i j
. . . 0 0
.
.
.
.
.
. 1 . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. . . .
.
.
.
.
.
.
0 0 0 . . . 1
i j
_

_
. (3.3)
Expanding the determinant, for the elements of the ath column we
receive:
f
n
(
i j
) = a
n

n
i j
+ f
n1
(
i j
) (3.4)
with f
0
(
i j
) = a
0
and n = 1, 2, . . . , n.
Summing sequential, by members, the f
n
(x), it emerges nally that:
f
n
(
i j
) = a
n

n
i j
+ a
n1

n1
i j
+ . . . + a
1

i j
+ a
0
. (3.5)
The recursive relation (3.4) is a rst degree difference equation with
constant factors, of the following form:
f
n
(
i j
) f
n1
(
i j
) = a
n

n
i j
and its solution is the following polynomial:
f
n
(
i j
) = a
n

n
i j
+ a
n1

n1
i j
+ . . . + a
1

i j
+ a
0
, a
n
0 .
Also, it is proved easily that, the polynomial (3.5) is a solution for the
(n + 1)th order differential equation:
d
(n+1)
f
n
dx
(n+1)
= 0 .
4. Mathematical models of pheromone updating in ant colony system
(ACS) and max-min ant system (MMAS) algorithms
ACO is an umbrella term for a number of ant algorithms: Ant System
(AS) [2], Ant Colony System, [7] and Max-Min Ant System [8].
In the Ant Colony Optimization (ACO) approach a discrete combi-
natorial optimization problem is represented by a so called construction
graph where feasible solutions to the problem correspond to paths of
164 E. FOUNDAS AND A. VLACHO
the graph. Articial ants move on the graph to generate feasible solutions.
Each move from one node to the next is based on a probabilistic decision
called random transition rule. The transition probability of each ant is
governed by two parameters assigned to the edges of the graph:
(1) pheromone values representing the common memory of the ants,
and
(2) visibility values representing the heuristic desirability of choosing
node j , when in node i of the graph.
Pheromones are updated by a combination of two mechanisms:
evaporation and reinforcement of pheromone on paths recognized as
good, [9].
In ACS the local update of pheromone trail is performed as follows,
[7]: When, while performing a tour, ant k is in node i and selects node
j J
k
i
the pheromone concentration of (i, j) is updated by the following
formula:

i j
(t + 1) = (1 )
i j
(t) +
0
(4.1)
with initial value
i j
(0) = 0, where:

0
is the initial value of pheromone trails,
is the pheromone evaporating parameter, (0, 1),
j
k
i
is the set of nodes that remain to be visited by ant k positioned on
node i (to make the solution feasible).
It has been experimentally found that setting
0
= (n L
mn
)
1
, where n is
the number of nodes and L
mn
is the length of a tour produced by nearest
neighbor heuristic, produces good results.
Equation (4.1) is a difference equation of the form:
y
t+1
(1 ) y
t
=
0
(4.1.1)
where
i j
(t) y
t
.
(1) The general solution of the homogeneous equation:
y
t+1
(1 )y
t
= 0
with characteristic equation
(1 ) = 0
PHEROMONE MODELS 165
is:
y
t
= (1 )
t
.
(2) The particular solution is:

t
=

0
1 + (1 +)
=
0
.
The general solution of the compete equation is: y
t
= c(1 )
t
+
0
,
i.e.,

i j
(t) = c(1 )
t
+
0
.
Since
i j
(0) = 0, we have: 0 = c +
0
and hence
c =
0
.
The general solution is given by:

i j
(t) =
0
[1 (1 )
t
]
with 0 < < 1.
The Max-Min Ant System, [8], which is basically similar to the Ant
System (AS), with the following differences:
(i) only one ant (the best in the current iteration) is allowed to update
pheromone trails,
(ii) pheromone trail values are restricted to an interval [
min
,
max
] ,
and
(iii) trails are initialized to their maximum value
max
.
In Max-Min Ant Systems the pheromone trail update rule is given by:

i j
(t + 1) = (1 )
i j
(t) +
i j
(t) (4.2)
where

i j
(t) =
_
1/L

(t) , if (i, j) T

(t)
0 , if (i, j) / T

(t)
(4.3)
where L

(t) is the length of either the global-best tour (L

best
(t)), or the
iteration-best solution (L
iter
(t)) and T

(t) is the corresponding order of

the nodes.
166 E. FOUNDAS AND A. VLACHO
The equation (4.2) is a linear non homogeneous (or complete) differ-
ence equation:

i j
(t + 1) (1 )
i j
(t) =
i j
(t) .
We will prove that the maximum value of a pheromone trail is:

max
=
1
L
opt
(4.4)
where L
opt
is the length of the optimal solution for a specic problem.
In Max-Min Ant System, we set the maximum pheromone trail
max
to an estimate of the asymptotically maximum value. This is achieved
by using L
best
(t) instead of L
opt
in equation (4.4); every time that a new
best solution is found,
max
is updated, leading actually to a dynamically
changing value of
max
(t).
(1) The general solution of the homogeneous:

i j
(t + 1) (1 )
i j
(t) = 0
is

i j
(t) = c(1 )
t
.
(2) Particular solution:
Since 1 + (1 +) = > 0, we have:
The particular solution T
i j
is:
T
i j
(t) =
1
L
opt

=
1
L
opt
.
Thus the general solution of the compete equation is given by:

i j
(t) = c(1 )
t
+
1
L
opt
. (4.5)
For t = 0 the equation (4.5) gives:
i j
(0) = c +
1
L
opt
, i.e.
c =
i j
(0)
1
L
opt
.
Equation (4.5) may be rewritten in the form:

i j
(t) =
_

i j
(0)
1
L
opt
_
(1 )
t
+
1
L
opt
PHEROMONE MODELS 167
since 0 < < 1, lim
t
(1 )
t
= 0 and

max
= lim
t

i j
(t) =
1
L
opt
. (4.6)
If = 0, the corresponding difference equation is:

i j
(t + 1)
i j
(t) =
1
L
opt
.
The corresponded homogeneous is

i j
(t + 1)
i j
(t) = 0
with characteristic 1 = 0, thus:

i j
(t) = c constant .
A particular solution T
i j
(t) is
T
i j
(t) =
1
L
opt
t
1
=
1
L
opt
t .
Finally, the general solution is:

i j
(t) = c +
1
L
opt
t . (4.7)
The equation (4.7) is a linear model according to time t .
References
[1] D. Corne, M. Dorigo and F. Elover, New Ideas in Optimization,
McGraw-Hill, 1999.
[2] M. Dorigo, V. Maniezzo and A. Colorni, The ant system: optimiza-
tion by a colony of cooperating agents, IEEE Trans. Systems Man
Cybernet., Vol. B 26 (1) (1966), pp. 2941.
[3] J. L. Deneubourg, S. Aron, S. Goss and J. M. Pasteels, The self-
organizing exploratory pattern of the argentine ant, Journal of Insect
Behavior, Vol. 3 (2) (1990), pp. 159169.
[4] M. Dorigo, V. Maniezzo and A. Colorni, Ant system: an autocat-
alytic optimizing process, Technical Report 91-016, Dip. di Elettronica,
Politechnico di Milano, Italy, 1991.
[5] S. Goss, J. L. Deneubourg and J. N. Pasteels, Self-organized shortcuts
in the Argentine ant, Naturwissenchaften, Vol. 76 (1989), pp. 579581.
168 E. FOUNDAS AND A. VLACHO
[6] M. Dorigo and T. St utzle, Ant Colony Optimization, MIT Press, 2004.
[7] J. E. Bonabeau, M. Dorigo and E. Theraulaz, Swarmintelligence from
natural to articial systems, Santa Fe Institute Studies in the Sciences of
Complexity, Oxford University Press, 1999.
[8] T. St utzle and H. H. Hoos, Max-min ant system, Future Generation
Computer Systems, Vol. 16 (8) (2000), pp. 889914.
[9] W. Gutjahr, ACO algorithms with guaranteed convergence to the op-
timal solution, Information Processing Letters, Vol. 82 (2002), pp. 145
153.
Received May, 2005 ; Revised September, 2005