Engineering Optimization DMS-6036

Title : Simulated Annealing

 Outline

Introduction
Background: annealing
Simulated annealing: the physical analogy
Simulated annealing algorithm
Simulated annealing examples
Advantages and disadvantages
Summary

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 Learning Objectives

Review background Annealing

Understand the basic assumptions and steps in Simulated
Annealing (SA)
Be able to transform design problems into a
combinatorial optimization problem suitable to SA
Understand strengths and weaknesses of SA

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 Introduction
Optimization problem consists of maximizing or minimizing
a real function by systematically choosing input values from
within an allowed set
greedy approach ,
Local steepest descend, etc

Optimization
search techniques
SIMMULATED ANNEALING
Global

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

Cont.

Difficulty in Searching Global Optima

starting
point
descend
direction
local minima

global minima

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 Background : Annealing

Annealing is a thermal process for obtaining low energy

states of a solid in a heat bath.
Solid is heated to melting point
High-energy, high-entropy state
Removes defects/irregularities
Temp is very slowly reduced
Low energy, organized crystal lattice formed
(regular structure).
Simulated annealing establishes the connection between
this type of thermodynamic behaviour and the search for
global minima for a discrete optimization problem.

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 Simulated Annealing (SA)

Simulated annealing (SA) is a probabilistic technique

for approximating the global optimum of a
given function.
For problems where finding a global optimum is more
important than finding a local optimum in a fixed amount
of time, simulated annealing may be preferable to
alternatives such as gradient descent.

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 Cont.

The Boltzmanns probability distribution implies that the

energy (E) of a system in thermal equilibrium at
temperature T is distributed probabilistically according
to the relation
P(E) = eE/kT
where P(E) denotes the probability of achieving the energy
level E, and k is called the Boltzmanns constant.
The method of implementing the Boltzmanns probability
distribution in simulated thermodynamic systems,
suggested by Metropolis, can also be used in the
context of minimization of functions.

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 Relationship between Physical
Annealing and Simulated Annealing
Thermodynamic Simulation Combinatorial Optimization

Metal Problem

Energy Cost

Change of State Neighbouring Solutions

Temperature Control Parameter T

Frozen State Heuristic Solution

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

Flow Chart: Start With an initial solution

Add new random stand at random

period

P(delta)>rand? Improvement?

yes
no yes
Accept new Solution

Stop criteria?

Stop

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 Simulated Annealing Algorithm
construct initial solution x0; xnow = x0
set initial temperature T = T1
repeat for i = 1 to TL do
generate randomly a neighboring solution x N(xnow)
compute change of cost C = C(x) - C(xnow)
if C 0 then
xnow = x (accept new state) , else
Generate q = random(0,1)
if q < exp(-C /T) then xnow = x end if
end for
set new temperature T = f(T) , until stopping criterion
return solution corresponding to the minimum cost function
presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017
 Stopping Criterion

A given minimum value of the temperature has been

reached.

A certain number of iterations (or temperatures) has

passed without acceptance of a new solution.

A specified number of total iterations has been executed

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 Simulated annealing examples

1.Travelling Salesman
A salesman has to travel to a number of cities and
then return to the initial city; each city has to be
visited once. The objective is to find the tour with
minimum distance and time.

If we have computer that can compute one

million tours per second, we can get the
following results for different number of cities.

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

Cont...

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 Advantages:
can deal with arbitrary systems and cost functions
statistically guarantees finding an optimal solution
is relatively easy to code, even for complex problems.
generally gives a ``good'' solution

main advantages over other local search methods are its

flexibility and its ability to approach global optimality.

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 a lot of choices are required to turn it into an actual
algorithm
the precision of the numbers used in implementation can
have a significant effect upon the quality of the outcome
The method cannot tell whether it has found an optimal
solution. Some other complimentary method (e.g. branch
and bound) is required to do this.

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 CONCLUSION

SA is based on neighborhood search and allows uphill

moves.
It has a strong analogy to the simulation of cooling of
material.
Uphill moves are allowed with a temperature dependent
probability.
Generic and problem-specific decisions have to be taken
at implementation.
Experimental tuning is very important!

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017

 Questions

Thank You

presented by: Goftila G. Engineering Optimization - DMS6036 August 28, 2017