THEORETICAL AND PRACTICAL

CONCEPTS OF GA
Presented By
Dr. Bhupendra Verma
Director,
Technocrats Institute of Technology( Excellence), Bhopal

ARTIFICIAL INTELLIGENCE
AI is the branch of computer science that is concerned with
the automation of Intelligent behavior.
Intelligence is not very well defined and therefore has been
less understood.
Task associated with intelligence such as learning, intuition
creativity and interference all seem to have been partially
understood.
AI in its quest to design intelligent systems has fanned out
to encompass a number of technologies in its fold.
Of these technologies NN,FL, GA are predominantly known
as Soft Computing

What is Soft Computing

Soft computing is a consortium of methodologies that works

synergistically and provides, in one form or another, flexible
information processing capability for handling real-life ambiguous
situations.

Its aim is to exploit the tolerance for imprecision, uncertainty,

approximate reasoning, and partial truth in order to achieve
tractability(Polynomial complexity), robustness, and low-cost
solutions.

The guiding principle is to devise methods of computation that lead

to an acceptable solution at low cost by seeking for an
approximate solution to an imprecisely/precisely formulated
problem.

Soft Computing Technologies

- Fuzzy Sets provide a natural framework for the
process in dealing with uncertainty.
- Neural Networks are widely used for classification
and rule generation.
- Genetic Algorithms (GAs) are involved in various
optimization and search processes.

GENETIC ALGORITHMS
GA are
a family of computational models inspired by genetic
evolution. The basic idea is that each individual of an evolving
population encodes a candidate solution (e.g. a prediction rule) to a
given problem (e.g. classification).
GAs are adaptive, robust, efficient, and global search methods,
suitable in situations where the search space is large. They optimize
a fitness function, corresponding to the preference criterion , to
arrive at an optimal solution using certain genetic operators.

GA has been successfully applied to problems that are difficult to

solve using conventional techniques such as scheduling problems,
traveling salesperson problem, network routing problems and
financial marketing etc.

SIMPLE GENETIC ALGORITHM

Step 1: Initialize a population P of n elements each

a potential solution.

Step 2: Until a specified termination condition

is
satisfied:
2a: Use a fitness function to evaluate each element
of the
current population. If an element passes
the fitness criteria,
it remains in P.
2b: The population now contains m elements. Use
genetic
operators to create new elements. Add
the new elements
to the population.

as

ADVANTAGES OF GA

Easy to understand
General purpose, robust search technique
We always get an answer and it gets better with time
Inherently parallel and easily distributed
Supports multi objective optimization
Good for noisy environment
Modular, separate from application
Simple, Powerful, Adaptive, Parallel
Guarantee near optimum solutions.
Give solutions of un-approximated form of problem.
Finer granularity of search spaces.

COMPONENTS OF A GA

A problem to solve, and ...

Encoding technique
(gene, chromosome)
Initialization procedure
(creation)
Evaluation function
(environment)
Selection of parents
(reproduction)
Genetic operators (mutation, recombination)
Parameter settings
(practice and art)

Representation of individuals
String of parameters (genes) :
chromosome
eg. F(p,q,r,s,t): p q r s t
We can use bit string, arrays, trees, lists
or any other objects
Chromosome represent a candidate
solution
Bit-string representation (Ex):
100110101101100

 Standard GAs use binary strings of fixed

length.
Can be easily modified for any finite

alphabets.
For example, can use 10 symbols {0,
1, . . . , 9}.
Example solution: 41662971.
Can use letters and numbers.
Example solution: THEANSWERIS42

Other Types of Encodings of

Parameters
Octal

Encoding
Hexadecimal Encoding
Permutation Encoding
( Ex Travelling Salesman Problem)
Value

Encoding

(This encoding Requires to modify the

Genetic operators Ex finding weights of NN)
Tree

Encoding

( Mainly Used for genetic programming

( Lisp) )

EXAMPLE BINARY-CODED REPRESENTATIONS

For Example, lets say that we are trying to optimize

the following function,

f(x) = x2
for 2 x 1

If we were to use binary-coded representations we

would first need to develop a mapping function form
our genotype representation (binary string) to our
phenotype representation (our CS). This can be done
using the following mapping function:

d(ub,lb,l,chrom) = (ub-lb) decode(chrom)/2l-1 + lb

GENETIC ALGORITHMS:
BINARY-CODED REPRESENTATIONS

d(ub,lb,l,c) = (ub-lb) decode(c)/2l-1 + lb , where

ub = 2,
lb = 1,
l
= the length of the chromosome in bits
c
= the chromosome

The parameter, l, determines the accuracy (and

resolution of our search).
What happens when l is increased (or decreased)?

TWO APPROACHES TO OTHER

REPRESENTATIONS

Map to binary representation

Map the other representation to binary strings of

fixed-length and use standard binary-string GAs.
Modify operators (crossover and mutation)

Modify operators to make them work for the

other representation. Can be done by designing
alternatives for standard crossover and mutation

Fitness Function

Survival of the fittest (Fitness function)

numerical figure of merit/utility measure of
an individual
Tradeoff amongst a multiple evaluation criteria
Efficient evaluation
Fitness Function is often derived from
objective function
F(X) = f(x) for Maximization problem
F(x)= 1/f(x) for minimization problem if f(x)
0
F(x)= 1/ (1 + f(x)) if f(x) = 0

CREATING NEW GENERATION OF OFFSPRING

OPERATORS: SELECTION
FITNESS PROPORTIONATE SELECTION (FI/F )
NUMBER OF REPRODUCTIVE TRIALS FOR
INDIVIDUALS

Roulette-wheel selection
wheel spaced in proportion to fitness values
N (pop size) spins of the wheel

Tournament Selection

List all individual with fitness value

Select any two individual at random
Take the one with higher fitness value
Repeat the same for selecting full size
of
population.

RANK SELECTION
Roulette wheel has a problem when the fitness
values of individual differs very much. If best
chromosome has fitness 90% then 90% wheel will
be occupid by the chromosome.
In Rank selection all chromosomes are assign
ranks irrespective of their fitness value.
If there are five chromosome then they will be
having proportionately 1/15 , 2/15, 3/15, 4/15 and
5/15 of the wheel

WHY CROSSOVER AND MUTATION?

Crossover

Produces new solutions while

remembering the characteristics of old
solutions
Partially preserves distribution of strings
across schemas

Mutation

Randomly generates new solutions

which cannot be produced from existing
population
Avoids local optimum

GENETIC ALGORITHMS:
EXAMPLE

The SGA for our example will use:

A population size of 6,
A crossover usage rate of 1.0, and
A mutation rate of 1/7.

Lets try to solve the following problem

f(x) = x2, where -2.0 x 2.0,

Let l = 7, therefore our mapping function will be

d(2,-2,7,c) = 4*decode(c)/127 - 2

GENETIC ALGORITHMS:
AN EXAMPLE RUN (BY HAND)

Randomly Generate an Initial Population

Genotype Phenotype Fitness
Person 1: 1001010
0.331 Fit: ?
Person 2: 0100101
- 0.835 Fit: ?
Person 3: 1101010
1.339 Fit: ?
Person 4: 0110110
- 0.300 Fit: ?
Person 5: 1001111
0.488 Fit: ?
Person 6: 0001101
- 1.591 Fit: ?

GENETIC ALGORITHMS:
AN EXAMPLE RUN (BY HAND)

Evaluate Population at t=0

Genotype Phenotype
Person 1: 1001010
0.331
Person 2: 0100101
- 0.835
Person 3: 1101010
1.339
Person 4: 0110110
- 0.300
Person 5: 1001111
0.488
Person 6: 0001101
- 1.591

Fitness
Fit: 0.109
Fit: 0.697
Fit: 1.790
Fit: 0.090
Fit: 0.238
Fit: 2.531

GENETIC ALGORITHMS:
AN EXAMPLE RUN (BY HAND)

Select Six Parents Using the Roulette Wheel

Genotype Phenotype Fitness
Person 6: 0001101
- 1.591 Fit: 2.531
Person 3: 1101010
1.339 Fit: 1.793
Person 5: 1001111
0.488 Fit: 0.238
Person 6: 0001101
- 1.591 Fit: 2.531
Person 2: 0100101
- 0.835 Fit: 0.697
Person 1: 1001010
0.331 Fit: 0.109

GENETIC ALGORITHMS:
AN EXAMPLE RUN (BY HAND)

Create Offspring 1 & 2 Using Single-Point Crossover

Genotype Phenotype Fitness
Person 6: 00|01101 - 1.591 Fit: 2.531
Person 3: 11|01010
1.339 Fit: 1.793
Child 1 : 0001010
- 1.685 Fit: ?
Child 2 : 1101101
1.433 Fit: ?

GENETIC ALGORITHMS:
AN EXAMPLE RUN (BY HAND)

Create Offspring 3 & 4

Genotype Phenotype Fitness
Person 5: 1001|111
0.488 Fit: 0.238
Person 6: 0001|101 - 1.591 Fit: 2.531
Child 3 : 1011100
0.898 Fit: ?
Child 4 : 0001011
- 1.654 Fit: ?

GENETIC ALGORITHMS:
AN EXAMPLE RUN (BY HAND)

Create Offspring 5 & 6

Genotype Phenotype Fitness
Person 2: 010|0101 - 0.835 Fit: 0.697
Person 1: 100|1010
0.331 Fit: 0.109
Child 5 : 1101010
1.339 Fit: ?
Child 6 : 1010101
0.677 Fit: ?

GENETIC ALGORITHMS:
AN EXAMPLE RUN (BY HAND)

Evaluate the Offspring

Genotype Phenotype Fitness
Child 1 : 0001010
- 1.685 Fit: 2.839
Child 2 : 1101101
1.433 Fit: 2.054
Child 3 : 1011100
0.898 Fit: 0.806
Child 4 : 0001011
- 1.654 Fit: 2.736
Child 5 : 1101010
1.339 Fit: 1.793
Child 6 : 1010101
0.677 Fit: 0.458

GENETIC ALGORITHMS:
AN EXAMPLE RUN (BY HAND)
Population at t=0
Genotype Phenotype
Person 1: 1001010
0.331
Person 2: 0100101 - 0.835
Person 3: 1101010
1.339
Person 4: 0110110 - 0.300
Person 5: 1001111
0.488
Person 6: 0001101 - 1.591

Fitness
Fit: 0.109
Fit: 0.697
Fit: 1.793
Fit: 0.090
Fit: 0.238
Fit: 2.531

Is Replaced by:
Child
Child
Child
Child
Child
Child

1
2
3
4
5
6

:
:
:
:
:
:

Genotype Phenotype Fitness

0001010 - 1.685
Fit: 2.839
1101101
1.433
Fit: 2.053
1011100
0.898
Fit: 0.806
0001011 - 1.654
Fit: 2.736
1101010
1.339
Fit: 1.793
1010101
0.677
Fit: 0.458

GENETIC ALGORITHMS:
AN EXAMPLE RUN (BY HAND)

Population at t=1
Genotype Phenotype Fitness
Person 1: 0001010
- 1.685 Fit: 2.839
Person 2: 1101101
1.433 Fit: 2.054
Person 3: 1011100
0.898 Fit: 0.806
Person 4: 0001011
- 1.654 Fit: 2.736
Person 5: 1101010
1.339 Fit: 1.793
Person 6: 1010101
0.677 Fit: 0.458

GENETIC ALGORITHMS:
AN EXAMPLE RUN (BY HAND)

The Process of:

Selecting six parents,

Allowing the parents to create six offspring,
Mutating the six offspring,
Evaluating the offspring, and
Replacing the parents with the offspring

Is repeated until a stopping criterion has been

reached.

CONVERGENCE

progression towards uniformity in

population
premature convergence?
--(local optima)

HOW GA WORKS I.E. SCHEMAS

Population

Strings over alphabet {0,1} of length L

s 10010
E.g.

Schema

A schema is a subset of the space of all

possible individuals for which all the
genes match the template for schema H.
Strings over alphabet {0,1,*} of length L
H [1* *10] {10010,10110,11010,11110 }
E.g.

HYPER-PLANE MODEL

Search space

Individuals

A hyper-cube in L dimensional space

Vertices of hyper-cube

Schemas

Hyper-planes formed by vertices

0**

SAMPLING HYPER-PLANES

Look for hyper-planes (schemas) with good

fitness value instead of vertices (individuals) to
reduce search space

Each vertex
Member of 3L hyper-planes
Samples hyper-planes

Average Fitness of a hyper-plane can be

estimated by sampling fitness of members in
population

Selection retains hyper-planes with good

SCHEMA THEOREM

Schema Order O(H)

Schema order, O(.) , is the number of non * genes in
schema H.
E.g. O(1**1*) = 2
Therefore schema H will represent 2 l-o(H) individuals.

Schema Defining Length (H)

Schema Defining Length, (H), is the distance between
first and last non * gene in schema H
E.g. (1**1*) = 4 1 = 3

Schemas with short defining length, low order with

fitness above average population are favored by GAs

Schema Theorem Terminology (Cont.):

Let m(H,t) denoted the number of instances of H that are in

the population at time t.
Let f(H,t) denote the average fitness of the instances of H
that are in the population at time t.
Let favg(t) represent the average fitness of the population at
time t.
Let pc and pm represent the single-point crossover and
mutation rates.
According to the Schema Theorem there will be:

m(H,t+1) = m(H,t) f(H,t)/favg(t) instances of H in the next

population if H has an above average fitness.

Schema Theorem Terminology (Cont.):

According to the Schema Theorem there will be:

m(H,t+1) = m(H,t) f(H,t)/favg(t)
instances of H in the next population if H has an above
average fitness.
If we let f(H,t) = favg(t) + c favg(t), for some c > 0, then
m(H,t+1) = m(H,t)(1+c), and
If m(H,0) > 0 then we can rewrite the equation as
m(H,t) = m(H,0)(1+c)t
What this says is that proportionate selection allocates an
exponentially increasing number of trials to above average
schemata.

FORMAL STATEMENT

Selection probability

Crossover probability

m( H , t ) f ( H , t )
E (m( H , t 1))
f
(H )
c
L 1

P(hcrossover ) p
Mutation probability

P (hmutation ) ( H ) pm
Expected number of members of a
schema m( H , t ) f ( H , t )
(H )
E (m( H , t 1)

(1 pc

L 1

pm( H ))

AREA OF APPLICATION
GAs can be used when:
Non-analytical problems.
Non-linear models.
Uncertainty.
Large state spaces.

NON-ANALYTICAL PROBLEMS

Fitness functions may not be

expressed analytically always.

Domain specific knowledge may not

be computable from fitness function.

Scarce domain knowledge to guide the

search.

NON-LINEAR MODELS

Solutions depend on starting values.

Non linear models may converge

to local optimum.

Impose conditions on fitness

functions such as convexity, etc.

UNCERTAINTY

Noisy / approximated fitness

functions.

Changing parameters.

Changing fitness functions.

LARGE STATE SPACES

Heuristics focus only on the immediate

area of initial solutions.

State-explosion problem: number of states

huge or even infinite! Too large to be
handled.

State space
understood.

may

not

be

completely

GA APPLICATION EXAMPLES

Function optimizers
difficult, discontinuous, multi-modal, noisy functions
Combinatorial optimization
layout of VLSI circuits, factory scheduling, traveling salesman
problem
Design and Control
bridge structures, neural networks, communication networks
design; control of chemical plants, pipelines
Machine learning
classification rules, economic modeling, scheduling strategies
Portfolio design, optimized trading models, direct marketing
models, sequencing of TV advertisements, adaptive agents, data
mining, etc

WHEN NOT TO USE GA!

Constrained mathematical
optimization problems especially
when there are few solutions.
Constraints are difficult to
incorporate into a GA.
Guided domain search is possible
and efficient.