Documente Academic
Documente Profesional
Documente Cultură
Genetic Algorithms:
Colour Image Segmentation
Keri Woods
kwoods@cs.uct.ac.za
Supervised by: Audrey Mbogho
Abstract
Image segmentation has great importance in many image processing applications,
and yet no general image segmentation exists. Image segmentation is complicated
task, often with many parameters needing to be tuned to get good results. This
report researches and discusses the concepts of image segmentation, genetic algorithms and segmentation evaluation. Due to the flexibility of genetic algorithms and
their ability to effectively explore large search spaces, it may be viable to use them
to improve existing image segmentation methods. A region merging algorithm was
implemented and evaluated using quantitative means. These segmentation results
were compared to those of two other image segmentation methods: region growing
and watershed segmentation. Our region merging method was shown to produce
average results. A genetic algorithm was implemented in an attempt to improve
the segmentation results by evolving the segmentation parameters. A fitness function needing neither human input nor a ground truth segmentation comparison was
proposed. The results on the effect of the genetic algorithm on the performance of
the genetic algorithm are inconclusive. However, even if the genetic algorithm does
offer an improvement, it has the major drawback of running very slowly. There are
many possible way of improving this system, and it seems to be an area where much
research is needed.
Contents
Abstract
List of Figures
iv
1 Introduction
1.1
Project Description . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2
Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3
Project Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3.1
Limited Scope . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1
General Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2
Image Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.1
Colour Segmentation . . . . . . . . . . . . . . . . . . . . . . .
2.2.2
2.2.3
2.3
Segmentation Evaluation . . . . . . . . . . . . . . . . . . . . . . . . .
2.4
Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4.1
2.4.2
Fitness Function . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.5
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2
14
Chromosome Structure . . . . . . . . . . . . . . . . . . . . . . 23
3.2.2
Reproduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.2.3
3.2.4
Fitness Function . . . . . . . . . . . . . . . . . . . . . . . . . 27
ii
4 Experimental Design
29
4.1
4.2
4.3
4.2.1
Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.2.2
Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.3.2
Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5 Results
32
5.1
Region Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.2
Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.2.1
5.3
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6 Future Work
38
7 Conclusions
39
iii
List of Figures
3.1
Module Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2
UML Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3
Module Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.4
RegionList Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.5
Region Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.6
3.7
Parameter Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.8
Chromosome Structure . . . . . . . . . . . . . . . . . . . . . . . . . . 26
5.1
5.2
5.3
This table shows the comparison of the RMSE values using genetic
algorithms vs. traditional algorithms . . . . . . . . . . . . . . . . . . 34
5.4
iv
Chapter 1
Introduction
1.1
Project Description
In this project we review image segmentation methods and look into the use genetic algorithms for colour image segmentation. We base the segmentation part of
our implementation on a region merging method described in Baatz and Schape[8].
For our genetic algorithm, we follow a later improvement by Feitosa et al [22] which
evolves the parameters for region merging to give visibly better results. The key
to this improvement is a well-chosen fitness function, which captures the requirements of good segmentation without needing a comparison with a ground truth
segmented image. However, Feitosas fitness function relies on manual (user-based)
segmentation to test the success or failure of the genetic algorithm. Our approach,
in contrast, does not need user feedback, making it useful to real-world, automated
segmentation applications.
1.2
Motivation
Image segmentation is an important process and its results are used in many image
processing applications. However, despite its importance, there doesnt to seem to be
any general method of image segmentation that works well on all images[42]. Image
segmentation involves a lot of uncertainty, often with many parameters that need to
be tuned to provide optimal results. For example, the Phoenix image segmentation
method has 14 adjustable parameters[12]. This large number of parameters create a
very large search space. Colour images have even more information than grey-scale
images, and this information can be used to create higher quality segmentation. It
does, however, increase the complexity of the problem. A way of handling the large
search space is to use a directed search method, such as genetic algorithms.
Genetic algorithms have many qualities that make them well suited to the problem
of image segmentation, such as the ability to forego a local optimum to reach a global
1
optimum and the ability to efficiently find an optimal solution from within a large
search space[9].
Genetic algorithms could allow an image segmentation process that usually requires
manual input to become unsupervised.
Genetic algorithms have been used to successfully colour segment images[12]. Due
to their flexibility, it seems feasible to be able to use them to come up with a general
segmentation method.
1.3
Project Scope
1.3.1
Limited Scope
Due to time limitations, some of the aims mentioned in the project proposal were
not met. These include:
1. Only one segmentation method was implemented and experimented with,
rather than comparing a number of different ones
2. The genetic algorithm implemented wasnt compared with those developed by
others
3. No experimentation on genetic algorithm parameters was done
Chapter 2
Background and Related Work
In this section, we start by looking at the concept and importance of image segmentation and mention the requirements for good image segmentation. The implication
of using colour in image segmentation is explored and methods for image segmentation are briefly discussed. Problems with existing image segmentation methods
are mentioned. The evaluation of image segmentation is briefly discussed. Genetic
algorithms are then introduced and their suitability for use in image segmentation
is examined. We explore various applications of genetic algorithms to the problem
of image segmentation. Genetic algorithm fitness functions that have been used by
others with image segmentation as discussed. Finally, the feasibility of the use of
genetic algorithms for general colour image segmentation is considered and design
issues for such an algorithm are discussed.
2.1
General Overview
Image segmentation is an important process and its results are used in many image
processing applications. However, there is no general way to successfully segment all
images[42]. Colour images have more information than grey-scale images, and this
information can be used to create higher quality segmentation. It does, however,
increase the complexity of the problem. A way of handling this complexity is to use
a directed search method, such as genetic algorithms. Genetic algorithms, which
mimic the process of evolution, have many qualities that make them well suited to
the problem of image segmentation, such as the ability to forego a local optimum
to reach a global optimum[9] and the ability too efficiently find an optimal solution
from within a large search space[6].
The main uses of genetic algorithms in image segmentation are for the modification
of parameters in existing segmentation algorithms and pixel-level segmentation[21].
Various algorithms that successfully apply genetic algorithms to image segmentation
have been developed. Though these results are promising, none of them solve this
open problem.
2.2
Image Segmentation
2.2.1
Colour Segmentation
Until recently, most image segmentation has been performed on grey-scale images. Processing colour images requires much more computation than the processing of grey-scale ones, but now with the increasing speed and decreasing cost
of computation, colour image processing has been much researched in the last
decade[33][19][15][42].
Colour images contain far more information than monochrome images. Each pixel
in a colour image has information about brightness, hue and saturation. There
are many models to represent the colours, including RGB (red, green, blue), CMY
(cyan, magenta, yellow), HSV (hue, saturation, intensity), YIQ, HSI and many
others. Several colour spaces have been used for image segmentation[19] and no
general advantage of one colour space has yet been found[42].
Many of the colour image segmentation algorithms are derived from methods of
grey-scale image segmentation. However, colour creates a more complete representation of an image and exploiting this fact can result in a more reliable segmentation. Specialised techniques suited to the nature of colour information have been
devised[33][19].
2.2.2
Image segmentation is an old and important problem, and there are numerous image
segmentation methods. Most of these methods were developed to be used on a
certain class of images and therefore arent general image segmentation methods[9].
Bhanu and Lee[10] divide the image segmentation algorithms into three major
categories:
1. Edge Based
5
2. Region Based
3. Clustering Based
4. Model Based1
This isnt discussed here as it is used to match particular objects and so isnt relevant to
general image segmentation
2.2.3
Image segmentation is easy when objects have distinct colours and are well separated, but can be a problem if there are many complex objects with less distinct colour. Gradual variation in colour[41], illumination[43], shading[49] and
textures[40] are also possible problems.
A brute force method of dealing with image segmentation would be to enumerate
all possible partitions of the image and evaluate each one. This creates an extremely
large search space, and so this method is not feasible[9].
Even once a segmentation method has been chosen, there are usually many parameters that need to be tuned to create high quality segmentation. For most methods,
it is not feasible to perform an exhaustive search of these parameters.
Despite the many methods for image segmentation, there is no general algorithm
that works well for all images. Because of the wide variety of images, a general
algorithm needs to be adaptable. Only then can a segmentation algorithm cope
with a wide variety of images[10].
Many adaptive methods have been used for image segmentation, including genetic algorithms[11], neural networks[20], self-adaptive regularisation[47], ant colony
optimization[37], fuzzy clustering[2] and simulated annealing[51].
2.3
Segmentation Evaluation
Methods of evaluating image segmentation are divided into two main categories[50]:
1. Analytical Methods
2. Empirical Methods
Analytical methods look at the actual segmentation algorithm itself, rather than
its results, while empirical methods evaluate the segmentation algorithm by looking
at its results. In this case, it is the empirical methods that we are more interested
in. Empirical methods can be further sub divided into:
1. Goodness Methods
2. Discrepancy Methods
Goodness methods evaluate the quality of the segmentation by looking at its desirable properties, and dont compare it to any other segmentation. It is this type of
method that we wish to use as a fitness function for the genetic algorithm, because
they dont need any prior knowledge of the image segmentation or ground truth
segmented image and so are advantageous for an unsupervised algorithm. Discrepancy methods compare the segmentation results to an ideally segmented image, to
see how much its segmentation differs from the target segmentation. It is this class
of method that we will use to evaluate the results of our segmentation algorithm
during experimentation.
2.4
Genetic Algorithms
Genetic algorithms are an optimization technique that can be used in image segmentation [11]. It mimics natural selection, allowing an algorithm to adapt. Solutions
are represented by a population of individual chromosomes, usually represented as
binary strings. A chromosome is made up of genes, each of which can represent a
particular characteristic. Each individual in the population is evaluated and given
a fitness score based on how well they solve the particular problem. The higher
the individuals fitness score, the greater their probability of breeding. Breeding
creates the next generation through crossover and mutation. Crossover combines
the chromosome of two individuals, creating a new individual which is unlike either
of the parents. Mutation, which occurs only a small percent of the time, randomly
alters a new individuals chromosome. Since the more optimal individuals have a
greater chance of breeding, the population tends to evolve and reach an optimal
solution[45][5][23].
Genetic algorithms have been used to solve a wide variety of problems, including
numerical and combinatorial optimisation, circuit design and cellular automata rule
design[23]. In image processing, genetic algorithms have successfully been used for
feature extraction, object recognition[44], knowledge based segmentation[36] and
image classification[18].
Image segmentation is easily and naturally formulated as on optimisation problem.
It can either be seen as finding the optimal segmentation amongst all candidate segmentations, or as finding the optimal parameters for an existing image segmentation
algorithm. In both cases, this creates an extremely large search space, indicating
the use of genetic algorithms[6].
Genetic algorithms are advantageous in that they are able to forego local optima
in an attempt to reach the global optimum[9]. This makes them far less likely to
8
2.4.1
Farmer and Shugars[21] divide the genetic algorithms used for image segmentation
into two major classes:
1. Parameter selection, where genetic algorithms are used to modify the parameters of an existing image segmentation method to improve its output.
2. Pixel-level segmentation, where genetic algorithms are used to perform region
labelling.
Most image segmentation methods have many parameters that need to be optimised, and therefore the first method is used more often[11]. Many such methods are
9
Parameter Modification
Most image segmentation methods have many parameters, constants and thresholds
that need to be adjusted to produce optimal segmentation results. This creates a
very large search space. Since the parameters typically interact in complex and nonlinear ways, an analytic solution is not generally possible. With a reasonable amount
of computation, genetic algorithms are able to find good approximations of a global
optimum within a large search space. They are therefore well suited to problems
involving parameter optimization. Most of the applications of genetic algorithms to
image segmentation involve the optimisation of various parameters[44][11][42][40].
Bhanu et al[11] pose image segmentation as an optimisation problem. They define a general segmentation method, whereby genetic algorithms are applied to the
parameters of various well known image segmentation methods. They advocate the
use of genetic algorithms to adapting the parameters of knows segmentation methods in order to be applicable to general images. They used outdoor colour imagery
and adapted 4 parameters of the Phoenix segmentation algorithm with genetic algorithms. They had successful results, producing high quality image segmentation
with a reasonable amount of computation. Even though they perform well on outdoor scenes, these algorithms have not been proved to be able to cope with general
images. The fact that these algorithms can be modified to adapt the parameters of
other segmentation methods makes this method very promising.
Feitosa et al[22] adopt a very similar approach and use genetic algorithms to modify
the parameters of a region merging segmentation algorithm. They use a fitness
function that measures the similarity of resulting segments to a target segmentation
provided by a user. Though computation is straight forward and intuitive, manual
segmentation is still necessary beforehand. This method can easily be adapted to
modify parameters of other segmentation methods.
Zingaretti el al[52] propose using genetic algorithms in unsupervised colour image
segmentation. This is another case of parameters of an existing image segmentation
method being tuned by genetic algorithms. A key difference in this method is
that it performs multi-pass thresholding. Different thresholds are adapted during
each pass of genetic algorithms. An important advantage of this method over the
previous one is that segmentation is performed totally unsupervised, without any
manual segmentation. It also doesnt rely on any prior information regarding the
type of image that is being processed or the task for which the segmentation results
will be used. This approach successfully segmented a wide variety of images, with
10
2.4.2
Fitness Function
As mentioned previously, the choice of an appropriate fitness function is very important. It is the fitness function that is largely responsible for the quality of the
image segmentation obtained. The fitness function is what the algorithm "aims" to
optimise. When dealing with image segmentation, one needs a fitness function that
indicates how well the image has been segmented.
Feitosa et al [22] calculate fitness by quantifying the difference between the segmentation obtained using the individuals parameters and a target segmentation. The
target segmentation is obtained by manual segmentation, making this a discrepancy
method of evaluation. This works fine for experimental purposes - it shows that
genetic algorithms can be used to improve the quality of segmentation. However,
this fitness function makes the segmentation program of little practical use. If one
needs to segment the image beforehand, no benefit is obtained from the program.
Many other implementations also used discrepancy methods for calculating fitness,
including [29][39]. These methods arent relevant for use as a fitness function in this
implementation, as we need an unsupervised method of image segmentation.
Instead of using comparing the segmentation to a target segmentation, Zingaretti
el al[52] compare it to an edge map obtained by applying a Roberts edge operator
to the image. This makes the application more useful, but using an edge map as
"perfect" segmentation means that their program tries to obtain this segmentation.
It doesnt seem that segmentation better than that obtained with the edge operator
can be obtained, and using an edge filter is much more efficient.
As discussed previously, good image segmentation meets certain requirements:[33][9][10][27]
1. Every pixel in the image belongs to a region
12
2.5
Conclusion
The use of genetic algorithms in image segmentation shows promising results. Genetic algorithms are a commonly used approach to optimising the parameters of
existing image segmentation algorithms The major decisions are choosing a method
of segmentation to which genetic algorithms will be applied, finding a fitness function that is a good measure of the quality of image segmentation and finding a
meaningful way to represent the chromosomes.
13
Chapter 3
System Design and Implementation
This system is composed of two modules:
1. Region Merging Module
2. Genetic Algorithm Module
The Region Merging Module performs a region merging method of image segmentation on the input image. The Genetic Algorithm Module optimises the region
merging parameters. varies the region merging parameters, in an attempt to improve the segmentation.
The Region Merging Module can be totally independent of the Genetic Algorithm
Module, with the parameters being set manually rather than letting them be altered
by the genetic algorithm. The Genetic Algorithm Module only calls the Region
Merging Module to calculate the fitness value. The interaction between the modules
is shown below in Figure 3.1:
The Region Merging Module receives and stores the input image. Whenever the
Genetic Algorithm Module requires a fitness value, it sends the segmentation parameters that it is currently working with to the Region Merging Module, requesting
a fitness value. The Region Merging Module uses these parameters, segments the
image, evaluates the fitness and sends the fitness value to the Genetic Algorithm
Module. This process is repeated until the genetic algorithm terminates. The Genetic Algorithm Module then sends the parameters that corresponded to the best
fitness value to the Region Merging Module. The Region Merging Modules uses
these parameters to segment the image and outputs the results of this segmentation.
These modules are loosely coupled, with the region merging module being able to
be replaced with a different segmentation method that needs its parameters tuned.
The fitness function can also be easily replace with different fitness function. This
14
Input Image
Send Parameters
(Request Fitness)
Region Merging
Module
Return Fitness
Genetic Algorithm
Module
Final Parameters
Output Result
15
3.1
The idea behind region merging[8][22] is to divide the image up into many small
regions. Adjacent regions can be merged together, based on whether certain criteria
are met. This continues until no regions meet the merging criteria.
This particular method of region merging looks at the heterogeneity of the various
regions in the image when deciding whether or not regions will be merged. The
goal of the merging procedure is to minimise the weighted heterogeneity. There are
two main classes of heterogeneity: spectral heterogeneity and spatial heterogeneity.
The spectral heterogeneity measures how varied the colour within the region is.
The spatial heterogeneity measures the deviation of the region from a compact,
smooth shape. This is separated into compactness and smoothness components.
Compactness looks at the ratio of the perimeter to the square root of the area,
while smoothness looks at the ratio of the perimeter to the length of a bounding
box of the region, parallel to the image borders.
Initially, each pixel is initialised as a region. Regions are then visited in turn,
to decide whether that region will merge with another region. The order in which
regions are visited is discussed later.
When considering whether a region is going to be merged, each of the regions adjacent to the region are considered. A fusion factor between the region and each
particular adjacent region is calculated. The fusion factor takes into account the
various heterogeneity components:
A colour heterogeneity of each of the colour channels (red, green and blue) is calculated separately, and added to get the spectral heterogeneity component (modified
from the version presented in the paper):
hcolour = c wc (nObj3 cObj3 nObj1 cObj1 nObj2 cObj2 )
where:
Obj1 is the region selected for merging
Obj2 is the region adjacent to Obj1
Obj3 is the result of merging Obj1 and Obj2
16
lObj2
)
nObj2
Once the fusion factor for each adjacent region has been calculated, the region with
the minimum fusion factor is considered. If its fusion factor is less than a certain
threshold, then that adjacent region will be merged to the region being initially
considered. The threshold value used here to the square of a scale parameter. This
is known as a scale parameter, as adjusting this will result in differences in the
resulting region size. The larger the scale parameter, the more merges that will
happen, resulting in larger regions.
This process is then repeated, with another region being selected and the fusion
factor between it and its neighbouring regions being calculated. This process is
repeated until no more regions can be merged.
Baatz[8] mentions that one of the problems with their method of region merging
is that the order in which regions are visited in an attempt to merge them makes
a difference in the segmentation results. They recommend that when choosing a
sequence of starting regions that sequential merges should be distributed as far
from each other as possible.
q
q
1
1
N x2 xbar 2 =
N
In this case, =
17
3.1.1
The system only depends on the CImg[46] class, with its accompanies cimg_library
namespace, as well as standard C++ templates and functions.
We make broad use of the Standard Template Library (STL) to create containers
for the point, edge and region data structures. In each case, we use the container
that minimises the segmentation algorithms running time.
The system is composed of:
1. class Region
2. class RegionList
3. struct Point
Their interactions can be seen in the UML diagram, shown below in Figure 3.2.
RegionList and Region are the two main classes, with Point being just a simple
struct. Along with this, various typedefs were defined:
typedef
typedef
typedef
typedef
typedef
typedef
typedef
Initially, the pixels are given region labels left to right, top to bottom. Region ordering is an
ordering of these regions, with regions that no longer exist being ignored
18
std::multimap
...
...
std::list
std::map
...
...
...
...
RegionList
Region
RImage rimage;
Image image;
RMap* regions;
unsigned int imageWidth;
unsigned int imageHeight;
float wRed;
float wGreen;
float wBlue;
float wColour;
float scale;
float wCompact;
RImage* rimage;
Image* image;
PointList* pixels;
EdgeMap* edges;
Label label;
unsigned int maxX;
unsigned int maxY;
unsigned int minX;
unsigned int minY;
Label sumR;
Label sumG;
Label sumB;
float averageR;
float averageG;
float averageB;
unsigned int area;
unsigned int perimeter;
RegionList( );
~RegionList();
getSize ();
mergeAll ();
mergeRandom ();
saveResults();
saveRegionMap();
addRegion ();
getRegion ();
removeRegion ();
getFusionFactor ();
getHCompact ();
getHSmooth ();
getHColour ();
getHShape ();
tryMerge ( );
coordToLabel ();
cimg_library::
cimg
...
...
Region();
~Region();
void addEdge ();
unsigned int getArea ();
Label getLabel ();
float getAverageR ();
float getAverageG ();
float getAverageB ();
unsigned int getPerimeter ();
unsigned int getPerimeterCombined ();
unsigned int getPerimeterBB ();
unsigned int getPerimeterBBCombined ();
float getStdDev ();
float getStdDevCombined ();
void splice ( );
EdgeMap* getEdges ();
PointList* getPixels ();
void removeAllEdgesTo ( );
void changeAllEdgesTo ( );
Point
int x
int y
Point( int a, int b)
19
Input Image
Send Parameters
(Request Fitness)
Region Merging
Module
Return Fitness
Genetic Algorithm
Module
Final Parameters
Output Result
RegionList
RImage rimage;
Image image;
RMap* regions;
unsigned int imageWidth;
unsigned int imageHeight;
float wRed;
float wGreen;
float wBlue;
float wColour;
float scale;
float wCompact;
RegionList( const char* filename, float r, float g, float b, float col, float s, float com );
~RegionList();
int getSize ();
void mergeAll ();
bool mergeRandom ();
void saveResults( const char* filename );
void saveRegionMap( const char* filename );
void addRegion ( const Label& label, const Point& p );
Region* getRegion ( const Label& label );
void removeRegion ( const Label& label );
float getFusionFactor ( Region* region1, Region* region2 );
float getHCompact ( Region* region1, Region* region2 );
float getHSmooth ( Region* region1, Region* region2 );
float getHColour ( Region* region1, Region* region2 );
float getHShape ( float hCompact, float hSmooth );
bool tryMerge ( Region* r );
Label coordToLabel( unsigned int x, unsigned int y );
21
Region
RImage* rimage;
Image* image;
PointList* pixels;
EdgeMap* edges;
Label label;
unsigned int maxX;
unsigned int maxY;
unsigned int minX;
unsigned int minY;
Label sumR;
Label sumG;
Label sumB;
float averageR;
float averageG;
float averageB;
unsigned int area;
unsigned int perimeter;
Region Class
The Region class represents a region in a segmented or partially segmented image.
The structure of the class is shown in Figure 3.5.
The main components of the Region class are:
1. PointList* pixels
22
2. EdgeMap* edges
3. Label label
Pixels represent the pixels making up the image. They are Points, stored as a list
(PointList). A list was used because insertion by pushing on to the front of the
list runs in constant time. Deletion of pixels only occurs when all the pixels in the
region are deleted3 are deleted, so at each pixel delete takes constant time. (Deletion
runs in linear time if the algorithm wasnt already iterating through the list.)
Edges are stored as a multimap (EdgeMap). A boundary pixel in a region can
point to one or more other regions. However, edges are stored separately from pixels,
since actual pixel positions are unimportant when using edges to check connectivity.
Insertion and find are both logarithmic time operations on a multimap. When
erasing, all the elements with the same key are deleted. This is an logarithmic time
operation, plus linear time is required for deleting n equal elements.
Regions are assigned unique integer values (Label) based on their initial positions
in the image.
Pointers to both the original image and the region map are included as this class
needs access to them. All the other attributes are stored, and updated every time
the region is altered. These values are needed for the fusion factor calculation. It
is worth using a little extra memory to store these values and save the numerous
computations that would be required otherwise.
3.2
The Genetic Algorithm Module is a simple module, with much of the functionality
being provided by the GALib library[1]. Its basic behaviour and interaction with
the Region Merging Module is shown in Figure 3.6.
The genetic algorithm is represented by a GASimpleGA object, provided by GALib.
This class is used for genetic algorithms with non-overlapping populations, and inherits from the general GAGeneticAlgorithm class.
3.2.1
Chromosome Structure
23
Initialize
Population
request fitnes
Select Indiviuals
for Mating
return fitness
Mate individulal
to produce offspring
Mutate Offspring
Insert Offspring
into Population
Are stopping
criteria satisfied
Select Indiviuals
for Mating
24
scale
wColour
wCompact
wRed
wGreen
wBlue
Range
Precision
0-100
0-1
0-1
0-1
0-1
0-1
0.1
0.01
0.01
0.1
0.1
0.1
No. of
Values
1000
100
100
10
10
10
No. of
Bits
10
7
7
4
4
4
Scale
wColour
wCompact
wRed
wGreen
wBlue
10 bits
7 bits
7 bits
4 bits
4 bits
4 bits
3.2.2
Reproduction
In this implementation, one point crossover is used. In one point crossover, a random
position on the chromosome is select and the two individuals exchange the genes
on one side of that point. The mutation used is flip mutation, which is the typical
mutation operation used on binary strings: bits in the chromosome are flipped with
a given probability.
Elitism is used, with the best individual in each generation being carried over to
the next generation.
3.2.3
26
3.2.4
Fitness Function
As mentioned previously, one would like a fitness function that evaluates the following requirements of good image segmentation:
1. Every pixel in the image belongs to a region
2. A region is connected
3. Each region is homogeneous
4. Adjacent regions cant be merged into a single homogeneous region
5. No regions overlap
Requirements 1, 2 and 5 are always obtained when using this region merging technique, therefore the important characteristics to quantify is that each region is homogeneous and that adjacent regions cant be merged into one homogeneous region.
Various different fitness function were. Liu[31] proposed goodness method to be
used for the quantitative evaluation of the performance of image segmentation, which
could be used as a fitness measure. The evaluation function F on segmented image
I is defined as:
F (I) =
1
1000N 2
R e2
Ri=1 Ai i
where: N 2 is the size of the image R is the total number of regions Ai is the area
of region i ei is the colour error, Euclidean distances in colour space between the
values of the pixels in region i in the original image and the values of the pixels in
the segmented image, defined as:
q
Ai
ei = j=1
(C1jR C2jR )2 + (C1jG C2jG )2 + (C1jB C2jB )2
where: R, G and B are the colour channels C1j is the colour of pixel j in the
original image C2j is the assigned colour of the same pixel in the segmented image
Lower values of F (I) correspond to better segmentation. F evaluates the results
Q(I) =
1
1000N 2
e2
R(Ai ) 2
i
RR
i=1 [ 1+logAi + ( Ai ) ]
27
28
Chapter 4
Experimental Design
The main areas that need to be tested are:
1. Evaluation of region merging image segmentation
2. Segmentation improvement provided by genetic algorithms
3. Improvement of genetic algorithm by altering genetic algorithm parameters
4.1
Various images from the Berkeley Image Dataset[34] will be chosen to be segmented.
These represent a wide range of images, from those that are easy to segment to
those that are much more difficult to segment, such as a camouflaged animal. These
results will be evaluated subjectively. For example, it can be seen whether or not a
camouflaged animal is segmented out.
Dorin Comaniciu[17] has images on his website with the corresponding successfully
segmented images. These segmented images will be used as ground truths when
evaluating our system. It would have been better if images manually segmented
by humans were used as the ground truth, as they represent "ideal" segmentation.
However, no manually segmented images in the correct format were found. The
region merging algorithm will be run on each of the images to produce the resulting
segmented images. One needs to find an objective way of evaluating the performance
of this segmentation in comparison to the ground truth images.
This evaluation will be done using ImageMagicks[32] compare function, using the
MSE (mean squared error) option. The difference between the values of each pixel of
the ground truth segmentation and the corresponding pixel in the segmented image
that is being evaluated. This value is then squared, to get rid of negative values
29
and penalise more heavily where the pixel value difference is large. These values are
then summed over the whole image to get the MSE. The root of this is taken to get
, the RMSE (root mean squared error). This value is then divided by the number
of pixels in the image to obtain the normalised RMSE, an evaluation measure that
is size independent.
This same procedure will be repeated with the region growing and watershed algorithms implemented by Marco Gallotta. The results of all three of these image
segmentation algorithms will be compared.
4.2
4.2.1
Hypothesis
The use of the genetic algorithm will improve the quality of the image segmentation.
4.2.2
Method
Each image to be evaluated will be segmented using the region merging algorithm.
The segmentation parameters will be tweaked to attempt to obtain the best results.
The best results will be evaluated using a normalised RMSE, as discussed above,
and recorded. Due to the fact that the results from genetic algorithms arent deterministic, the images will be segmented five number times1 by the genetic algorithm
version. Each run will be evaluated and the average score for each image recorded.
The average score for each algorithm type will calculated and the results will be
compared.
The population size used in the genetic algorithm should be made as large as
computation time allows. The other genetic algorithm parameters should be tweaked
to try and obtain the best possible results. Note of the values used should be made,
so that the experiment can be repeated2 .
1
5 was an arbitrary choice. The more times the segmentation is run, the more accurate the
results, so it should be repeated as many times as is feasible
2
It may be preferable to perform the next part of the experimentation first, examining the effect
of modifying the parameters of the genetic algorithm, to determine the optimal parameters
30
4.3
4.3.1
The main purpose of this section of the experimentation is to see what effect changing
the genetic algorithm parameters and other factors - such as reproduction operators
and termination criteria - will have on the quality of the segmentation results. This
knowledge can then be used to find the optimal settings for the genetic algorithm.
4.3.2
Method
31
Chapter 5
Results
5.1
Region Merging
The segmentation program outputs two result images. The first is the segmented
image, with each region being shown in the average colour of all its pixels. The
second is the region map, where each region is given a different value, and therefore
a different shade in the image. This is done to make the distinction between regions
clearer. If two adjacent regions have the same or very similar average values, the
fact that they werent merged (and probably should have been) wouldnt show up
in the segmented image. An example of the results are shown in Figure 5.1.
The results of the comparison of the region merging implementation with the region
growing and watershed algorithms is shown in Figure 5.2.
As can by seen, this implementation had rather average results. The watershed
method performed the best, and ran much faster than either of the other algorithms.
5.2
Genetic Algorithm
Two different fitness functions were implemented. Q(I), as defined previously, could
be subjectively seen to perform badly, appearing to produce worse segmentation
results than the original region merging algorithm. No further evaluation using this
fitness function was performed.
E(I) subjectively seemed to perform better than the original region merging algorithm. To confirm or refute this, further evaluation was performed. This evaluation
was objective, being deterministic and calculated by the computer.
The results obtained are shown below in Figure 5.3:
As can be seen, the genetic algorithm only showed an improvement over the original
region merging implementation for one of the images. Reasons for this could include
32
(a)
(b)
(c)
33
Watershed
Region
Merging
Region
Growing
House
5597
Hand
3095
Woman
4296
5076
3324.65
4959
4622
3558
3699
House
5076
4655
5195
4787
5249
4852
4947
Hand
3324
3490
3408
3435
3613
3414
3472
Woman
4959
6575
5437
5172
4938
5067
5437
Figure 5.3: This table shows the comparison of the RMSE values using genetic
algorithms vs. traditional algorithms
a poorly chosen fitness function, using a population size that is too small to offer
enough variation for the individuals to evolve or not running the genetic algorithm
for enough generations for the population to evolve sufficiently.
It may seem totally infeasible to find the ideal parameter combination for the
region merging algorithm. However, the fact that there are 1010 , different parameter
combinations is deceptive. From experimenting with varying the parameters, it
was found1 that the only parameter that made a significant different was the scale
parameter. Even the scale parameter only had any great effect on the segmentation
when altered by a large amount. A feasible solution would be to give each colour
channel an equal weighting, setting wRed, wGreen and wBlue all to 0.33; letting
compactness and smoothness have the same influence, setting wCompact to 0.5.
The effect of changing the scale parameter is shown below in Figure 5.4.
The effect of changing the scale parameter by 10 isnt that great. Suppose, to be
on the safe side, we allow the scale parameter to vary by increments of 5. This
means that there are only 20 different combinations of parameters to try - which
should be slightly faster than running 2 generations of a genetic algorithm with a
population size of 10.
With a bit of prior knowledge, one doesnt even have to try all of this limited range
of scale factor. If the various objects in the image appear distinct (such as the flowers
shown below), one can set the scale parameter high. This high scale parameter will
mean that more region merging occurs. This means that small colour variations,
such as shadows or reflections, within an object hopefully wont be recognised as
separate objects, while the separate objects are distinct enough from each other not
to be merged together. However, if one is looking to distinguish objects of similar
1
These are just general observations, and havent been formally tested
34
(a)
(b)
(c)
35
colour, such as the camouflaged crocodile, one needs to set the scale parameter
much lower. The negative aspect of doing this, however, is that it means that the
segmentation isnt unsupervised and does require user interaction.
A major problem in using this region merging- genetic algorithm system is that
it is very slow to run. When processing a 481 by 321 pixel image, using a genetic
algorithm with a population size of 10 and running 20 generations, the take taken is
about an hour2 . This makes getting results a rather tedious process, especially since
because of the random nature of genetic algorithms, one needs to take the average
performance over a few trials when evaluating the performance of the segmentation.
Marco Gallotta came up with a solution to this problem and ran this algorithm on
a grid.
5.2.1
Conclusion
It can be concluded that we are unable to reject the hypothesis that the use of
a genetic algorithm would improve the quality of the image segmentation. The
genetic algorithm could provide an improvement in segmentation quality, though
this hasnt been shown. Because of the time taken to run the genetic algorithm, its
full potential hasnt been explored.
Although there is a chance it could offer an improvement in segmentation quality,
the major drawback is that running the genetic algorithm is extremely slow. This
can be improved by running it on a grid, but not many people have access to a grid,
and so in most cases this isnt a feasible solution. Other options would be to limit
the range through which the parameters are allowed two vary or to provide a totally
different underlying image segmentation algorithm. An example of an algorithm that
could be used would by the watershed segmentation algorithm. In the experiments
done with Marco Gallottas implementation of this algorithm, it was found to run
far quicker than either the region growing or region merging methods.
5.3
It was not feasible to run these experiments due to the extremely long execution
time of the region merging segmentation with the genetic algorithm3 . Not only does
one have to wait a long time to get results for the performance of various parameter
settings, but it makes it totally infeasible to increase the maximum number of generations or increase the population size. This is unfortunate, as one predicts that the
segmentation performance would increase the as the number of generations increase
2
3
Running on a laptop with a Core Duo 1.8 GHz with 1gig RAM
It takes exactly an hour to run the algorithm for 20 generations, each with ten individuals
36
(because there is more time to evolve) and also increase with increasing population
size (a larger population provides larger variation, which makes finding an optimum
solution more likely).
37
Chapter 6
Future Work
Improvements on the system could include using different merging criteria (instead
of the fusion factor) or using a different colour space. As mentioned the fitness
function is very important to a genetic algorithm. Various fitness functions were
suggested, though only one was experimented with. These fitness functions, as
well as others could be experimented with. A fitness function is just a function,
so there is the possibility that genetic programming1 could be used to evolve a
fitness function. Manually segmented images could be used as ground truths and a
discrepancy measure could be used as a fitness function. If this training set of images
is a good representation of the type of images that one wishes to segment, this could
be successful. The evolved fitness function could be used in a program, such as the
one implemented in this paper to provide unsupervised image segmentation. The
main challenge of doing this would be to provide suitable terminal and function sets.
38
Chapter 7
Conclusions
We successfully implemented region merging segmentation method. We improved
on this by using a genetic algorithm to modify the segmentation parameters. In
the process of doing this, we discovered the importance of a well chosen fitness
function and a novel fitness function was defined. A major advantage of this fitness
function is that it requires no comparison with a manually segmented image. This
has a major advantage over many other fitness functions requiring such input, this
method useful to real-world, automated segmentation applications.
The effect of the genetic algorithm on the image segmentation results is inconclusive, its major drawback is that it is extremely slow to run. This means that the
small improvement in segmentation quality probably isnt worth the extra computation time. A way of getting around this drawback is to run the genetic algorithm
on a grid.
A major problem with the development of a general image segmentation method
is that different applications and different users require different results from image
segmentation. In reality, there is no ideal segmentation and the quality of the
segmentation depends on required use of the output.
39
Bibliography
[1] Galib: A c++ library of genetic algorithm components. WWW.
[2] P. D. Acton, L. S. Pilowsky, H. F. Kung, and P. J. Ell. Automatic segmentation of dynamic neuroreceptor single-photon emission tomography images using
fuzzy clustering. European Journal of Nuclear Medicine, 26(6):581590, June
1999.
[3] B. Ahrens. Genetic algorithm optimization of superresolution parameters. In
Proceedings of the 2005 conference on Genetic and evolutionary computation,
GECCO, pages 20832088, June 2005.
[4] A. Albiol, L. Torres, and E.J. Delp. An unsupervised color image segmentation
algorithm for facedetection applications. In Proceedings of 2001 International
Conference on Image Processing, volume 2, pages 681684, October 2001.
[5] M. Alfonseca. Genetic algorithms. In Proceedings of the international conference
on APL, pages 16, 1991.
[6] P. Andrey. Selectionist relaxation: Genetic algorithms applied to image segmentation. In Image and Vision Computing, volume 17, pages 175187, 1999.
[7] M. Aoyagi and K. Tsuji. A modified genetic algorithm for image segmentation
based on feature clustering. In International Symposium on Information Theory
and its Applications, 2004.
[8] M. Baatz and A. Schape.
Multiresolution segmentation - an optimization approach for high quality multi-scale image segmentation. In
Strobl/Blaschke/Griesebner (editors): Angewandte Geographische Infomationsverarbeitung XII, pages 1223.
[9] S.M. Bhandarkar and H. Zhang. Image segmentation using evolutionary computation. In IEEE Transactions on Evolutionary Computation, volume 3, pages
121, April 1999.
[10] B. Bhanu and S. Lee. Genetic Learning for Adaptive Image Segmentation.
Springer, 1994.
40
[11] B. Bhanu, S. Lee, and J. Ming. Adaptive image segmentation using a genetic
algorithm. In IEEE Transactions on Systems, Man and Cybernetics, volume 25,
pages 15431567, December 1995.
[12] B. Bhanu and J. Peng. Adaptive integrated image segmentation and object
recognition. In IEEE Transactions on Systems, Man and Cybernetics, Part C,
volume 30, pages 427441, November 2000.
[13] M. Borsotti, P. Campadelli, and R. Schettini. Quantitative evaluation of color
image segmentation results. Pattern Recogn. Lett., 19(8):741747, 1998.
[14] A. Bouman. Ee637 digital image processing i course notes. Purdue University
School of Electrical and Computer Engineering, Video and Image Processing
Program, January 2007.
[15] H. Chen, W. Chien, and S. Wang. Contrast-based color image segmentation.
IEEE Signal Processing Letters, 11(7):641644, July 2004.
[16] D.N. Chun and H.S. Yang. Robust image segmentation using genetic algorithm
with a fuzzy measure. In Pattern Recognition 29(7), pages 11951211, 1996.
[17] D. Comaniciu. Color image segmentation results. WWW, 2007.
[18] J. M. Daida, T. F. Bersano-Begey, and J. F. Vesechy. Computer-assisted design
of image classification algorithms: Dynamic and static fitness evaluations in
a scaffold genetic programming environment. In Genetic Programming 1996:
Proceedings of the First Annual Conference, July 1996.
[19] K. S. Deshmukh and G. N. Shinde. An adaptive color image segmentation.
Electronic Letters on Computer Vision and Image Analysis, 5(4):1223, 2005.
[20] G. Dong and M. Xie. Color clustering and learning for image segmentation
based on neural networks. In IEEE Transactions on Neural Networks, volume 16, pages 925936, July 2005.
[21] M.E. Farmer and D. Shugars. Application of genetic algorithms for wrapperbased image segmentation and classification. In IEEE Congress on Evolutionary
Computation, pages 13001307, July 2006.
[22] R.Q. Feitosa, G.A.O.P. Costa, and T.B. Cazes. A genetic approach for the
automatic adaptation of segmentation parameters. In OBIA06, 2006.
[23] S. Forrest. Genetic algorithms. ACM Computing Surveys, 28(1):7780, March
1996.
41
42
[37] S. Ouadfel and M. Batouche. MRF-based image segmentation using ant colony
system. Electronic Letters on Computer Vision and Image Analysis, 2(2):1224,
2003.
[38] H. Peng, F. Long, Z. Chi, and W. Su. A hierarchical distributed genetic algorithm for image segmentation. In Proceedings of the 2000 Congress on Evolutionary Computation, volume 1, pages 272276, 2000.
[39] G. Pignalberi, R. Cucchiara, L. Cinque, and S. Levialdi. Tuning range image segmentation by genetic algorithm. EURASIP Journal on Applied Signal
Processing, 8:780790, 2003.
[40] V. Ramos and F. Muge. Image colour segmentation by genetic algorithms. In
Proceedings of the 11th Portuguese Conference on Pattern Recognition, May
2000.
[41] S. J. Sangwine and R. E. N. Horne. The Colour Image Processing Handbook.
Springer, 1998.
[42] W. Skarbek and A. Koschan. Colour image segmentation - a survey, 1994.
[43] I. Stainvas and D. Lowe. A generative model for separating illumination and
reflectance from images. Journal of Machine Learning, 4:14991519, 2003.
[44] D. L. Swets, B. Punch, and J. Weng. Genetic algorithms for object recognition
in a complex scene. In Proceedings of the 1995 International Conference on
Image Processing (ICIP95), 1995.
[45] S.M. Thede. An introduction to genetic algorithms. In Journal of Computing
Sciences in Colleges, volume 20, pages 115123, October 2004.
[46] David Tschumperl. C++ template image processing library. WWW, 2000.
[47] W. Vanzella and V. Torre. A versatile segmentation procedure. In IEEE Transactions on Systems, Man and Cybernetics, Part B, volume 36, pages 366378,
April 2006.
[48] S. Vitabile, G. Pilato, and F. Sorbello. Road signs recognition using a dynamic
pixel aggregation technique in the hsv color space. In Proceedings of the 11th
Internation Conference on Image Analysis and Processing (ICIAP01), 2001.
[49] S. Wesolkowski, S. Tominaga, and R.D. Dony. Shading and highlight invariant
color image segmentation. SPIE, 4300:229240, 2001.
[50] Y.J. Zhang. A survey on evaluation methods for image segmentation. In Pattern
Recognition, volume 29, pages 13351346, 1996.
43
44