INVESTIGATING THE ROLE OF SEARCH-BASED ALGORITHMS

IN SOFTWARE TESTING
1.) SOURCE SELECTION:
Search string: Search based Algorithms AND Software Testing"

ACM Digital Library

Science direct

Number of Included Papers

# Papers After
After Applying Search Query Stage 1
(Exclusion
Based on Titles)
25
16
28
19

# Papers After
Stage 2
(Exclusion Based
on Abstract)
10
14

# Papers After
Stage 3
(Exclusion
Based on Text)
4
11

Total

53

24

15

Repository

35

SURVEY:
Sr.No

Paper
Reference

Description

Input
Data

Advantages

Disadva
ntages

Results/remarks

Symbolic
Search Based
Testing. [1]

In this paper an algorithm for

constructing fitness functions is
presented
that
improve
the
efficiency of search-based testing
when trying to generate branch
adequate test data. The algorithm
combines symbolic information
with dynamic analysis

Control
flow graph
of
the
program
under test
and Target
edge to be
covered.

It
does
not
require
any
change in the
underlying test
data generation
technique
and
avoids many
problems
traditionally
associated with
symbolic
execution, in
particular
the
presence of loops

Efficiency
of
algorithm
is
improved
but
effectiven
ess is not.

GA is able to cover a
branch
that
it
previously failed to
cover .Similarly, the
local search is able to
cover more branches
when
using
the
enhanced
fitness
functions

2.

Transition
Coverage
Testing for
Simulink or
State flow
Models using
genetic
algorithm. [2]

This paper presents a messy-GA

based framework for generating test
data for SL/SF models. It consists of
three major components; the
executable model generator, the
coverage goal generator and the test
case generator. The framework
creates an executable model from a
SL/SF model (executable
model generator), sets up a test plan

A SL/SF
(Simulink /
state- flow)
model

A
prior
knowledge of the
length of the
input sequence is
not required and
it can handle
Simulink blocks,
concurrency,
nested loops and
cyclic paths.

Messy-GA
outperforms other
traditional algorithms
with respect to the
full transition
coverage criterion.

Minimizing
Test Suites in
Software
Product Lines
Using Weightbased Genetic
Algorithms. [3]

Achievement
of minimized
combinatorial
test suite for
configurationaware software
functional
testing using
the Cuckoo
Search
algorithm. [4]
Adapting ant
colony
optimization to
generate test
data
for
software
Structural
testing. [5]

5.

An integrated
search-based
approach for
automatic
testing
from
extended finite
state machine
(EFSM)
models. [6]

based on a
coverage criterion (coverage goal
generator) then dynamically
generates test data by applying
messy-GA using the
feedback from the executable model
(test data generator)
Weight-based GAs are applied to
minimize the test suite for testing a
product at the same time achieving
high feature pairwise coverage and
fault detection capability in the
context of software product line.
The algorithms are: WeightBased
Genetic
Algorithm
(WBGA),
Weight-Based Genetic Algorithm
for Multi-objective Optimization
(WBGA-MO),
and
RandomWeighted
Genetic
Algorithm
(RWGA).The working theory of
these three weight-based GAs is to
assign a particular weight to each
objective function for converting the
multi-objective problem to a single
objective problem with a scalar
objective function
Cuckoo Search is used in the design
and implementation of a strategy to
construct optimized combinatorial
sets. The strategy consists of
different
algorithms
for
construction. These algorithms are
combined to serve the Cuckoo
Search.

In the paper, the basic ACO

algorithm is transformed to a
discrete version so as to handle the
test data generation problem in
structural testing. Some strategies
such as local transfer, global transfer
and pheromone update are used to
improve the coverage capability of
test suite.
The extended finite state machine
(EFSM) is a modelling approach
that has been used to represent a
wide range of systems. The
approach has two phases, the aim of
the first phase being to produce a
feasible Transition paths (TP) while
the second phase searches for an
input sequence to trigger this TP.

The
fault
detection
capability
and
testing coverage
of the original
test
suite
is
preserved.

---

The
program
under test,
and its input
variable list

---

The generated
results were
independent of
the value of the
tuning
Parameters.

---

---

The
practical
set
of
weights
obtained
by
empirical
evaluation
may not
be
applicable
to
all
product
line
testing.

---

---

---

RWGA has the best

performance among
the three weightbased GAs and RS.
The main reason is
that RWGA assigns
weights dynamically
during search.

Generated test suites

can detect faults
effectively

ACO is better than

SA and GA in
coverage capability,
convergence
speed
and stability

Experimental results
using five EFSMs
found the first phase
to be effective in
generating FTPs
With a success rate of
approximately
96.6%. Furthermore,
the proposed input

7.

8.

9.

A tabu search
algorithm for
structural
software
testing. [7]

Automatic test
data generation
based
on
reduced
adaptive
particle
Swarm
optimization
algorithm. [8]

Design
and
implementatio
n
of
a
harmonysearch-based
variablestrength t-way
testing strategy
with
constraints
support. [9]

The first phase uses a Genetic

Algorithm whose fitness function is
a TP feasibility metric based on
dataflow dependence. The second
phase uses a Genetic Algorithm
whose fitness function is based on a
combination of a branch distance
function and approach level.
This paper presents a tabu search
metaheuristic algorithm for the
automatic generation of structural
software tests. The developed test
generator has a cost function for
intensifying the search and another
for diversifying the search that is
used when the intensification is not
successful. It also combines the use
of memory with a backtracking
process to avoid getting stuck in
local minima.
This paper proposes a new approach
of reduced adaptive particle swarm
optimization for generating the test
data automatically. First, the
approach reduces the particle swarm
evolution equations and gets an
evolution equation without velocity.
Then, the approach makes an
adaptive adjustment scheme based
on inertia weight for the reduced
evolution equation. The approach
directly impacts on the particle
position, namely actual problem
solution. Next, according to the
particle fitness and the particle
aggregation degree, the population
will be divided into three parts and
inertia weight of each part will be
designed accordingly. This can
balance the search capabilities of
algorithm between global and local.
Finally, the approach is applied to
automatic test data generation.
Harmony search strategy (HSS) is a
composition of two main algorithms
(1) HSS interaction generation
algorithm for generating the
interaction tuples and
(2) HSS test suite generation for test
suite Construction

sequence generator
could trigger all the
generated
feasible
TPs (success rate =
100%).

Complex
programs
under test
and
large
ranges for
input
variables.

---

Developed generator
is both effective and
efficient.

---

Source code
of program.

---

---

HSS has the

potential
to
address
constrained
covering arrays
for
high
interaction
strength.

---

It
can
enhance
convergence speed of
algorithm and solve
the problems that
particle
swarm
algorithm easily falls
into the local optimal
solution and has low
search accuracy.
It can also improve
the efficiency of
generating test data
automatically.

HSS
gives
competitive results
against most existing
AI based (and pure
computational)
counterparts.

10.

Input-based
adaptive
randomized
test case
prioritization:
A local beam
search
approach. [10]

Local Beam Search techniques

effectively prune the search space
and evenly spread the test cases with
in the space rendered by the inputs
of the regression test suites.

A set of
unordered
test cases

11.

Search-based
automated
testing
of
continuous
controllers:
Framework,
tool support,
and case
studies.[11]

Signals

12.

Search based
constrained test
case selection
using
execution
effort. [12]

A set of requirements characterizing

the
behavior
of
continuous
controllers are identified, and a
search-based technique is developed
based on random search, adaptive
random search, hill climbing and
simulated annealing algorithms to
automatically identify worst-case
test scenarios which are utilized to
generate test cases for these
requirements.
Since large test suites usually
contain redundancies, it is possible
to reduce them in order to respect
time/people constraints without
severely compromising coverage. In
this paper a test case selection
problem is formulated as a
constrained search based
optimization task, using
requirements coverage as the fitness
function to be maximized and the
execution effort (time) of the
selected TCs as a constraint in the
search process. This work is based
on the Particle Swarm Optimization
algorithm.

13.

Search based
software
testing
of
object-oriented
containers.
[13]

This paper focuses on the

difficulties of testing container
classes with nature inspired search
algorithms. The Different search
algorithms are analyzed, and then
they are compared with traditional
techniques. A search space
reduction is used that exploits the
characteristics of the containers.
Without this reduction, the use of
search algorithms would have
required too much computational
time.

This techniques is
significantly
more
efficient than the
Genetic and Greedy,
but are less efficient
than ART.

This
approach
generate
significantly
better test cases
faster
than
random test case
generation

Generated test cases

uncover
discrepancies
between environment
models and the real
world when they are
applied
at
the
Hardware-in-theLoop level.

Input test
suits

---

---

---

---

---

A more complex suite

required the use of
more powerful search
techniques in order to
achieve satisfactory
results in statistical
terms.
For
the
simpler
suite,
a
greater variety of
algorithms
were
found
to
be
statistically
equivalent in the test
case selection task
and, in this case,
simpler
algorithms
would
be
more
indicated
Nature
inspired
search
algorithms
seem to give better
results
than
the
traditional techniques

14.

15.

Test
data
generation with
a
Kalman
filter-based
adaptive
genetic
algorithm.[14]

A
novel
strategy
for
automatic test
data generation
using
soft
computing
technique. [15]

In this paper, a new approach is

introduced for generating test data,
based on adaptive optimization. The
method uses a Kalman filter to
reduce the effect of the stochastic
behavior of GAs when estimating
the appropriate parameter values to
use in each iteration of the
optimization process. KFGA was
implemented in EvoSuite, a frame
work for whole test suite generation.
A set of experiments were
conducted
using
open-source
libraries and programs with different
characteristics, such as number of
methods, number of branches and
number of testing goals. The
performance of the KFGA was
compared to the performance of a
standard GA with pre-tuned
parameter setting and a GA with
probability matching parameter
control.
GA and PSO are combined for
automatic generation of test data.
Two population sets P1 and P2 are
taken and then genetic operators and
PSO calculation operations are
applied separately. Computation of
fitness function values is carried out
on population P1 and particles are
sorted based upon their fitness
function values in population P1
and best fitness particles are added
to population P2 at ratio. PSO
operations are applied to population
P2 and updated fitness function of
particles in population P2 is
computed. The particle transferred
to P2 and removed from P1 and new
random particles are added to P1.
The average execution time of GAPSO was found more than PSO but
lesser than GA.

---

---

---

The adaptive method

Outperformed the
standard GA and the
probability matching
method over the
majority of the
problem instances

The
proposed
strategy
usually
performs better than
the other traditional
algorithms.

---

---

---

REFERENCES:
[1] Arthur Baars, Mark Harman, Youssef Hassoun, Kiran Lakhotia, Phil McMinn, Paolo Tonella, Tanja Vos, Symbolic Search-Based Testing,in:
IEEE/ACM International Conference on Automated Software Engineering (ASE), 2011, pp. 53-62.
[2] Jungsup Oh, Mark Harman, Shin Yoo, Transition Coverage Testing for Simulink or State flow Models using genetic algorithm , in: Genetic
and Evolutionary Computation Conference (GECCO),2011.
[3] Shuai Wang, Shaukat Ali, Arnaud Gotlieb, Minimizing Test Suites in Software Product Lines Using Weight-based Genetic Algorithms,in: Genetic and
Evolutionary Computation Conference (GECCO),2013.
[4] Bestoun S. Ahmed, Taib Sh. Abdulsamad, Moayad Y. Potrus, Achievement of minimized combinatorial test suite for configuration-aware software
functional testing using the Cuckoo Search algorithm, in: Information and Software Technology, vol. 66, 2015, pp. 13-29.

[5] Chengying Mao, Lichuan Xiao, Xinxin Yu, Jinfu Chen, Adapting ant colony optimization to generate test data for software Structural testing ,
in: Swarm and Evolutionary Computation, vol. 20, 2015, pp. 23-36.
[6] Abdul Salam Kalaji , Robert Mark Hierons , Stephen Swift , An integrated search-based approach for automatic testing from extended finite state
machine (EFSM) models, in: Information and Software Technology, vol. 53, 2011, pp. 1297-1318.
[7] Eugenia Daz, Javier Tuya, Raquel Blanco, Jos Javier Dolado, A tabu search algorithm for structural software testing, in: Computers & Operations
Research, vol. 35, 2008, pp. 3052 3072.
[8] Shujuan Jiang, JiaojiaoShin, Yanmei Zhang, Han Han Automatic test data generation based on reduced adaptive particle Swarm optimization
algorithm, in: Neurocomputing, vol. 158, 2015, pp. 109-116.
[9] Abdul Rahman A. Alsewari, Kamal Z. Zamli, Design and implementation of a harmony-search-based variable-strength t-way testing strategy with
constraints support, in: Information and Software Technology, vol. 54, 2012, pp. 553568
[10] BoJianga, W.K.Chan, Input-based adaptive randomized test case prioritization: A local beam search approach, in: The Journal of Systems and
Software, vol. 105, 2015, pp. 91106.
[11] Reza Matinnejad, Shiva Nejati, Lionel Briand, Thomas Bruckmann, Claude Poull, Search-based automated testing of continuous controllers:
Framework, tool support, and case studies, in: Information and Software Technology, vol. 57, 2015 pp. 705722.
[12] Luciano S. deSouza , Ricardo B.C.Prudncio , Flavia deA.Barros, Eduardo H.daS.Aranha, Search based constrained test case selection using
execution effort, in: Expert Systems with Applications, vol. 40, 2013, pp. 48874896.
[13] Andrea Arcuri, Xin Yao, Search based software testing of object-oriented containers, in: Information Sciences, vol. 178, 2008, pp. 30753095.
[14] Aldeida Aletia, Lars Grunskeb, Test data generation with a Kalman filter-based adaptive genetic algorithm, in: The Journal of Systems and Software,
vol. 103, 2015, pp. 343352.
[15] Priyanka Chawla, Inderveer Chana, Ajay Rana, A novel strategy for automatic test data generation using soft computing technique, in: Front. Comput.
Sci., vol. 9, 2015, pp. 346363.