All Rights Reserved 2014 IJARTES Visit: www.ijartes.org Page 30
Higher Order Mutant Generation to Decrease the Cost of Mutation Testing Sudhir Kumar Mohapatra 1 , Manoranjan Pradhan 2 ______________________________________________ Abstract: Although powerful, mutation could be a computationally terribly high-priced testing technique. In fact, its 3 main stages (mutant generation, mutant execution and result analysis) need several resources to be with success accomplished. Thus, researchers have created necessary efforts to scale back its prices. This paper represents a further effort during this sense. It describes the results of 2 experiments within which, by suggests that of mixing the initial set of mutants and so getting a replacement set of mutantseach one with 2 faultsthe range of mutants used is reduced to 0.5. Results cause believes that mutant combination doesn't decrease the standard of the test suite, whereas it supposes necessary savings in mutant execution and result analysis. _______________________________________________
Keyword: Mutant, First order mutant, higher order mutant, Mutant generation, Mutant cost reduction ________________________________________________
Introduction: Mutation is a testing technique that usually involves the following three stages [2], [3], [4]: (1) Mutant generation, whose goal is the generation of mutants of the program under test. (2) Mutant execution, whose goal is the execution of test cases against the original program and the mutants. (3) Result analysis, whose goal is to check the mutation score obtained by the test suite. A mutant M of a program beneath test P may be a copy of P, however it contains a little modification in its code, that is taken as a fault. the thought of mutation testing is to jot down test cases detection those faults: simplifying, the a lot of the faults found by the test cases, the higher the standard of the test suite. _______________________________________________ First Authors Name: Sudhir Kumar Mohapatra, Dept of Computer science & engg. GITA, Bhubaneswar, Odisha. Second Authors Name: Manoranjan Pradhan,Dept of Computer science & engg. GITA, Bhubaneswar, Odisha ____________________________________________________________
Mutant generation is typically meted out with machine- controlled tools that apply a group of mutation operators to the sentences of the first program, therefore manufacturing a high range of mutants[9], [10], [11]. for instance, an easy program with simply a sentence like come a+b (where a, b ar integers) is also a minimum of mutated into twenty other ways (ab,a +b,a/b,a+b++,a+b,a+b,0+b,a+0, |a|+b,a+|b|, etc.). this instance is illustrated in Figure one. Figure one shows the ASCII text file of the first program and of some mutants; Figure one presents the results obtained from execution some test cases on the various program versions. The test suit resembling the test information (1,1)produces completely different outputs on the first program (whose output is correct) and on Mutant 1;thus, this test suit has found the fault introduced within the mutant, and it's aforesaid that the mutant is killed. On the opposite hand, as all test cases supply a similar output on the first program and on Mutant four, it's aforesaid that Mutant four is alive. Moreover, this mutant can ne'er be killed by any test suit, as variable b is incremented when returning the result. Mutants like this ar referred to as functionally equivalent mutants and should be thought-about as noise within the result analysis step, as they represent obstacles in knowing the standard of the test suite: they need a fault (actually, a syntactical change) with regard to the first ASCII text file that can't be found tho'. the standard of a take a look at suite is measured in terms of the mutation score, that is that the share of non-equivalent mutants killed. A take a look at suite is mutation-adequate if its mutation score is 100 percent. As shown higher than, so as to visualize whether or not a test suite is mutation-adequate, the detection of equivalent mutants is needed [13], [14], [15]. However, this can be a really expensive task, as a result of equivalent mutants should be searched (usually by hand) among the large range of mutants engineered up by the generation tool applied. for instance, in a very additional study from 1996, Offutt et al. [12] reduced the quantity of purposeful operators of the Mothra mutation tool from twenty two to five , therefore considerably reducing the quantity of the test suite has additionally a powerful dependence on the mutation operators applied [4]. The 3 same steps of mutation testing ar sometimes terribly expensive , and researchers have created vital efforts to cut back its prices. this text describes a way to decrease the prices of mutation testing. the thought relies on the reduction within the range of mutants by combining mutant pairs generated by a tool into new mutants, every with the 2 faults introduced within the mutants they are available from. International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access Online Journal) Volume1, Issue2, Sept
All Rights Reserved 2014 IJARTES Visit: www.ijartes.
Fig 1. Source code and their mutants
Proposed methods 1. Decreasing the cost of mutation by mutant combination: A substantive a part of the benefits of mutation testing resides within the coupling impact, by that a take a look at information set that detects all straightforward faults during a program is thus sensitive that it conjointly detects a lot of advanced faults. In fact, mutant generation tools introduce one straightforward fault in every mutant. Given a group of mutants containing one straightforward fa to use identical mutant generation tool to every of those mutants, so obtaining a replacement set of mutants, wherever each has 2 faults (the initial continuing from the primary mutant, and also the second, introduced within the second generation). In testing literature, mutants containing one straightforward fault area unit referred to as first mutants and mutants containing 2 straightforward faults area unit referred to as second-order mutants. As the expansion of mutants range with every new generation is exponential: ifn first-order mutants (M1 . . .Mn) area unit obtained from P, then n second-order mutants are obtained from every Mi . As the range of equivalent mutants generated by mutation tools is comparatively, our plan cons range by combining pairs of first-order mutants to get a group of second-order mutants, each with 2 faults. At a primary look, the result's terribly completely different from that shown in Figure three, because the range of mutants obtained is near 0.5 the dimensions of the first suite .As proportion of equivalent mutants is not up to the percentage of non-equivalent mutants, every equivalent mutant are in all probability combined with a non mutant.When a first-order mutant Mi is combined with another first-order mutant Mj to supply a second mutant Mi, j , the ensuing mutant is also equivalent or non International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access Online Journal) Volume1, Issue2, Sept-Oct, 2014.ISSN:2349-7173(Online) All Rights Reserved 2014 IJARTES Visit: www.ijartes.org
Source code and their mutants Proposed methods: Decreasing the cost of mutation by mutant A substantive a part of the benefits of mutation testing resides within the coupling impact, by that a take a look at straightforward faults during a program is thus sensitive that it conjointly detects a lot of advanced faults. In fact, mutant generation tools introduce one straightforward fault in every mutant. Given a group of mutants containing one straightforward fault, it's potential to use identical mutant generation tool to every of those mutants, so obtaining a replacement set of mutants, wherever each has 2 faults (the initial continuing from the primary mutant, and also the second, introduced within the generation). In testing literature, mutants containing one straightforward fault area unit referred to as first-order mutants and mutants containing 2 straightforward faults order mutants. As the ith every new generation is order mutants (M1 . . .Mn) area unit order mutants are obtained As the range of equivalent mutants generated by mutation tools is comparatively, our plan consists in reducing its order mutants to get a order mutants, each with 2 faults. At a primary look, the result's terribly completely different from that shown in Figure three, because the range of mutants ained is near 0.5 the dimensions of the first suite .As the of equivalent mutants is not up to the percentage equivalent mutants, every equivalent mutant are in all probability combined with a non-equivalent order mutant Mi is combined with order mutant Mj to supply a second-order mutant Mi, j , the ensuing mutant is also equivalent or non- equivalent, reckoning on the equivalence of Mi and Mj . The potential cases area unit summarized in fourth case of Table I could be a special case: normally, the mix of 2 first-order non-equivalent mutants can turn out one second-order non-equivalent mutant; but, it's potential that the ensuing mutant is equivalent, as within the example of Table II, wherever mutations area unit highlighted with . However, in our experiment, this terribly inconceivable case has not appeared. With these assumptions, and setting aside the same inconceivable case of Table II, if a take a look at suite T is mutation order mutants M, T also will be mutation wherever M is that the set of second continuing from the mix of mutants in M. Fig 2 No of mutant Generated
Fig 3 Higher order mutant generation order mutant
2. Decreasing the cost of mutation by higher order mutant Mutants is classified into 2 types: 1st Order Mutants (FOMs) higher Order Mutants (HOMs). FOMs are generated by applying mutation operators one time. HOMs are generated by applying mutation operators over once. This paper introduces the thought of subsuming HOMs. A subsuming HOMs is more durable to kill than the FOMs from that it\'s made. As such, it interchange the FOMs with the onl paper introduces the thought of a powerfully subsuming HOMs. A subsuming HOMs is simply killed by a set of the International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access 7173(Online) Page 31 equivalent, reckoning on the equivalence of Mi and Mj . The potential cases area unit summarized in Table I.The fourth case of Table I could be a special case: normally, the equivalent mutants can turn out one equivalent mutant; but, it's potential that the ensuing mutant is equivalent, as within the example of Table II, wherever mutations area unit highlighted with . However, in our experiment, this terribly inconceivable case has not appeared. With these assumptions, and setting aside the same inconceivable case of Table II, if a take a ion-adequate for a group of first- order mutants M, T also will be mutation-adequate for M wherever M is that the set of second-order mutants continuing from the mix of mutants in M.
No of mutant Generated
Higher order mutant generation by combining 1 st
order mutant Decreasing the cost of mutation by higher Mutants is classified into 2 types: 1st Order Mutants (FOMs) higher Order Mutants (HOMs). FOMs are generated by applying mutation operators one time. HOMs generated by applying mutation operators over once. introduces the thought of subsuming HOMs. A subsuming HOMs is more durable to kill than the FOMs 's made. As such, it\'s going to be preferred to interchange the FOMs with the only HOM. especially, the introduces the thought of a powerfully subsuming HOMs. A subsuming HOMs is simply killed by a set of the International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access Online Journal) Volume1, Issue2, Sept-Oct, 2014.ISSN:2349-7173(Online)
All Rights Reserved 2014 IJARTES Visit: www.ijartes.org Page 32
intersection of test cases that kill every FOM from that it\'s made. Consider a subsuming, h, made from the FOMs f1... fn. The set of test cases that kill h additionally kill every and each FOM f1... fn. Therefore, h will replace all of the mutants f1... fn while not loss of take a look at effectiveness. The converse doesn\'t hold; there exist take a look at sets that kill all FOMs f1... fn however that fail to kill h. The FOMs cannot, even taken together, replace the HOM while not attainable loss of test effort. {this is|this is often|this is} the sense during which h can be same to strongly subsume f1... fn. HOMs is classified in terms of the manner that they\'re coupled and subsuming, as shown in Figure one. In Figure one, the region space within the central diagram represents the domain of all HOMs. The sub-diagrams close the central region illustrate every class. For sake of simplicity of exposition these examples illustrate the second order mutant case; one that assumes that there ar 2 FOMs f1 and f2, and h denotes the HOM made from the FOMs f1 and f2. the 2 regions represented by every sub diagram represent the test sets containing all the take a look at cases that kill FOMs f1 and f2. The shaded space represents the test set that contains all take a look at cases that kill HOM h. The areas of the regions indicate the proportion of the domain of HOMs for every class. Following the coupling result hypothesis, if a test set that kills the FOMs additionally contains cases that kill the HOM, we have a tendency to shall say that the HOM may be a coupled HOM, otherwise we have a tendency to shall say it\'s a de-coupled HOM
Subsuming HOMs, by definition, is more durable to kill than their constituentFOMs. Therefore, in Figure one, the subsuming HOMs is described as those wherever the shaded space is smaller than the world of the union of the 2 unshaded regions, like sub-diagrams (a), (b) and (c). against this, (d), (e) and (f) ar non-subsuming. what is more, the subsuming HOMs is classified into powerfully subsuming HOMs and weak subsuming HOMs. By definition, if a legal action kills a powerfully subsuming HOM, it guarantees that its constituent FOMs ar killed furthermore. 2.1. Reasons to support hom 1) Cost: Work on Mutant Sampling and Selective Mutation has shown however the amount of mutants are often reduced with solely a little impact on check effectiveness [1], [8], [7]. 2) Uncertainty: Work on reducing the impact of equivalent mutants has reduced, tho' not eradicated, this drawback [6], [5], [4], [1]. 3) Realism: Empirical proof has been on condition that the faults denoted by mutants do, indeed, overlap with a category of real faults [12], [5].
2.2. Proposed work In order to explain Higher Order Mutation Testing we take the following example. Original Program { if ( (a>b) && (a>c))
} We create a first order mutant of the Original Program by adding a single fault i.e. we change a>b to a<b and name it FOM1 as below: FOM1 { if ( (a<b) && (a>c))
} We create another first order mutant of the Original Program by adding a single fault i.e. we change a>c to a<c and name it FOM2 as below: FOM2 { if ( (a>b) && (a<c))
} FOM1 and FOM2 are first order mutants as they vary by a single fault from the Original program. Now we create a HOM from FOM1 and FOM2 by having more than one fault from the Original program .Our HOM differs from original program by 2 faults. We change a>b to a<b and also a>c to a<c. HOM { if ((a<b) && (a<c))
} Now we prove that if we are able to find a subsuming HOM in particular a strongly subsuming HOM, it will kill all the FOMs from which it is constructed thereby reducing the number of test cases without loss of test case effectiveness .We take simple example to find largest of 3 numbers a, b and c. Our program takes as input 3 numbers and as output it gives the largest of these numbers.
International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access Online Journal) Volume1, Issue2, Sept-Oct, 2014.ISSN:2349-7173(Online)
All Rights Reserved 2014 IJARTES Visit: www.ijartes.org Page 33
Original program
#include<stdio.h> #include<conio.h> void main () { int a,b,c; clrscr(); printf(ENTER THE 3 NUMBERS); scanf(%d%d%d,&a,&b,&c); if((a>b) &&(a>c)) printf(A IS GREATEST); else if ((b>a) &&(b>c)) printf(B IS GREATEST); else if ((c>a) && (c>b)) printf(C IS GREATEST); else printf ( WRONG RESULT); }
We create First Order Mutant FOM1 from the Original_program by changing a> b to a<b FOM1 if((a<b) &&(a>c)) printf(A IS GREATEST); else if ((b>a) &&(b>c)) printf(B IS GREATEST); else if ((c>a) && (c>b)) printf(C IS GREATEST); else printf( WRONG RESULT);
We create another First Order Mutant FOM2 from the Original_program by changing a> c to a<c FOM2 if((a>b) &&(a<c)) printf(A IS GREATEST); else if ((b>a) &&(b>c)) printf(B IS GREATEST); else if ((c>a) && (c>b)) printf(C IS GREATEST); else printf( WRONG RESULT);
We create Higher order mutant HOM from First order mutant FOM1 and FOM2 by changing a> b to a<b and a>c to a<c. This differs from Original_program by 2 faults. HOM if((a<b) &&(a<c)) printf(A IS GREATEST); else if ((b>a) &&(b>c)) printf(B IS GREATEST); else if ((c>a) && (c>b)) printf(C IS GREATEST); else printf( WRONG RESULT);
TABLE 1: TEST CASE WHICH KILL FOM1, FOM2 & HOM
From the Table-1 we find that there are 2 test cases which kill FOM1 and also 2 test cases which kill FOM2. There is one test case which kills HOM and is found in the intersection of FOM1 and FOM2. This test case kills both FOM1 and FOM2. The converse is not true. The test case ((a<b) && (a>c)) which kills FOM1 does not kill FOM2 and HOM. Similarly the test case ((a>b) && (a<c)) which kills FOM2 does not kill FOM1 and HOM. From the Table- 2 its clear that the test case a<b && a>c which kills FOM1 and the Original_program does not kill FOM2 and HOM, similarly the test case a>b && a<c which kills FOM2 and the Original_program does not kill FOM1 and HOM but the test case a>b &&a>c found in the intersection of FOM1 and FOM2 kills our HOM and also FOM1 and FOM2. So if use this test case automatically both FOM1 and FOM2 will get killed thereby reducing the number of test cases without leading to loss of effectiveness.
International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access Online Journal) Volume1, Issue2, Sept
All Rights Reserved 2014 IJARTES Visit: www.ijartes.
TABLE 2:TEST DATA WITH FOM1, FOM2 & HOM
Conclusion: In this paper, we have a tendency to examine the utilities of upper Order Mutation Testing by making mutants first initial and higher Order. From the higher than work we have a tendency to conclude that although HOMs area unit tough to make however if we have a tendency to area unit able to notice a HOM than the amount of test cases scale backs that ultimately reduce the cost of testing. References [1] R. A. DeMillo, D. S. Guindi, K. N. King, W. M. McCracken, and A. J. Offutt. An Extended Overview of the Mothra Software Testing Environment. In Proceedings of the 2nd Workshop on Software Testing, Verification, and Analysis (TVA88), pages 142 Canada, July 1988. IEEE Computer society. [2] T. A. Budd. Mutation Analysis of Program Test Data. Phd thesis, Yale University, New Haven, Connecticut, 1980. [3] W. E.Wong. On Mutation and Data Flow. Phd thesis, Purdue University, West Lafayette, Indiana, 1993. [4] A. P. Mathur and W. E. Wong. An Empirical Comparison of Mutation and Data Flow Based Test Adequacy Criteria. Technique repo University, West Lafayette, Indiana, 1993. [5] A. S. Namin and J. H. Andrews. On Sufficiency of Mutants. In Proceedings of the 29th International Conference on Software Engineering (ICSE COMPANION07), pages 7374, Minneapolis, Minnesota, 20 May 2007. [6] A. P. Mathur. Performance, Effectiveness, and Reliability Issues in Software Testing. In Proceedings of the 5th International Computer Software and Applications Conference (COMPSAC79), pages 604 Tokyo, Japan, 11-13 September 1991. International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access Online Journal) Volume1, Issue2, Sept-Oct, 2014.ISSN:2349-7173(Online) All Rights Reserved 2014 IJARTES Visit: www.ijartes.org TABLE 2:TEST DATA WITH FOM1, FOM2 & HOM
examine the utilities of upper Order Mutation Testing by making mutants first initial and higher Order. From the higher than work we have a tendency to conclude that although HOMs area unit tough to make however if we have a tendency to area unit notice a HOM than the amount of test cases scale backs that ultimately reduce the cost of testing. R. A. DeMillo, D. S. Guindi, K. N. King, W. M. McCracken, and A. J. Offutt. An Extended Overview of the Mothra Software Testing In Proceedings of the 2nd Workshop on Software Testing, Verification, and Analysis (TVA88), pages 142151, Banff Alberta, [2] T. A. Budd. Mutation Analysis of Program Test Data. Phd thesis, Yale [3] W. E.Wong. On Mutation and Data Flow. Phd thesis, Purdue [4] A. P. Mathur and W. E. Wong. An Empirical Comparison of Mutation and Data Flow Based Test Adequacy Criteria. Technique report, Purdue [5] A. S. Namin and J. H. Andrews. On Sufficiency of Mutants. In Proceedings of the 29th International Conference on Software Engineering 74, Minneapolis, Minnesota, 20-26 [6] A. P. Mathur. Performance, Effectiveness, and Reliability Issues in Software Testing. In Proceedings of the 5th International Computer Software and Applications Conference (COMPSAC79), pages 604605, [7] M. Sahinoglu and E. H. Spafford. A Bayes Sequential Statistical Procedure for Approving Software Products. In Proceedings of the IFIP Conference on Approving Software Products (ASP90), pages 43 Garmis Partenkirchen, Germany, September 1990. Elsevier [8] R. A. DeMillo, D. S. Guindi, K. N. King, W. M. McCracken, and A. J. Offutt. An Extended Overview of the Mothra Software Testing Environment. In Proceedings of the 2nd Workshop on Software Testing, Verification, and Analysis (TVA88), pages 142 Canada, July 1988. IEEE Computer society. [9] A. J. Offutt, G. Rothermel, and C. Zapf. An Experimental Evaluation of Selective Mutation. InProceedings of the 15th International Conference on Software Engineering (ICSE93), pages 100 May 1993. IEEE Computer Society Press. [10] W. E. Wong and A. P. Mathur. Reducing the Cost of Mutation Testing: An Empirical Study. Journal of Systems and Software, 31(3):185 196, December 1995. [11] K. N. King and A. J. Offutt. A Fortran Language System for Mutation- Based Software Testing Software: Practice and Experience, 21(7):685718, October 1991 [12] E. S. Mresa and L. Bottaci. Efficiency of Mutation Operators and Selective Mutation Strategies: An Empirical Study. Software Testing, Verification and Reliability, 9(4):205 [13] A. S. Namin and J. H. Andrews. Finding Sufficient Mutation Operators via Variable Reduction. In Proceedings of the 2nd Workshop on Mutation Analysis (MUTATION06), page 5, Raleigh, North Carolina, November 2006. IEEE Computer Society. [14] A. S. Namin and J. H. Andrews. On Sufficiency of Mutants. In Proceedings of the 29th International Conference on Software Engineering (ICSE COMPANION07), pages 7 May 2007. [15] A. J. Offutt, A. Lee, G. Rothermel, R. H. Untch, and C. Zapf. An Experimental Determination of Sufficient Mutant Operators. ACM Transactions on Software Engineering and Methodology, 5(2):99 April 1996. Author Profile Sudhir kumar Mohapatra is an Assistant Professor in the Department of Computer Science& Engineering, Gandhi Institute for Technological Advancement (GITA), Bhubaneswar, Odisha ,India. national seminars and international journals. His includes software engineering and soft computing
Manoranjan Pradhan holds a Ph. D Degree in Science. He is presently working as a the Department of Computer Gandhi Institute for Bhubaneswar, Odisha, India. experience. He has published many journals. His research interests include Computer Se Detection, Soft Computing & Cloud computing
International Journal of Advanced Research in Technology, Engineering and Science (A Bimonthly Open Access 7173(Online) Page 34 ] M. Sahinoglu and E. H. Spafford. A Bayes Sequential Statistical Procedure for Approving Software Products. In Proceedings of the IFIP Conference on Approving Software Products (ASP90), pages 4356 Garmis Partenkirchen, Germany, September 1990. Elsevier Science. [8] R. A. DeMillo, D. S. Guindi, K. N. King, W. M. McCracken, and A. J. Offutt. An Extended Overview of the Mothra Software Testing Environment. In Proceedings of the 2nd Workshop on Software Testing, Verification, and Analysis (TVA88), pages 142151, Banff Alberta, Canada, July 1988. IEEE Computer society. [9] A. J. Offutt, G. Rothermel, and C. Zapf. An Experimental Evaluation of Selective Mutation. InProceedings of the 15th International Conference on Software Engineering (ICSE93), pages 100107, Baltimore, Maryland, May 1993. IEEE Computer Society Press. [10] W. E. Wong and A. P. Mathur. Reducing the Cost of Mutation Testing: An Empirical Study. Journal of Systems and Software, 31(3):185 [11] K. N. King and A. J. Offutt. A Fortran Language System for Based Software Testing Software: Practice and Experience, [12] E. S. Mresa and L. Bottaci. Efficiency of Mutation Operators and gies: An Empirical Study. Software Testing, Verification and Reliability, 9(4):205232, December 1999. [13] A. S. Namin and J. H. Andrews. Finding Sufficient Mutation Operators via Variable Reduction. In Proceedings of the 2nd Workshop on s (MUTATION06), page 5, Raleigh, North Carolina, November 2006. IEEE Computer Society. [14] A. S. Namin and J. H. Andrews. On Sufficiency of Mutants. In Proceedings of the 29th International Conference on Software Engineering (ICSE COMPANION07), pages 7374, Minneapolis, Minnesota, 20-26 [15] A. J. Offutt, A. Lee, G. Rothermel, R. H. Untch, and C. Zapf. An Experimental Determination of Sufficient Mutant Operators. ACM Transactions on Software Engineering and Methodology, 5(2):99118, udhir kumar Mohapatra is an Assistant Professor in the Department of Computer Science& Engineering, Gandhi Institute for Technological Advancement (GITA), Bhubaneswar, Odisha ,India. He published many papers in national seminars and international journals. His research interest software engineering and soft computing. Pradhan holds a Ph. D Degree in Computer Science. He is presently working as a professor and Head of Department of Computer Science & Engineering, Gandhi Institute for Technological Advancement (GITA), Bhubaneswar, Odisha, India. He has 15 years of teaching experience. He has published many papers in national and international include Computer Security, Intrusion Cloud computing.