Abstract
Boundary value testing in the white-box setting tests relational expressions with boundary values. These relational expressions are often a part of larger conditional expressions or decisions. It is therefore important, for effective testing that the outcome of a relational expression independently influences the outcome of the expression or decision in which it is embedded. Extending MC/DC to boundary value testing was proposed in the literature as a technique to achieve this independence. Based on this idea, in this paper we formally define a new coverage criterion - masking boundary value coverage (MBVC). MBVC is an adaptation of masking of conditions to boundary value testing. Mutation based analysis is used to show that test data satisfying MBVC is more effective in detecting relational mutants than test data satisfying BVC.
In this paper, we give a formal argument justifying why test data for MBVC is more effective compared to that for BVC in detecting relational mutants. We performed an experiment to evaluate effectiveness and efficiency of MBVC test data relative to that for BVC, in detecting relational mutants. Firstly, mutation adequacy of the test set for MBVC was higher than that for BVC in 56% of cases, and never lower. Secondly, the test data for MBVC killed 80.7% of the total number of mutants generated, whereas the test data for BVC killed only 70.3% of them. A further refined analysis revealed that some mutants are such that they cannot be killed. We selected a small set of mutants randomly to get an estimate of percentage of such mutants. Then the extrapolated mutation adequacies were 92.75% and 80.8% respectively. We summarize the effect of masking on efficiency. Details of the experiment, tools developed for automation and analysis of the results are also provided in this paper.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Do-178b: Software considerations in airborne systems and equipment certification (1982)
Andrews, J.H., Briand, L.C., Labiche, Y., Namin, A.S.: Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Trans. Softw. Eng. 32(8), 608–624 (2006)
Bokil, P., Darke, P., Shrotri, U., Venkatesh, R.: Automatic test data generation for c programs. Secure System Integration and Reliability Improvement, 359–368 (2009)
Chilenski, J.J.: An investigation of three forms of the modified condition decision coverage (mcdc) criterion. Technical Report DOT/FAA/AR-01/18, Office of Aviation Research (2001)
Clarke, E., Kroening, D., Lerda, F.: A tool for checking ANSI-C programs. In: Jensen, K., Podelski, A. (eds.) TACAS 2004. LNCS, vol. 2988, pp. 168–176. Springer, Heidelberg (2004)
De Millo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: Help for the practicing programmer. Computer 11(4), 34–41 (1978)
Gupta, A., Jalote, P.: An approach for experimentally evaluating effectiveness and efficiency of coverage criteria for software testing. Int. J. Softw. Tools Technol. Transf. 10(2), 145–160 (2008)
Howden, W.E.: Weak mutation testing and completeness of test sets. IEEE Trans. Softw. Eng. 8(4), 371–379 (1982)
Hayhurst Kelly, J., Veerhusen Dan, S., Chilenski John, J., Rierson Leanna, K.: A practical tutorial on modified condition/decision coverage. Technical Report NAS 1.15:210876 (2001)
Myers, G.J.: The Art of Software Testing. Wiley, New York (1979)
Namin, A.S., Andrews, J.H.: The influence of size and coverage on test suite effectiveness. In: ISSTA 2009: Proceedings of the eighteenth international symposium on Software testing and analysis, pp. 57–68. ACM, New York (2009)
Jefferson Offutt, A., Untch, R.H.: Mutation 2000: uniting the orthogonal, pp. 34–44 (2001)
Perry, W.: Effective methods for software testing. Wiley-QED Publishing, Somerset (1995)
Rapps, S., Weyuker, E.J.: Selecting software test data using data flow information. IEEE Trans. Softw. Eng. 11(4), 367–375 (1985)
Reid, S.C.: An empirical analysis of equivalence partitioning, boundary value analysis and random testing. In: METRICS 1997: Proceedings of the 4th International Symposium on Software Metrics, p. 64. IEEE Computer Society Press, Washington (1997)
Namin, A.S., Andrews, J.H., Murdoch, D.J.: Sufficient mutation operators for measuring test effectiveness. In: ICSE 2008: Proceedings of the 30th international conference on Software engineering, pp. 351–360. ACM, New York (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Suman, P.V., Muske, T., Bokil, P., Shrotri, U., Venkatesh, R. (2010). Masking Boundary Value Coverage: Effectiveness and Efficiency. In: Bottaci, L., Fraser, G. (eds) Testing – Practice and Research Techniques. TAIC PART 2010. Lecture Notes in Computer Science, vol 6303. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15585-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-15585-7_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15584-0
Online ISBN: 978-3-642-15585-7
eBook Packages: Computer ScienceComputer Science (R0)