Abstract
The aim has been to minimize regression test suites while retaining fault detection capability of the test suite admissible. An appropriate minimized test suite should exercise different execution paths within a program. However, minimization of test suites may result in significant fault detection loss. To alleviate the loss, a new bi-criteria heuristic algorithm, using cluster analysis of test cases execution profiles is proposed in this paper. Cluster analysis of execution profiles categorizes test cases according to their similarity in terms of exercising a certain coverage criterion. Considering additional coverage criteria the proposed algorithm samples some test cases from each cluster. These additional criteria exercise execution paths, different from those covered by the main testing criteria. Experiments on the Siemens suite manifest the applicability of the proposed approach and present interesting insights into the use of cluster analysis to the bi-criteria test suite reduction.
Chapter PDF
Similar content being viewed by others
Keywords
References
Rothermel, G., Harrold, M.J., von Ronne, J., Hong, C.: Empirical Studies of Test-Suite Reduction. Journal of Software Testing, Verification, and Reliability 12(4), 219–249 (2002)
Rothermel, G., Harrold, M.J., Ostrin, J., Hong, C.: An Empirical Study of the Effects of Minimization on the Fault Detection Capabilities of Test Suites. In: Proceedings of the International Conference on Software Maintenance. IEEE Computer Society (1998)
Harrold, M.J., Gupta, R., Soffa, M.L.: A methodology for Controlling the Size of a Test Suite. ACM Transactions on Software Engineering and Methodology 2, 270–285 (1993)
Chen, T.Y., Lau, M.F.: Heuristics Toward the Optimization of the Size of a Test Suite. In: Proc. of the 3rd International Conference on Software Quality Management, vol. 2, pp. 415–424. Seville, Spain (1995)
Jones, J.A., Harrold, M.J.: Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage. IEEE Transactions on Software Engineering 29, 195–209 (2003)
McMaster, S., Memon, A.: Call-Stack Coverage for GUI Test Suite Reduction. IEEE Transactions on Software Engineering 34, 99–115 (2008)
Tallam, S., Gupta, N.: A Concept Analysis Inspired Greedy Algorithm for Test Suite Minimization. In: Proc. of the 6th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering. ACM, Lisbon (2005)
Leon, D., Podgurski, A.: A Comparison of Coverage-Based and Distribution-Based Techniques for Filtering and Prioritizing Test Cases. In: Proc. of the 14th International Symposium on Software Reliability Engineering. IEEE Computer Society (2003)
Chen, Z., Xu, B., Zhang, X., Nie, C.: A Novel Approach for Test Suite Reduction Based on Requirement Relation Contraction. In: Proc. of the 2008 ACM Symposium on Applied Computing. ACM, Fortaleza (2008)
Jeffrey, D., Gupta, N.: Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction. IEEE Transactions on Software Engineering 33, 108–123 (2007)
Marick, B.: The Craft of Software Testing: Subsystem Testing. Prentice Hall, Englewood Cliffs (1995)
Leon, D., Podgurski, A., White, L.J.: Multivariate Visualization in Observation-based Testing. In: Proc. of the 22nd International Conference on Software Engineering. ACM, Limerick (2000)
Dickinson, W., Leon, D., Podgurski, A.: Finding Failures by Cluster Analysis of Execution Profiles. In: Proc. of the 23rd International Conference on Software Engineering. IEEE Computer Society, Toronto (2001)
Dickinson, W., Leon, D., Podgurski, A.: Pursuing Failure: the Distribution of Program Failures in a Profile Space. In: Proceedings of the 8th European Software Engineering Conference Held Jointly with 9th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, Vienna (2001)
Zhong, H., Zhang, L., Mei, H.: An Experimental Study of Four Typical Test Suite Reduction Techniques. Information and Software Technology 50, 534–546 (2008)
Chen, T.Y., Lau, M.: A New Heuristic for Test Suite Reduction. Information and Software Technology 40(5-6) (1998)
Wong, W.E., Horgan, J.R., London, S., Mathur, A.P.: Effect of Test Set Minimization on Fault Detection Effectiveness. Software-Practice and Experience 28, 347–369 (1998)
Mansour, N., El-Fakih, K.: Simulated Annealing and Genetic Algorithms for Optimal Regression Testing. Journal of Software Maintenance 11, 19–34 (1999)
Black, J., Melachrinoudis, E., Kaeli, D.: Bi-Criteria Models for All-Uses Test Suite Reduction. In: Proc. of the 26th International Conference on Software Engineering. IEEE Computer Society (2004)
Harrold, M.J., Rothermel, G., Wu, R., Yi, L.: An Empirical Investigation of Program Spectra. In: Proc. of the 1998 ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering. ACM, Montreal (1998)
Rothermel, G., Elbaum, S., Kinneer, A., Do, H.: Software-artifact Infrastructure Repository, http://www.cse.unl.edu/galileo/sir
Witten, H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Kaufmann series in Data Management Systems (2005)
SAS 9.1.3 Documentation, SAS/GRAPH 9.1 Reference, http://support.sas.com/documentation/onlinedoc/91pdf/index_913.html
Yang, Y., Guan, X., You, J.: CLOPE: a Fast and Effective Clustering Algorithm for Transactional Data. In: Proc. of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, Edmonton (2002)
Horgan, J.R., London, S.A.: ATAC: A Data Flow Coverage Testing Tool for C. In: Proc. Symp. Assessment of Quality Software Development Tools, pp. 2–10 (1992)
Freund, J.E.: Mathematical Statistics, 5th edn. Prentice-Hall (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 IFIP International Federation for Information Processing
About this paper
Cite this paper
Khalilian, A., Parsa, S. (2012). Bi-criteria Test Suite Reduction by Cluster Analysis of Execution Profiles. In: Szmuc, T., Szpyrka, M., Zendulka, J. (eds) Advances in Software Engineering Techniques. CEE-SET 2009. Lecture Notes in Computer Science, vol 7054. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28038-2_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-28038-2_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28037-5
Online ISBN: 978-3-642-28038-2
eBook Packages: Computer ScienceComputer Science (R0)