Skip to main content
SpringerLink
Log in

Enhancing Efficiency of the Test Case Prioritization Technique by Improving the Rate of Fault Detection

  • Research Article - Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

An Erratum to this article was published on 20 April 2017

This article has been updated

Abstract

Test case prioritization techniques organize test cases for implementation in a manner that enhance their efficacy in accordance with some performance goal. The main aim of regression testing is to test the amended software to assure that the amendments performed in software are correct. It is not always feasible to retest entire test cases in a test suite due to limited resources. Therefore, it is necessary to develop some effective techniques that can enhance the regression testing effectiveness by organizing the test cases in an order following some testing criterion. One possible criterion of such prioritization is to enhance a test suite’s fault detection rate. It aspires to arrange test cases in an order that higher priority test cases run earlier than lower ones. This paper proposed a methodology for prioritizing regression test cases based on four factors namely the rate of fault detection, the number of faults detected, the test case ability of risk detection and the test case effectiveness. The proposed approach is implemented on two projects. The resultant test case order is analyzed with other prioritization techniques such as no prioritization, random prioritization, reverse prioritization, optimal prioritization and along with previous works for project 1. We have applied our proposed approach for prioritizing test cases in an order that maximize fault coverage with least test suite execution and compared its effectiveness with other orderings. The result of proposed approach shows higher average percentage of fault detected value and outperforms all other approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Change history

  • 20 April 2017

    An erratum to this article has been published.

References

  1. Myers, G.J.: The Art of Software Testing. Wiley, New York (1979)

    Google Scholar 

  2. Pressman, R.S.: Software Engineering: A Practitioner’s Approach, 5th edn. McGraw-Hill International edition, New York (2001)

    MATH  Google Scholar 

  3. I.S.G. of Software Engg. Terminology. IEEE standards collection, IEEE std 610.12-1990 (1990)

  4. Rothermel, G.; Untch, R.; Chu, C.; Harrold, M.: Test case prioritization: An empirical study. In: Proceedings of IEEE international conference on Software Maintenance (ICSM’ 1999), pp 179–188 (1999)

  5. Elbaum, S.; Malishevsky, A.; Rothermel, G.: Prioritizing test cases for regression testing. In: Proceedings of the 2000 ACM SIGSOFT International Symposium on Software Testing and Analysis, Portland, Oregon, USA, pp 102–112 (2000)

  6. Marre, M.; Bertolino, A.: Using spanning sets for coverage testing. IEEE Trans. Softw. Eng. 29(11), 974–984 (2003)

    Article  Google Scholar 

  7. Wong, W.E.; Horgan, J.R.; London, S.; Mathur, A.P.: Effect of test set minimization on fault detection effectiveness. Softw. Pract. Exp. 28(4), 347–369 (1998)

    Article  Google Scholar 

  8. Wong, W.E.; Horgan, J.R.; Mathur, A.P.; Pasquini, A.: Test set size minimization and fault detection effectiveness: A case study in a space application. In: Proceedings of 21st Annual International Computer Software Application Conference, pp. 522–528 (1997)

  9. Rotherrmel, G.; Harrold, M.J.; Ostrin, J.; Hong, C.: An emprical study of the effects of minimization on the fault detection capabilities of test suites. In: Proceedings of the International Conference on Software Maintenance, pp. 34–43 (1998)

  10. Ball, T.: On the limit of control flow analysis for regression test selection. In: Proceedings of ACM International Symposium on Software Testing and Analysis, pp. 134–142 (1998)

  11. Chen, Y.F.; Rosenblum, D.S.; Vo, K.P.: TestTube: A system for selective regression testing. In: Proceedings of 16th International Conference of Software Engineering, pp. 211–222 (1994)

  12. Rothermel, G.; Harrold, M.J.: A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Methodol. 6(2), 173–210 (1997)

    Article  Google Scholar 

  13. Vokolos, F.I.; Frankl, P.G.: Pythia: A regression test selection tool based on textual differencing. In: Proceedings of Third International Conference of Reliability, Quality and Safety of Software Intensive Systems (ENCRESS 1997) (1997)

  14. Rothermel, G.; Harrold, M.J.: Analyzing regression test selection techniques. IEEE Trans. Softw. Eng. 22(8), 529–551 (1996)

    Article  Google Scholar 

  15. Kavitha, R.; Sureshkumar, N.: Test case prioritization for regression testing based on severity of fault. Int. J. Comput. Sci. Eng. (IJCSE) 2(5), 1462–1466 (2010)

    Google Scholar 

  16. Chernak, Y.: Validating and improving test case effectiveness. IEEE Softw. 16(1), 81–86 (2001)

    Article  Google Scholar 

  17. Tyagi, M.; Malhotra, S.: An approach for test case prioritization based on three factors. Int. J. Inf. Technol. Comput. Sci. 4, 79–86 (2015)

    Google Scholar 

  18. Rothermel, G.; Untch, R.; Chu, C.; Harrold, M.: Prioritizing test cases for regression testing. IEEE Trans. Softw. Eng. 27(10), 929–948 (2001)

    Article  Google Scholar 

  19. Do, H.; Elbaum, S.G.; Rothermel, G.: Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empir. Softw. Eng. 10(4), 405–435 (2005)

    Article  Google Scholar 

  20. Beizer, B.: Software System Testing and Quality Assurance. Van Nostrand, New York (1984)

    Google Scholar 

  21. Elbaum, S.; Malishevsky, A.G.; Rothermel, G.: Test case prioritization: a family of empirical studies. IEEE Trans. Softw. Eng. 28(2), 159–182 (2002)

    Article  Google Scholar 

  22. Hamlet, R.G.: Probable correctness theory. Inf. Process. Lett. 25, 17–25 (1987)

    Article  MathSciNet  Google Scholar 

  23. Hamlet, R.G.: Testing programs with the aid of a compiler. IEEE Trans. Softw. Eng. 3(4), 279–290 (1997)

    MATH  Google Scholar 

  24. Avritzer, A.; Weyuker, E.J.: The automatic generation of load test suites and the assessment of the resulting software. IEEE Trans. Softw. Eng. 21(9), 705–716 (1995)

    Article  Google Scholar 

  25. Malishevsky, A.; Ruthruff, J.R.; Rothermel, G.; Elbaum, S.: Cost-cognizant test case prioritization. Technical report TR-UNL-CSE-2006-004, Dept. of CSE, University of Nebraska-Lincoln, USA (2006)

  26. Dickinson, W.; Leon, d.; Podgurski, A.: Finding failures by cluster analysis of execution profiles. In: International Conference on Software Engineering, Toronto, Canada, pp. 339–348 (2001)

  27. Wong, W.E.; Horgan, J.R.; London, S.; Agrawal, H.: A study of effective regression testing in practice. In: Proceedings of Eighth International Symposium on Software Reliability Engineering, pp. 230–238 (1997)

  28. Harrold, M.J.; Gupta, R.; Soffa, M.L.: A methodology for controlling the size of a test suite. ACM Trans. Softw. Eng. Methodol. 2(3), 270–285 (1993)

    Article  Google Scholar 

  29. Jeffrey, D.; Gupta, R.: Test suite reduction with selective redundancy. In: 21st IEEE Conference on Software Maintenance (ICSM’05), pp. 549–558 (2005)

  30. Ren, X.; Saha, F.; Tip, F.; Ryder, B.G.; Chesley, O.: Chianti: A Tool for Change Impact Analysis of Java Programs. OOPSLA’04, Vancouver, Canada, ACM Sigplan, pp. 432-448 (2004)

  31. Elbaum, S.; Rothermel, G.; Kanduri, S.: Selecting cost-effective test case prioritization technique. Softw. Qual. J. 12(3), 185–210 (2004)

    Article  Google Scholar 

  32. Kim, J.; Porter, A.: A history based test prioritization technique for regression testing in resource constraint environments. In: ICSE’02, ACM, Orlando, U.S.A., pp. 119–129 (2002)

  33. Walcott, K.R.; Soffa, M.L.; Kapfhammer, G.M.; Roos, R.S.: Time-aware test suite prioritization. In: The Proceedings of the ACM SIGSOFT/SIGPLAN International Symposium on Software Testing and Analysis, Portland, Maine (2006)

  34. Jeffrey, D.; Gupta, R.: Test case prioritization using relevant slices. In: COMPSAC’06, IEEE International Conference, Chicago, pp. 411–420 (2006)

  35. Srikanth, H.; Williams, L.; Osborne, J.: Towards the prioritization of system test cases. North Carolina State University TR-2005-44 (2005)

  36. Aggarwal, K.K.; Singh, Y.; Kaur, A.; Malhotra, R.: Software reuse metrics for object-oriented systems. In: Proceedings of the Third ACIS Interanational on Conference On Software Engineering Research, Management and Applications (SERA ‘05), pp. 48–55 (2005)

  37. Li, Z.; Harman, M.; Hierons, R.M.: Search algorithms for regression test case prioritization. IEEE Trans. Softw. Eng. 33(4), 225–237 (2007)

    Article  Google Scholar 

  38. Masri, W.; Podgurski, A.; Leon, D.: An empirical study of test case filtering techniques based on exercising information flows. IEEE Trans. Softw. Eng. 33(7), 454–477 (2007)

    Article  Google Scholar 

  39. Qu, B.; Nie, C.; Xu, B.; Zhang, X.: Test case prioritization for black box testing. In: Proceedings of Computer Software and Applications Conference, pp. 465–474 (2007)

  40. Korel, B.; Koutsogiannakis, G.; Tahat, L.H.: Model-based test prioritization heuristic methods and their evaluation. In: Proceeding A-MOST’07 Proceedings of the 3rd International Workshop on Advances in Model-based Testing, ACM, NY, USA, pp. 34–43 (2007)

  41. Kavitha, R.; Kavitha, V.R.; Suresh, N.: Requirement Based Test Case Prioritization. In: IEEE ICCCCT (2010)

  42. Maheswari, R.; Mala, D.: A novel approach for test case prioritization. In: IEEE International Conference on Computational Intelligence and Computing Research (2013)

  43. Kayes, I.: Test case prioritization for regression testing based on fault dependency. In: IEEE (2011)

  44. Singh, Y.; Kaur, A.; Suri, B.: Test case prioritization using ant colony optimization. ACM SIGSOFT Softw. Eng. Notes 35(4), 1–7 (2010)

    Article  Google Scholar 

  45. Hla, K.H.S.; Choi, Y.; Park, J.S.: Applying particle swarm optimization to prioritizing test cases for embedded real time software retesting. In: Proceedings of the IEEE 8th International Conference on Computer and Information Technology Workshops, pp. 527–532 (2008)

  46. Sabharwal, S.; Sibal, R.C.; Sharma, C.: A genetic algorithm based approach for prioritization of test case scenarios in static testing. In: Proceedings of the 2nd International Conference on computer and Communication Technology, IEEE Xplore Press, Allahabad, pp. 304–309 (2011)

  47. Ray, M.; Mohapatra, D.P.: Multi-Objective Test Prioritization via a Genetic Algorithm. Innovations in System Software Engineering. Springer, New York (2014)

    Google Scholar 

  48. Mala, D.J.; Kamalapriya, M.; Shobana, R.; Mohan, V.: A non-pheromone based intelligent swarm optimization technique in software test suite optimization. IEEE (2009)

  49. Huang, Y.C.; Huang, C.Y.; Chang, J.R.; Chen, T.Y.: Design and analysis of cost cognizant test case prioritization using genetic algorithm with test history. In: 34th Annual Computer Software and Applications Conference, IEEE (2010)

  50. Souza, L.S.; Prudencio, R.B.C.; Barros, F.D.A.: A constrained particle swarm optimization approach for test case selection. In: Proceedings of the 22nd International Conference on Software Engineering and Knowledge Engineering (SEKE 2010), Redwood City, CA, USA (2010)

  51. Sherriff, M.; Lake, M.; Williams, L.: Prioritization of regression tests using singular value decomposition with empirical change records. In: 18th IEEE International Symposium on Software Reliability Engineering (2007)

  52. Alsmadi, I.; Alda, S.: Test case reduction and selection optimization in testing web services. Int. J. Inf. Eng. Electron. Bus. MECS Publisher (2012)

  53. Muthusamy, T.; Seetharaman, K.: A test case prioritization method with weight factors in regression testing based on measurement metrics. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 3(12) (2013)

  54. Kumar, S.; Singh, S.: Test case prioritization: Various techniques—A review. Int. J. Sci. Eng. Res. 4(4) (2013)

  55. Jenifer, I.; Ilavarasi, A.K.: A review on test case prioritization techniques. IJARCSSE, 4(3) (2014)

  56. Elbaum, S.; Malishevsky, A.; Rothermel, G.: Incorporating varying test costs and fault severities into test case prioritization. In: ICSE ’01: Proceedings of the 23rd International Conference on Software Engineering. IEEE Computer Society, Washington, pp 329–338 (2001)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology (ISM), Dhanbad, 826004, India

    Soumen Nayak, Chiranjeev Kumar & Sachin Tripathi

Authors
  1. Soumen Nayak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chiranjeev Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sachin Tripathi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Soumen Nayak.

Additional information

The original version of this article was revised: At the end of Sect. 4 the algorithm after the sentence “The proposed TCP technique is presented in the form of an algorithm as follows:” had been missing.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nayak, S., Kumar, C. & Tripathi, S. Enhancing Efficiency of the Test Case Prioritization Technique by Improving the Rate of Fault Detection. Arab J Sci Eng 42, 3307–3323 (2017). https://doi.org/10.1007/s13369-017-2466-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-017-2466-6

Keywords

Search

Navigation