Advertisement

Arabian Journal for Science and Engineering

, Volume 42, Issue 8, pp 3307–3323 | Cite as

Enhancing Efficiency of the Test Case Prioritization Technique by Improving the Rate of Fault Detection

  • Soumen NayakEmail author
  • Chiranjeev Kumar
  • Sachin Tripathi
Research Article - Computer Engineering and Computer Science

Abstract

Test case prioritization techniques organize test cases for implementation in a manner that enhance their efficacy in accordance with some performance goal. The main aim of regression testing is to test the amended software to assure that the amendments performed in software are correct. It is not always feasible to retest entire test cases in a test suite due to limited resources. Therefore, it is necessary to develop some effective techniques that can enhance the regression testing effectiveness by organizing the test cases in an order following some testing criterion. One possible criterion of such prioritization is to enhance a test suite’s fault detection rate. It aspires to arrange test cases in an order that higher priority test cases run earlier than lower ones. This paper proposed a methodology for prioritizing regression test cases based on four factors namely the rate of fault detection, the number of faults detected, the test case ability of risk detection and the test case effectiveness. The proposed approach is implemented on two projects. The resultant test case order is analyzed with other prioritization techniques such as no prioritization, random prioritization, reverse prioritization, optimal prioritization and along with previous works for project 1. We have applied our proposed approach for prioritizing test cases in an order that maximize fault coverage with least test suite execution and compared its effectiveness with other orderings. The result of proposed approach shows higher average percentage of fault detected value and outperforms all other approaches.

Keywords

Software testing Regression testing Test case prioritization Average percentage of fault detected metric Severity of faults 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Myers, G.J.: The Art of Software Testing. Wiley, New York (1979)Google Scholar
  2. 2.
    Pressman, R.S.: Software Engineering: A Practitioner’s Approach, 5th edn. McGraw-Hill International edition, New York (2001)zbMATHGoogle Scholar
  3. 3.
    I.S.G. of Software Engg. Terminology. IEEE standards collection, IEEE std 610.12-1990 (1990)Google Scholar
  4. 4.
    Rothermel, G.; Untch, R.; Chu, C.; Harrold, M.: Test case prioritization: An empirical study. In: Proceedings of IEEE international conference on Software Maintenance (ICSM’ 1999), pp 179–188 (1999)Google Scholar
  5. 5.
    Elbaum, S.; Malishevsky, A.; Rothermel, G.: Prioritizing test cases for regression testing. In: Proceedings of the 2000 ACM SIGSOFT International Symposium on Software Testing and Analysis, Portland, Oregon, USA, pp 102–112 (2000)Google Scholar
  6. 6.
    Marre, M.; Bertolino, A.: Using spanning sets for coverage testing. IEEE Trans. Softw. Eng. 29(11), 974–984 (2003)CrossRefGoogle Scholar
  7. 7.
    Wong, W.E.; Horgan, J.R.; London, S.; Mathur, A.P.: Effect of test set minimization on fault detection effectiveness. Softw. Pract. Exp. 28(4), 347–369 (1998)CrossRefGoogle Scholar
  8. 8.
    Wong, W.E.; Horgan, J.R.; Mathur, A.P.; Pasquini, A.: Test set size minimization and fault detection effectiveness: A case study in a space application. In: Proceedings of 21st Annual International Computer Software Application Conference, pp. 522–528 (1997)Google Scholar
  9. 9.
    Rotherrmel, G.; Harrold, M.J.; Ostrin, J.; Hong, C.: An emprical study of the effects of minimization on the fault detection capabilities of test suites. In: Proceedings of the International Conference on Software Maintenance, pp. 34–43 (1998)Google Scholar
  10. 10.
    Ball, T.: On the limit of control flow analysis for regression test selection. In: Proceedings of ACM International Symposium on Software Testing and Analysis, pp. 134–142 (1998)Google Scholar
  11. 11.
    Chen, Y.F.; Rosenblum, D.S.; Vo, K.P.: TestTube: A system for selective regression testing. In: Proceedings of 16th International Conference of Software Engineering, pp. 211–222 (1994)Google Scholar
  12. 12.
    Rothermel, G.; Harrold, M.J.: A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Methodol. 6(2), 173–210 (1997)CrossRefGoogle Scholar
  13. 13.
    Vokolos, F.I.; Frankl, P.G.: Pythia: A regression test selection tool based on textual differencing. In: Proceedings of Third International Conference of Reliability, Quality and Safety of Software Intensive Systems (ENCRESS 1997) (1997)Google Scholar
  14. 14.
    Rothermel, G.; Harrold, M.J.: Analyzing regression test selection techniques. IEEE Trans. Softw. Eng. 22(8), 529–551 (1996)CrossRefGoogle Scholar
  15. 15.
    Kavitha, R.; Sureshkumar, N.: Test case prioritization for regression testing based on severity of fault. Int. J. Comput. Sci. Eng. (IJCSE) 2(5), 1462–1466 (2010)Google Scholar
  16. 16.
    Chernak, Y.: Validating and improving test case effectiveness. IEEE Softw. 16(1), 81–86 (2001)CrossRefGoogle Scholar
  17. 17.
    Tyagi, M.; Malhotra, S.: An approach for test case prioritization based on three factors. Int. J. Inf. Technol. Comput. Sci. 4, 79–86 (2015)Google Scholar
  18. 18.
    Rothermel, G.; Untch, R.; Chu, C.; Harrold, M.: Prioritizing test cases for regression testing. IEEE Trans. Softw. Eng. 27(10), 929–948 (2001)CrossRefGoogle Scholar
  19. 19.
    Do, H.; Elbaum, S.G.; Rothermel, G.: Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empir. Softw. Eng. 10(4), 405–435 (2005)CrossRefGoogle Scholar
  20. 20.
    Beizer, B.: Software System Testing and Quality Assurance. Van Nostrand, New York (1984)Google Scholar
  21. 21.
    Elbaum, S.; Malishevsky, A.G.; Rothermel, G.: Test case prioritization: a family of empirical studies. IEEE Trans. Softw. Eng. 28(2), 159–182 (2002)CrossRefGoogle Scholar
  22. 22.
    Hamlet, R.G.: Probable correctness theory. Inf. Process. Lett. 25, 17–25 (1987)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Hamlet, R.G.: Testing programs with the aid of a compiler. IEEE Trans. Softw. Eng. 3(4), 279–290 (1997)zbMATHGoogle Scholar
  24. 24.
    Avritzer, A.; Weyuker, E.J.: The automatic generation of load test suites and the assessment of the resulting software. IEEE Trans. Softw. Eng. 21(9), 705–716 (1995)CrossRefGoogle Scholar
  25. 25.
    Malishevsky, A.; Ruthruff, J.R.; Rothermel, G.; Elbaum, S.: Cost-cognizant test case prioritization. Technical report TR-UNL-CSE-2006-004, Dept. of CSE, University of Nebraska-Lincoln, USA (2006)Google Scholar
  26. 26.
    Dickinson, W.; Leon, d.; Podgurski, A.: Finding failures by cluster analysis of execution profiles. In: International Conference on Software Engineering, Toronto, Canada, pp. 339–348 (2001)Google Scholar
  27. 27.
    Wong, W.E.; Horgan, J.R.; London, S.; Agrawal, H.: A study of effective regression testing in practice. In: Proceedings of Eighth International Symposium on Software Reliability Engineering, pp. 230–238 (1997)Google Scholar
  28. 28.
    Harrold, M.J.; Gupta, R.; Soffa, M.L.: A methodology for controlling the size of a test suite. ACM Trans. Softw. Eng. Methodol. 2(3), 270–285 (1993)CrossRefGoogle Scholar
  29. 29.
    Jeffrey, D.; Gupta, R.: Test suite reduction with selective redundancy. In: 21st IEEE Conference on Software Maintenance (ICSM’05), pp. 549–558 (2005)Google Scholar
  30. 30.
    Ren, X.; Saha, F.; Tip, F.; Ryder, B.G.; Chesley, O.: Chianti: A Tool for Change Impact Analysis of Java Programs. OOPSLA’04, Vancouver, Canada, ACM Sigplan, pp. 432-448 (2004)Google Scholar
  31. 31.
    Elbaum, S.; Rothermel, G.; Kanduri, S.: Selecting cost-effective test case prioritization technique. Softw. Qual. J. 12(3), 185–210 (2004)CrossRefGoogle Scholar
  32. 32.
    Kim, J.; Porter, A.: A history based test prioritization technique for regression testing in resource constraint environments. In: ICSE’02, ACM, Orlando, U.S.A., pp. 119–129 (2002)Google Scholar
  33. 33.
    Walcott, K.R.; Soffa, M.L.; Kapfhammer, G.M.; Roos, R.S.: Time-aware test suite prioritization. In: The Proceedings of the ACM SIGSOFT/SIGPLAN International Symposium on Software Testing and Analysis, Portland, Maine (2006)Google Scholar
  34. 34.
    Jeffrey, D.; Gupta, R.: Test case prioritization using relevant slices. In: COMPSAC’06, IEEE International Conference, Chicago, pp. 411–420 (2006)Google Scholar
  35. 35.
    Srikanth, H.; Williams, L.; Osborne, J.: Towards the prioritization of system test cases. North Carolina State University TR-2005-44 (2005)Google Scholar
  36. 36.
    Aggarwal, K.K.; Singh, Y.; Kaur, A.; Malhotra, R.: Software reuse metrics for object-oriented systems. In: Proceedings of the Third ACIS Interanational on Conference On Software Engineering Research, Management and Applications (SERA ‘05), pp. 48–55 (2005)Google Scholar
  37. 37.
    Li, Z.; Harman, M.; Hierons, R.M.: Search algorithms for regression test case prioritization. IEEE Trans. Softw. Eng. 33(4), 225–237 (2007)CrossRefGoogle Scholar
  38. 38.
    Masri, W.; Podgurski, A.; Leon, D.: An empirical study of test case filtering techniques based on exercising information flows. IEEE Trans. Softw. Eng. 33(7), 454–477 (2007)CrossRefGoogle Scholar
  39. 39.
    Qu, B.; Nie, C.; Xu, B.; Zhang, X.: Test case prioritization for black box testing. In: Proceedings of Computer Software and Applications Conference, pp. 465–474 (2007)Google Scholar
  40. 40.
    Korel, B.; Koutsogiannakis, G.; Tahat, L.H.: Model-based test prioritization heuristic methods and their evaluation. In: Proceeding A-MOST’07 Proceedings of the 3rd International Workshop on Advances in Model-based Testing, ACM, NY, USA, pp. 34–43 (2007)Google Scholar
  41. 41.
    Kavitha, R.; Kavitha, V.R.; Suresh, N.: Requirement Based Test Case Prioritization. In: IEEE ICCCCT (2010)Google Scholar
  42. 42.
    Maheswari, R.; Mala, D.: A novel approach for test case prioritization. In: IEEE International Conference on Computational Intelligence and Computing Research (2013)Google Scholar
  43. 43.
    Kayes, I.: Test case prioritization for regression testing based on fault dependency. In: IEEE (2011)Google Scholar
  44. 44.
    Singh, Y.; Kaur, A.; Suri, B.: Test case prioritization using ant colony optimization. ACM SIGSOFT Softw. Eng. Notes 35(4), 1–7 (2010)CrossRefGoogle Scholar
  45. 45.
    Hla, K.H.S.; Choi, Y.; Park, J.S.: Applying particle swarm optimization to prioritizing test cases for embedded real time software retesting. In: Proceedings of the IEEE 8th International Conference on Computer and Information Technology Workshops, pp. 527–532 (2008)Google Scholar
  46. 46.
    Sabharwal, S.; Sibal, R.C.; Sharma, C.: A genetic algorithm based approach for prioritization of test case scenarios in static testing. In: Proceedings of the 2nd International Conference on computer and Communication Technology, IEEE Xplore Press, Allahabad, pp. 304–309 (2011)Google Scholar
  47. 47.
    Ray, M.; Mohapatra, D.P.: Multi-Objective Test Prioritization via a Genetic Algorithm. Innovations in System Software Engineering. Springer, New York (2014)Google Scholar
  48. 48.
    Mala, D.J.; Kamalapriya, M.; Shobana, R.; Mohan, V.: A non-pheromone based intelligent swarm optimization technique in software test suite optimization. IEEE (2009)Google Scholar
  49. 49.
    Huang, Y.C.; Huang, C.Y.; Chang, J.R.; Chen, T.Y.: Design and analysis of cost cognizant test case prioritization using genetic algorithm with test history. In: 34th Annual Computer Software and Applications Conference, IEEE (2010)Google Scholar
  50. 50.
    Souza, L.S.; Prudencio, R.B.C.; Barros, F.D.A.: A constrained particle swarm optimization approach for test case selection. In: Proceedings of the 22nd International Conference on Software Engineering and Knowledge Engineering (SEKE 2010), Redwood City, CA, USA (2010)Google Scholar
  51. 51.
    Sherriff, M.; Lake, M.; Williams, L.: Prioritization of regression tests using singular value decomposition with empirical change records. In: 18th IEEE International Symposium on Software Reliability Engineering (2007)Google Scholar
  52. 52.
    Alsmadi, I.; Alda, S.: Test case reduction and selection optimization in testing web services. Int. J. Inf. Eng. Electron. Bus. MECS Publisher (2012)Google Scholar
  53. 53.
    Muthusamy, T.; Seetharaman, K.: A test case prioritization method with weight factors in regression testing based on measurement metrics. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 3(12) (2013)Google Scholar
  54. 54.
    Kumar, S.; Singh, S.: Test case prioritization: Various techniques—A review. Int. J. Sci. Eng. Res. 4(4) (2013)Google Scholar
  55. 55.
    Jenifer, I.; Ilavarasi, A.K.: A review on test case prioritization techniques. IJARCSSE, 4(3) (2014)Google Scholar
  56. 56.
    Elbaum, S.; Malishevsky, A.; Rothermel, G.: Incorporating varying test costs and fault severities into test case prioritization. In: ICSE ’01: Proceedings of the 23rd International Conference on Software Engineering. IEEE Computer Society, Washington, pp 329–338 (2001)Google Scholar

Copyright information

© King Fahd University of Petroleum & Minerals 2017

Authors and Affiliations

  • Soumen Nayak
    • 1
    Email author
  • Chiranjeev Kumar
    • 1
  • Sachin Tripathi
    • 1
  1. 1.Department of Computer Science and EngineeringIndian Institute of Technology (ISM)DhanbadIndia

Personalised recommendations