Provably Optimal and Human-Competitive Results in SBSE for Spectrum Based Fault Localisation

  • Xiaoyuan Xie
  • Fei-Ching Kuo
  • Tsong Yueh Chen
  • Shin Yoo
  • Mark Harman
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8084)

Abstract

Fault localisation uses so-called risk evaluation formulæ to guide the localisation process. For more than a decade, the design and improvement of these formulæ has been conducted entirely manually through iterative publication in the fault localisation literature. However, recently we demonstrated that SBSE could be used to automatically design such formulæ by recasting this as a problem for Genetic Programming(GP). In this paper we prove that our GP has produced four previously unknown globally optimal formulæ. Though other human competitive results have previously been reported in the SBSE literature, this is the first SBSE result, in any application domain, for which human competitiveness has been formally proved. We also show that some of these formulæ exhibit counter-intuitive characteristics, making them less likely to have been found solely by further human effort.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abreu, R., Zoeteweij, P., van Gemund, A.J.C.: An evaluation of similarity coefficients for software fault localization. In: Proceedings of the 12th Pacific Rim International Symposium on Dependable Computing, pp. 39–46. Riverside, USA (2006)Google Scholar
  2. 2.
    Abreu, R., Zoeteweij, P., van Gemund, A.J.C.: An observation-based model for fault localization. In: Proceedings of the 2008 International Workshop on Dynamic Analysis: Held in Conjunction with the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2008), WODA 2008, pp. 64–70. ACM, New York (2008)Google Scholar
  3. 3.
    Artzi, S., Dolby, J., Tip, F., Pistoia, M.: Directed test generation for effective fault localization. In: Proceedings of the 19th International Symposium on Software Testing and Analysis, ISSTA 2010, pp. 49–60. ACM, New York (2010)CrossRefGoogle Scholar
  4. 4.
    Chen, M., Kiciman, E., Fratkin, E., Fox, A., Brewer, E.: Pinpoint: problem determination in large, dynamic internet services. In: Proceedings of the 32th IEEE/IFIP International Conference on Dependable Systems and Networks, Washington DC, USA, pp. 595–604 (2002)Google Scholar
  5. 5.
    Dallmeier, V., Lindig, C., Zeller, A.: Lightweight bug localization with ample. In: Proceedings of the Sixth International Symposium on Automated Analysis-driven Debugging, AADEBUG 2005, pp. 99–104. ACM, New York (2005)CrossRefGoogle Scholar
  6. 6.
    DiGiuseppe, N., Jones, J.A.: On the influence of multiple faults on coverage-based fault localization. In: Proceedings of the 2011 International Symposium on Software Testing and Analysis, ISSTA 2011, pp. 210–220. ACM, New York (2011)Google Scholar
  7. 7.
    de Freitas, F.G., de Souza, J.T.: Ten years of search based software engineering: A bibliometric analysis. In: Cohen, M.B., Ó Cinnéide, M. (eds.) SSBSE 2011. LNCS, vol. 6956, pp. 18–32. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  8. 8.
    Harman, M., Jones, B.: Search based software engineering. Information and Software Technology 43(14), 833–839 (2001)CrossRefGoogle Scholar
  9. 9.
    Harman, M.: The current state and future of search based software engineering. In: FOSE 2007: 2007 Future of Software Engineering, pp. 342–357. IEEE Computer Society, Washington, DC (2007)Google Scholar
  10. 10.
    Harman, M.: The relationship between search based software engineering and predictive modeling. In: Proceedings of the 6th International Conference on Predictive Models in Software Engineering, Timişoara, Romania, pp. 1:1–1:13 (2010)Google Scholar
  11. 11.
    Harman, M., Mansouri, S.A., Zhang, Y.: Search-based software engineering: Trends, techniques and applications. ACM Computing Surveys 45(1), 11:1–11:61 (2012)Google Scholar
  12. 12.
    Jones, J.A., Harrold, M.J., Stasko, J.: Visualization of test information to assist fault localization. In: Proceedings of the 24th International Conference on Software Engineering, Florida, USA, pp. 467–477 (2002)Google Scholar
  13. 13.
    Jones, J.A., Harrold, M.J.: Empirical evaluation of the tarantula automatic fault-localization technique. In: Proceedings of the 20th International Conference on Automated Software Engineering (ASE 2005), pp. 273–282. ACM Press (2005)Google Scholar
  14. 14.
    Jones, J.A., Harrold, M.J., Stasko, J.T.: Visualization for fault localization. In: Proceedings of ICSE Workshop on Software Visualization, pp. 71–75 (2001)Google Scholar
  15. 15.
    McMinn, P.: Search-based software test data generation: A survey. Software Testing, Verification and Reliability 14(2), 105–156 (2004)CrossRefGoogle Scholar
  16. 16.
    Naish, L., Lee, H.J., Ramamohanarao, K.: A model for spectra-based software diagnosis. ACM Transactions on Software Engineering Methodology 20(3), 11:1–11:32 (2011)Google Scholar
  17. 17.
    Räihä, O.: A survey on search-based software design. Computer Science Review 4(4), 203–249 (2010)CrossRefGoogle Scholar
  18. 18.
    Renieres, M., Reiss, S.: Fault localization with nearest neighbor queries. In: Proceedings of the 18th International Conference on Automated Software Engineering, pp. 30–39 (October 2003)Google Scholar
  19. 19.
    de Souza, J.T., Maia, C.L., de Freitas, F.G., Coutinho, D.P.: The human competitiveness of search based software engineering. In: Proceedings of 2nd International Symposium on Search based Software Engineering (SSBSE 2010), pp. 143–152. IEEE Computer Society Press, Benevento (2010)CrossRefGoogle Scholar
  20. 20.
    Wong, W.E., Qi, Y., Zhao, L., Cai, K.Y.: Effective fault localization using code coverage. In: Proceedings of the 31st Annual International Computer Software and Applications Conference, COMPSAC 2007, vol. 1, pp. 449–456. IEEE Computer Society, Washington, DC (2007)Google Scholar
  21. 21.
    Xie, X.Y., Chen, T.Y., Kuo, F.C., Xu, B.W.: A Theoretical Analysis of the Risk Evaluation Formulas for Spectrum-Based Fault Localization. Accepted by the ACM Transactions on Software Engineering and Methodology (2012)Google Scholar
  22. 22.
    Xie, X.: On the analysis of spectrum-based fault localization. Ph.D. thesis, Swinburne University of Technology (May 2012)Google Scholar
  23. 23.
    Yoo, S.: Evolving human competitive spectra-based fault localisation techniques. In: Fraser, G., Teixeira de Souza, J. (eds.) SSBSE 2012. LNCS, vol. 7515, pp. 244–258. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  24. 24.
    Yu, Y., Jones, J.A., Harrold, M.J.: An empirical study of the effects of test-suite reduction on fault localization. In: Proceedings of the International Conference on Software Engineering (ICSE 2008), pp. 201–210. ACM Press (May 2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Xiaoyuan Xie
    • 1
  • Fei-Ching Kuo
    • 1
  • Tsong Yueh Chen
    • 1
  • Shin Yoo
    • 2
  • Mark Harman
    • 2
  1. 1.Swinburn UniversityHawthornAustralia
  2. 2.University College LondonLondonUK

Personalised recommendations