A Qualitative Survey of Regression Testing Practices

  • Emelie Engström
  • Per Runeson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6156)


Aim: Regression testing practices in industry have to be better understood, both for the industry itself and for the research community. Method: We conducted a qualitative industry survey by i) running a focus group meeting with 15 industry participants and ii) validating the outcome in an on line questionnaire with 32 respondents. Results: Regression testing needs and practices vary greatly between and within organizations and at different stages of a project. The importance and challenges of automation is clear from the survey. Conclusions: Most of the findings are general testing issues and are not specific to regression testing. Challenges and good practices relate to test automation and testability issues.


Regression testing Survey Industry practice 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    IEEE: IEEE standard for software test documentation. IEEE Std(829-1983, Revision) (1998)Google Scholar
  2. 2.
    Chittimalli, P.K., Harrold, M.J.: Recomputing coverage information to assist regression testing. IEEE Transactions on Software Engineering 35(4), 452–469 (2009)CrossRefGoogle Scholar
  3. 3.
    Engström, E., Runeson, P., Skoglund, M.: A systematic review on regression test selection techniques. Information and Software Technology 52(1), 14–30 (2010)CrossRefGoogle Scholar
  4. 4.
    Engström, E., Runeson, P., Wikstrand, G.: An empirical evaluation of regression testing based on fix-cache recommendations. In: Proceedings of the 3rd International Conference on Software Testing Verification and Validation, pp. 75–78 (2010)Google Scholar
  5. 5.
    Skoglund, M., Runeson, P.: A case study of the class firewall regression test selection technique on a large scale distributed software system. In: International Symposium on Empirical Software Engineering., pp. 72–81 (2005)Google Scholar
  6. 6.
    White, L., Robinson, B.: Industrial real-time regression testing and analysis using firewalls. In: Proceedings 20th IEEE International Conference on Software Maintenance, pp. 18–27 (2004)Google Scholar
  7. 7.
    Rooksby, J., Rouncefield, M., Sommerville, I.: Testing in the wild: The social and organisational dimensions of real world practice. Computer Supported Cooperative Work (CSCW) 18(5), 559–580 (2009)CrossRefGoogle Scholar
  8. 8.
    Flink, A.: The survey handbook, 2nd edn. SAGE Publications, Thousand Oaks (2003)Google Scholar
  9. 9.
    Onoma, A.K., Tsai, W.T., Poonawala, M.H., Suganuma, H.: Regression testing in an industrial environment: Progress is attained by looking backward. Association for Computing Machinery. Communications of the ACM 41(5), 81–86 (1998)Google Scholar
  10. 10.
    Causevic, A., Sundmark, D., Punnekkat, S.: An industrial survey on contemporary aspects of software testing. In: Proceedings of the 3rd International Conference on Software Testing Verification and Validation, pp. 393–401 (2010)Google Scholar
  11. 11.
    Grindal, M., Offutt, J., Mellin, J.: On the testing maturity of software producing organizations. In: Testing: Academia & Industry Conference-Practice And Research Techniques, TAIC/PART (2006)Google Scholar
  12. 12.
    Runeson, P.: A survey of unit testing practices. IEEE Software 23(4), 22 (2006)CrossRefGoogle Scholar
  13. 13.
    Runeson, P., Andersson, C., Höst, M.: Test processes in software product evolution - a qualitative survey on the state of practice. Journal of Software Maintenance and Evolution: Research and Practice 15, 41–59 (2003)zbMATHCrossRefGoogle Scholar
  14. 14.
    Robson, C.: Real World Research, 2nd edn. Blackwell Publishing, Malden (2002)Google Scholar
  15. 15.
    Runeson, P., Beremark, P., Larsson, B., Lundh, E.: SPIN-syd - a non-profit exchange network. In: 1st International Workshop on Software Engineering Networking Experiences, Joensuu, Finland (2006)Google Scholar
  16. 16.
    Surveygizmo (December 2009) a web tool for questionnaires and polls,
  17. 17.
    Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering 14(2), 131–164 (2009)CrossRefGoogle Scholar
  18. 18.
    Zachman, J.A.: A framework for information systems architecture. IBM Systems Journal 26(3), 276–293 (1987)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Emelie Engström
    • 1
  • Per Runeson
    • 1
  1. 1.Department of Computer ScienceLund UniversityLUNDSweden

Personalised recommendations