International Conference on Agile Software Development

XP 2015: Agile Processes in Software Engineering and Extreme Programming pp 27-39

An Industrial Case Study on Test Cases as Requirements

  • Elizabeth Bjarnason
  • Michael Unterkalmsteiner
  • Emelie Engström
  • Markus Borg
Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 212)

Abstract

It is a conundrum that agile projects can succeed ‘without requirements’ when weak requirements engineering is a known cause for project failures. While Agile development projects often manage well without extensive requirements documentation, test cases are commonly used as requirements. We have investigated this agile practice at three companies in order to understand how test cases can fill the role of requirements. We performed a case study based on twelve interviews performed in a previous study. The findings include a range of benefits and challenges in using test cases for eliciting, validating, verifying, tracing and managing requirements. In addition, we identified three scenarios for applying the practice, namely as a mature practice, as a de facto practice and as part of an agile transition. The findings provide insights into how the role of requirements may be met in agile development including challenges to consider.

Keywords

Agile development Behaviour-driven development Acceptance test Requirements and test alignment Case study 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Sommerville, I.: Integrated requirements engineering: a tutorial. IEEE Softw. 22, 16–23 (2005)CrossRefGoogle Scholar
  2. 2.
    Layman, L., Williams, L., Cunningham, L.: Motivations and measurements in an agile case study. J. Syst. Archit. 52, 654–667 (2006)CrossRefGoogle Scholar
  3. 3.
    Beck, K.: Manifesto for Agile Software Development. http://agilemanifesto.org/
  4. 4.
    Ramesh, B., Cao, L., Baskerville, R.: Agile requirements engineering practices and challenges: an empirical study. Inf. Syst. J. 20, 449–480 (2010)CrossRefGoogle Scholar
  5. 5.
    Bjarnason, E., Runeson, P., Borg, M., et al.: Challenges and practices in aligning requirements with verification and validation: a case study of six companies. Empir. Softw. Eng. 19, 1809–1855 (2014)CrossRefGoogle Scholar
  6. 6.
    Davis, A.M.: Just Enough Requirements Management: Where Software Development Meets Marketing. Dorset House, New York (2005)Google Scholar
  7. 7.
    van Lamsweerde, A.: Formal specification: a roadmap. In: Conf. on The Future of Software Engineering, pp. 147–159. ACM, Limerick (2000)Google Scholar
  8. 8.
    Pohl, K.: Requirements Engineering - Fundamentals, Principles, and Techniques. Springer, Heidelberg (2010)Google Scholar
  9. 9.
    Mavin, A., Wilkinson, P.: Big ears (the return of “easy approach to requirements engineering”). In: 18th Int. Reqts. Engineering Conf., pp. 277–282. IEEE, Sydney (2010)Google Scholar
  10. 10.
    Cohn, M.: User Stories Applied: For Agile Software Development. Addison-Wesley Professional, Boston (2004)Google Scholar
  11. 11.
    Heitmeyer, C.L., Jeffords, R.D., Labaw, B.G.: Automated Consistency Checking of Requirements Specifications. ACM Trans. Softw. Eng. Methodol. 5, 231–261 (1996)CrossRefGoogle Scholar
  12. 12.
    Dromey, R.G.: From requirements to design: formalizing the key steps. In: 1st Int’l Conf. on Software Engineering and Formal Methods, pp. 2–11. IEEE, Brisbane (2003)Google Scholar
  13. 13.
    Miller, T., Strooper, P.: A case study in model-based testing of specifications and implementations. Softw. Test. Verification Reliab. 22, 33–63 (2012)CrossRefGoogle Scholar
  14. 14.
    Davis, A., Overmyer, S., Jordan, K., et al.: Identifying and measuring quality in a software requirements specification. In: 1st Int. Softw. Metrics Symp., Baltimore, USA, pp. 141–152 (1993)Google Scholar
  15. 15.
    Martin, R.C., Melnik, G.: Tests and Requirements, Requirements and Tests: A Möbius Strip. IEEE Softw. 25, 54–59 (2008)CrossRefGoogle Scholar
  16. 16.
    Whittaker, J.A.: What is software testing? And why is it so hard? IEEE Softw. 17, 70–79 (2000)CrossRefGoogle Scholar
  17. 17.
    Runeson, P.: A survey of unit testing practices. IEEE Softw. 23, 22–29 (2006)CrossRefGoogle Scholar
  18. 18.
    Hsia, P., Kung, D., Sell, C.: Software requirements and acceptance testing. Ann. Softw. Eng. 3, 291–317 (1997)CrossRefGoogle Scholar
  19. 19.
    Lethbridge, T.C., Singer, J., Forward, A.: How software engineers use documentation: the state of the practice. IEEE Softw. 20, 35–39 (2003)CrossRefGoogle Scholar
  20. 20.
    Haugset, B., Hanssen, G.K.: Automated acceptance testing: a literature review and an industrial case study. In: Agile Conf., pp. 27–38. IEEE, Toronto (2008)Google Scholar
  21. 21.
    Park, S., Maurer, F.: A literature review on story test driven development. In: 11th Int. Conf. on Agile Processes in Softw. Engin. and Extreme Progr., pp. 208–213 (2010)Google Scholar
  22. 22.
    Melnik, G., Maurer, F., Chiasson, M.: Executable acceptance tests for communicating business requirements: customer perspective. In: IEEE Agile Conf., USA, pp. 35–46 (2006)Google Scholar
  23. 23.
    Causevic, A., Sundmark, D., Punnekkat, S.: Factors limiting industrial adoption of test driven development: a systematic review. In: 4th Int’l Conf. on Software Testing, Verification and Validation, pp. 337–346. IEEE, Berlin (2011)Google Scholar
  24. 24.
    George, B., Williams, L.: A structured experiment of test-driven development. Inf. Softw. Technol. 46, 337–342 (2004)CrossRefGoogle Scholar
  25. 25.
    Janzen, D.S., Saiedian, H.: A leveled examination of test-driven development acceptance. In: 29th Int’l Conf. on Software Engineering, pp. 719–722. IEEE, Minneapolis (2007)Google Scholar
  26. 26.
    North, D.: Behavior Modification: The evolution of behavior-driven development (2006)Google Scholar
  27. 27.
    Runeson, P., Höst, M., Rainer, A., Regnell, B.: Case Study Research in Software Engineering: Guidelines and Examples. Wiley, Hoboken (2012)CrossRefGoogle Scholar
  28. 28.
    Lauesen, S.: Software Requirements: Styles & Techniques. Addison-Wesley Professional, Harlow (2002)Google Scholar
  29. 29.
    Melnik, G., Maurer, F.: Multiple perspectives on executable acceptance test-driven development. In: Concas, G., Damiani, E., Scotto, M., Succi, G. (eds.) XP 2007. LNCS, vol. 4536, pp. 245–249. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  30. 30.
    Park, S., Maurer, F.: Communicating domain knowledge in executable acceptance test driven development. In: Abrahamsson, P., Marchesi, M., Maurer, F. (eds.) Agile Processes in Software Engineering and Extreme Programming. LNBIP, vol. 31, pp. 23–32. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  31. 31.
    Latorre, R.: A successful application of a Test-Driven Development strategy in the industrial environment. Empir. Softw. Eng. 19, 753–773 (2014)CrossRefGoogle Scholar
  32. 32.
    Kongsli, V.: Towards agile security in web applications. In: 21st ACM SIGPLAN Symp. on Object-oriented Progr. Systems, Languages, & Appl., Portland, USA, pp. 805–808 (2006)Google Scholar
  33. 33.
    Mugridge, R.: Managing Agile Project Requirements with Storytest-Driven Development. IEEE Softw. 25, 68–75 (2008)CrossRefGoogle Scholar
  34. 34.
    Uusitalo, E.J., Komssi, M., Kauppinen, M., Davis, A.M.: Linking requirements and testing in practice. In: 16th Int. Conf. Reqts. Engineering, pp. 265–270. IEEE, Catalunya (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Elizabeth Bjarnason
    • 1
  • Michael Unterkalmsteiner
    • 2
  • Emelie Engström
    • 1
  • Markus Borg
    • 1
  1. 1.Lund UniversityLundSweden
  2. 2.Blekinge Institute of TechnologyKarlskronaSweden

Personalised recommendations