Abstract
Weak alignment of requirements engineering (RE) with verification and validation (VV) may lead to problems in delivering the required products in time with the right quality. For example, weak communication of requirements changes to testers may result in lack of verification of new requirements and incorrect verification of old invalid requirements, leading to software quality problems, wasted effort and delays. However, despite the serious implications of weak alignment research and practice both tend to focus on one or the other of RE or VV rather than on the alignment of the two. We have performed a multi-unit case study to gain insight into issues around aligning RE and VV by interviewing 30 practitioners from 6 software developing companies, involving 10 researchers in a flexible research process for case studies. The results describe current industry challenges and practices in aligning RE with VV, ranging from quality of the individual RE and VV activities, through tracing and tools, to change control and sharing a common understanding at strategy, goal and design level. The study identified that human aspects are central, i.e. cooperation and communication, and that requirements engineering practices are a critical basis for alignment. Further, the size of an organisation and its motivation for applying alignment practices, e.g. external enforcement of traceability, are variation factors that play a key role in achieving alignment. Our results provide a strategic roadmap for practitioners improvement work to address alignment challenges. Furthermore, the study provides a foundation for continued research to improve the alignment of RE with VV.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Reference to source is given by interviewee code, see Table 2.
References
Barmi ZA, Ebrahimi AH, Feldt R (2011) Alignment of requirements specification and testing: A systematic mapping study. In: Proc 4th Int. Conf. On Software Testing, Verification and Validation Workshops (ICSTW) pp. 476–485
Cao L, Ramesh B (2008) Agile requirements engineering practices: An empirical study. In: IEEE Software Jan/Feb 2008
Cheng BH, Atlee JM (2007) Research directions in requirements engineering. In: Proc. Future of Software Engineering (FOSE) pp. 285–303
Cleland-Huang J, Chang CK, Christensen M (2003) Event-based traceability for managing evolutionary change. In: IEEE Transactions on Software 29(9)
Damian D, Chisan J, Vaidyanathasamy L, Pal Y (2005) Requirements engineering and downstream software development: findings from a case study. Empir Softw Eng 10:255–283
Damian D, Chisan J (2006) An empirical study of the complex relationship between requirements engineering processes and other processes that lead to payoffs in productivity, quality, and risk management. IEEE Trans Softw Eng 32(7):33–453
De Lucia A, Fasano F, Oliveto R, Tortora G (2007) Recovering traceability links in software artifact management systems using information retrieval methods. In: ACM Transactions on Software Engineering and Methodology, 16(4):Article 13
Dias Neto AC, Arilo C, Subramanyan R, Vieira M, Travassos GH (2007) A survey on model-based testing approaches: A systematic review. In: Proc of 1st ACM Int workshop on Empirical Assessment of Software Engineering Languages and Technologies, pp. 31–36
Ferguson RW, Lami G (2006) An empirical study on the relationship between defective requirements and test failures. In: Proc of 30th Annual IEEE/NASA Software Engineering Workshop SEW-30 (SEW’06)
Fogelström N, Gorschek T (2007) Test-case driven versus checklist-based inspections of software requirements—an experimental evaluation, In: Proc. of 10th Workshop on Requirements Engineering (WER’07)
Gotel O, Finkelstein A (1994) An analysis of the requirements traceability problem. In: Proc. First Int Conf. Requirements Eng., pp. 94–101
Gorschek T, Davis AM (2007) Requirements engineering: in search of the dependent variables. Inf Softw Technol 50(1–2):67–75
Gorschek T, Wohlin C (2004) Packaging software process improvement issues—a method and a case study. Softw Pract & Experience 34:1311–1344
Gorschek T, Wohlin C (2006) Requirements abstraction model. Requir Eng J 11:79–101
Graham D (2002) Requirements and testing: seven missing-link myths. IEEE Softw 19:15–17
Grieskamp W, Kicillof N, Stobie K, Braberman V (2011) Model-based quality assurance of protocol documentation: tools and methodology. Softw Test Verification Reliab 21(1):55–71
Hasling B, Goetz H, Beetz K (2008) Model based testing of system requirements using UML use case models. In: Proc of 2008 international Conference on Software Testing, Verification, and Validation
Hayes JH, Dekhtyar A, Sundaram SK, Holbrook EA, Vadlamudi S, April A (2007) REquirements TRacing On target (RETRO): improving software maintenance through traceability recovery. Innov Syst Softw Eng 3(3):193–202
Höst M, Feldt R, Lüders F (2010) Support for different stakeholders in software engineering master thesis projects. IEEE Trans Educ 52(2):288–296. doi:10.1109/TE.2009.2016106
IEEE (1990) IEEE standard glossary of software engineering terminology. Technical Report 610.12–1990. IEEE, New York
ISO/IEC. (2001) 9126–1:2001(E), International standard software engineering product quality part 1: Quality model. Technical report, ISO/IEC
Jarke, M (1998) Requirements Traceability. Commun ACM 41(12):32–36
Jones JA, Grechanik M, van der Hoek A (2009) Enabling and enhancing collaborations between software development organizations and independent test agencies. In: Cooperative and Human Aspects of Software Engineering (CHASE’09), May 17, 2009, Vancouver, Canada
Kraut RE, Streeter L (1995) Coordination in software development. Commun ACM 38(3):69–81
Kukkanen J, Vakevainen K, Kauppinen M, Uusitalo E (2009) Applying a systematic approach to link requirements and testing: A case study, In: Proc of Asia-Pacific Software Engineering Conference (APSEC ’09):482–488
Lormans M, van Deursen A, Gross H (2008) An industrial case study in reconstructing requirements views, In: Empirical Software Engineering, online first, September 03, 2008
Lubars M, Potts C, Richter C (1993) A review of the state of the practice in requirements modelling. In: Proc of 1st IEEE Int. Symposium on Requirements Engineering, pp. 2–14
Martin R, Melnik G (2008) Tests and requirements, requirements and tests a Möbius strip. IEEE Softw 25(1):54–59
Melnik G, Maurer F, Chiasson M (2006) Executable acceptance tests for communicating business requirements: Customer perspective. In: Proc. of Agile Conference, Minneapolis, USA, pp. 12–46
Mohagheghi P, Dehlen V (2008) Where is the proof? - A review of experiences from applying MDE in industry. In: LNCS, Model driven architecture—foundations and applications, vol 5095:432–443
Nebut C, Fleurey F, Traon YL, Jézéquel J (2006) Automatic test generation: a use case driven approach. IEEE Trans Softw Eng 32(3):140–155
Paci F, Massacci F, Bouquet F, Debricon S (2012) Managing evolution by orchestrating requirements and testing engineering processes. In: Proc. of IEEE 5th Int. Conf. On, pp.834–841, 17–21 April 2012
Pettersson F, Ivarsson M, Gorschek (2008) A practitioner’s guide to light weight software process assessment and improvement planning. J Syst Softw 81(6):972–995
Post H, Sinz C, Merz F, Gorges T, Kropf T (2009) Linking functional requirements and software verification. In: Proceedings of 17th IEEE International Requirements Engineering Conference, pp. 295–302
Ramesh B, Stubbs C, Powers T, Edwards M (1997) Requirements traceability: theory and practice. Ann Softw Eng 3(1):397–415
Ramesh B (1998) Factors influencing requirements traceability practice. In: Communications of the ACM CACM Homepage archive, 41(12):37–44
Randell B (1969) Towards a methodology of computing system design. In: Naur P, Randell B (eds) NATO working conference on software engineering 1968. Report on a Conference Sponsored by NATO Scientific Committee, Garmisch, pp 204–208
Regnell B, Runeson P (1998) Combining scenario-based requirements with static verification and dynamic testing. In: Proc. 4th Int. Working Conf. Requirements Engineering: Foundation for Software Quality, pp.195–206
Regnell B, Runeson P, Wohlin C (2000) Towards integration of use case modelling and usage-based testing. J Syst Softw 50(2):117–130
Ricca F, Torchiano M, Di Penta M, Ceccato M, Tonella P (2009) Using acceptance tests as a support for clarifying requirements: a series of experiments. Inf Softw Technol 51:270–283
Robson C (2002) Real world research: A resource for social scientists and practitioner researchers, 2nd edition. Blackwell Publishing
Runeson P, Andersson C, Höst M (2003) Test processes in software product evolution—a qualitative survey on the state of practice. J Softw Maint Evol Res Pract 15(1):41–59
Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering. Empir Softw Eng 14(2):131–164
Runeson P, Höst M, Rainer A, Regnell B (2012) Case study research in software engineering—guidelines and examples. Wiley
Sabaliauskaite G, Loconsole A, Engström E, Unterkalmsteiner M, Regnell B, Runeson P, Gorschek T, Feldt R (2010) Challenges in aligning requirements engineering and verification in a large-scale industrial context. In: Proceedings of REFSQ 2010
Sikora E, Tenbergen B, Pohl K (2012) Industry needs and research directions in requirements engineering for embedded systems. Requir Eng 17(1):57–78
Uusitalo EJ, Komassi M, Kauppinen M, Davis AM (2008) Linking requirements and testing in practice. In: Proceedings of 16th IEEE International Requirements Engineering Conference, pp. 295–302
Watkins R, Neal M (1994) Why and how of requirements tracing. IEEE Softw 11(4):104–106
Yue T, Briand L, Labiche Y (2011) A systematic review of transformation approaches between user requirements and analysis models. Requir Eng 16(2):75–99
Acknowledgment
We want to thank Börje Haugset for acting as interviewer in three of the interviews. We would also like to thank all the participating companies and anonymous interviewees for their contribution to this project. The research was funded by EASE Industrial Excellence Center for Embedded Applications Software Engineering (http://ease.cs.lth.se).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Nachiappan Nagappan
Rights and permissions
About this article
Cite this article
Bjarnason, E., Runeson, P., Borg, M. et al. Challenges and practices in aligning requirements with verification and validation: a case study of six companies. Empir Software Eng 19, 1809–1855 (2014). https://doi.org/10.1007/s10664-013-9263-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-013-9263-y