Advertisement

Empirical Software Engineering

, Volume 20, Issue 2, pp 413–441 | Cite as

Do developers benefit from requirements traceability when evolving and maintaining a software system?

  • Patrick Mäder
  • Alexander Egyed
Article

Abstract

Software traceability is a required component of many software development processes. Advocates of requirements traceability cite advantages like easier program comprehension and support for software maintenance (i.e., software change). However, despite its growing popularity, there exists no published evaluation about the usefulness of requirements traceability. It is important, if not crucial, to investigate whether the use of requirements traceability can significantly support development tasks to eventually justify its costs. We thus conducted a controlled experiment with 71 subjects re-performing real maintenance tasks on two third-party development projects: half of the tasks with and the other half without traceability. Subjects sketched their task solutions on paper to focus on the their ability to solving the problems rather than their programming skills. Our findings show that subjects with traceability performed on average 24 % faster on a given task and created on average 50 % more correct solutions—suggesting that traceability not only saves effort but can profoundly improve software maintenance quality.

Keywords

Requirements traceability Software traceability Software maintenance Software evolution Traceability usage Traceability effect Traceability benefit Empirical software engineering Controlled experiment Study 

Notes

Acknowledgments

We would like to thank all participants for their dedicated work and the developers of Gantt and iTrust for making their work publicly available. This work was funded by the Austrian Science Fund (FWF) M1268-N23, P23115-N23, the German Research Foundation (DFG) Ph49/8-1, and the German Ministry of Education and Research (BMBF) 16V0116.

Supplementary material

10664_2014_9314_MOESM1_ESM.pdf (1.3 mb)
(PDF 1.27 MB)

References

  1. Ahmad A, Ghazali M (2007) Documenting requirements traceability information for small projects. In: IEEE international multitopic conference (INMIC 2007), pp 1–5. doi:  10.1109/INMIC.2007.4557711
  2. Arkley P, Riddle S (2005) Overcoming the traceability benefit problem. In: Proceedings of the 13th international requirements engineering conference (RE05), pp 385–389. doi:  10.1109/RE.2010.35
  3. Barashev D, Thomas A (2013) GanttProject: Free project scheduling and management. http://www.ganttproject.biz
  4. Basili V, Shull F, Lanubile F (1999) Building knowledge through families of experiments. IEEE Trans Softw Eng 25(4):456–473. doi:  10.1109/32.799939 CrossRefGoogle Scholar
  5. Box GEP, Hunter JS, Hunter WG (2005) Statistics for experimenters: design, innovation, and discovery, 2nd edn. Wiley, ISBN 978-0-471-71813-0Google Scholar
  6. Briand L, Falessi D, Nejati S, Sabetzadeh M, Yue T (2014) Traceability and sysml design slices to support safety inspections: a controlled experiment. ACM Trans Softw Eng Methodol (TOSEM) 23(1):1–43. doi:  10.1145/2559978 CrossRefGoogle Scholar
  7. Cleland-Huang J, Gotel O, Huffman Hayes J, Mäder P, Zisman A (2014) Software traceability: trends and future directions. In: Proceedings of the 36th international conference on software engineering (ICSE). Hyderabad. pp 55–69. doi:  10.1145/2593882.2593891
  8. Crawley MJ (2002) Statistical computing—an introduction to data analysis using S-plus. Wiley, ISBN 978-0-471-56040-1Google Scholar
  9. Cuddeback D, Dekhtyar A, Huffman Hayes J (2010) Automated requirements traceability: the study of human analysts. In: Proceedings of the 18th IEEE international requirements engineering conference (RE10), pp 231–240. doi:  10.1109/RE.2010.35
  10. Curtis B, Sheppard S B, Milliman P, Borst M A, Love T (1979) Measuring the psychological complexity of software maintenance tasks with the Halstead and McCabe metrics. IEEE Trans Softw Eng 5(2):96–104. doi:  10.1109/TSE.1979.234165 CrossRefzbMATHGoogle Scholar
  11. De Lucia A, Oliveto R, Zurolo F, Di Penta M (2006) Improving comprehensibility of source code via traceability information: a controlled experiment. In: Proceedings of the 14th international conference on program comprehension (ICPC 2006), pp 317–326. doi:  10.1109/ICPC.2006.28
  12. De Lucia A, Oliveto R, Tortora G (2008) IR-based traceability recovery processes: an empirical comparison of “one-shot” and incremental processes. In: Proceedings of the 23rd IEEE/ACM international conference on automated software engineering (ASE 2008). IEEE Computer Society, pp 39–48. doi:  10.1109/ASE.2008.14
  13. Dömges R, Pohl K (1998) Adapting traceability environments to project-specific needs. Commun ACM 41(12):54–62. doi:  10.1145/290133.290149 CrossRefGoogle Scholar
  14. Dzidek WJ, Arisholm E, Briand LC (2008) A realistic empirical evaluation of the costs and benefits of UML in software maintenance. IEEE Trans Softw Eng 34(3):407–432. doi:  10.1109/TSE.2008.15 CrossRefGoogle Scholar
  15. Eclipse Metrics plugin (2013) Eclipse metrics plugin, http://sourceforge.net/projects/metrics/
  16. Egyed A, Graf F, Gr¨unbacher P (2010) Effort and quality of recovering requirements-to-code traces: two exploratory experiments. In: Proceedings of the 18th IEEE international requirements engineering conference (RE), pp 221–230. doi:  10.1109/RE.2010.34
  17. Glass R L (2002) Facts and fallacies of software engineering. Addison-Wesley Professional, Boston. ISBN 978-0321117427Google Scholar
  18. Gotel O, Finkelstein A (1994) An analysis of the requirements traceability problem. In: Proceedings of the 1st international conference on requirements engineering ICRE94. Colorado Springs, pp 94–101. doi:  10.1109/ICRE.1994.292398
  19. Gotel O, Cleland-Huang J, Huffman Hayes J, Zisman A, Egyed A, Grünbacher P, Dekhtyar A, Antoniol G, Maletic J, Mäder P (2012) Traceability fundamentals. In: Cleland-Huang J, Gotel O, Zisman A (eds) Software and systems traceability. Springer, London, pp 3–22. doi:  10.1007/978-1-4471-2239-5_1 CrossRefGoogle Scholar
  20. HIPPA (1996) Health insurance portability and accountability act (HIPPA)—wikipedia, the free encyclopedia. http://en.wikipedia.org/w/index.php?title=Health_Insurance_Portability_and_Accountability_Act&oldid=593856731. Accessed 14 Feb 2014
  21. ISO (2011) ISO:26262-6:2011 Road vehicles—functional safety—part 6: product development at the software levelGoogle Scholar
  22. Leffingwell D (1997) Calculating your return on investment from more effective requirements management. Tech. rep., Available online at http://www.ibm.com/developerworks/rational/library/347.html
  23. Lindvall M, Sandahl K (1996) Practical implications of traceability. Softw Pract Exp 26(10):1161–1180. doi:  10.1002/(SICI)1097-024X(199610)26:10<1161::AID-SPE58>3.0.CO;2-X CrossRefGoogle Scholar
  24. Mäder P, Cleland-Huang J (2010) A visual traceability modeling language. In: Model driven engineering languages and systems—13th international conference, MODELS 2010, Oslo, Norway, October 3–8, 2010, Proceedings, Springer, vol LNCS 6394, pp 226–240. doi:  10.1007/978-3-642-16145-2_16
  25. Mäder P, Cleland-Huang J (2013) A visual language for modeling and executing traceability queries. Softw Syst Model 12(3):537–553. doi:  10.1007/s10270-012-0237-0 CrossRefGoogle Scholar
  26. Mäder P, Egyed A (2012) Assessing the effect of requirements traceability for software maintenance. In: Proceedings of the 28th IEEE international conference on software maintenance (ICSM). IEEE Computer Society, pp 171–180. doi:  10.1109/ICSM.2012.6405269
  27. Mäder P, Gotel O (2012) Towards automated traceability maintenance. J Syst Softw 85(10):2205–2227. doi:  10.1016/j.jss.2011.10.023 CrossRefGoogle Scholar
  28. Mäder P, Gotel O, Philippow I (2008) Rule-based maintenance of post-requirements traceability relations. In: 16th IEEE international requirements engineering conference (RE’08), 8–12 September 2008, Barcelona, pp 23–32. doi: 10.1109/RE.2008.24
  29. Mäder P, Gotel O, Philippow I (2009) Motivation matters in the traceability trenches. In: Proceedings of the 17th international requirements engineering conference (RE09), Atlanta, pp 143–148. doi:  10.1109/RE.2009.23
  30. Mäder P, Jones PL, Zhang Y, Cleland-Huang J (2013) Strategic traceability for safety critical projects. IEEE Softw 30(3):58–66. doi:  10.1109/MS.2013.60 CrossRefGoogle Scholar
  31. Omoronyia I, Sindre G (2011) Exploring a bayesian and linear approach to requirements traceability. Inf Softw Technol 53(8):851–871. doi:  10.1016/j.infsof.2011.03.001 CrossRefGoogle Scholar
  32. Pohl K (1996a) PRO-ART: enabling requirements pre-traceability. In: Proceedings of the 2nd international conference on requirements engineering (ICRE). IEEE Computer Society, pp 76–84. doi:  10.1109/ICRE.1996.491432
  33. Pohl K (1996b) Process-centered requirements engineering. Research Studies Press, ISBN 0-86380-193-5Google Scholar
  34. R Project (2013) The R project for statistical computing. http://www.r-project.org
  35. Ramesh B, Jarke M (2001) Toward reference models of requirements traceability. IEEE Trans Softw Eng 27(1):58–93. doi:  10.1109/32.895989 CrossRefGoogle Scholar
  36. Ramesh B, Powers T, Stubbs C, Edwards M (1995) Implementing requirements traceability: a case study. In: Proceedings of the 2nd IEEE international symposium on requirements engineering (ISRE), pp 89–95. doi:  10.1109/ISRE.1995.512549
  37. Rempel P, Mäder P, Kuschke T (2013) An empirical study on project-specific traceability strategies. In: Proceedings of the 21st international requirements engineering conference (RE13). Rio de Janeiro, pp 195–204. doi:  10.1109/RE.2013.6636719
  38. Rempel P, Mäder P, Kuschke T, Cleland-Huang J (2014) Mind the gap: assessing the conformance of software traceability to relevant guidelines. In: Proceedings of the 36th international conference on software engineering (ICSE14). Hyderabad, pp 943–954. doi:  10.1145/2568225.2568290
  39. Ricca F, Di Penta M, Torchiano M, Tonella P, Ceccato M (2010) How developers’ experience and ability influence web application comprehension tasks supported by uml stereotypes: a series of four experiments. IEEE Trans Softw Eng 36(1):96–118. doi:  10.1109/TSE.2009.69 CrossRefGoogle Scholar
  40. RTCA/EUROCAE (2011) DO-178C/ED-12C: software considerations in airborne systems and equipment certificationGoogle Scholar
  41. Williams L, Meneely A, Smith S, Hayward L, Smith B, King J (2013) iTrust: role-based healthcare. http://agile.csc.ncsu.edu/iTrust/

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Software Systems/Process Informatics GroupTechnische Universität IlmenauIlmenauGermany
  2. 2.Institute for Software Systems Engineering (ISSE)Johannes Kepler UniversityLinzAustria

Personalised recommendations