Skip to main content

Do developers benefit from requirements traceability when evolving and maintaining a software system?

Abstract

Software traceability is a required component of many software development processes. Advocates of requirements traceability cite advantages like easier program comprehension and support for software maintenance (i.e., software change). However, despite its growing popularity, there exists no published evaluation about the usefulness of requirements traceability. It is important, if not crucial, to investigate whether the use of requirements traceability can significantly support development tasks to eventually justify its costs. We thus conducted a controlled experiment with 71 subjects re-performing real maintenance tasks on two third-party development projects: half of the tasks with and the other half without traceability. Subjects sketched their task solutions on paper to focus on the their ability to solving the problems rather than their programming skills. Our findings show that subjects with traceability performed on average 24 % faster on a given task and created on average 50 % more correct solutions—suggesting that traceability not only saves effort but can profoundly improve software maintenance quality.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. 1.

    The U.S. Health Insurance Portability and Accountability Act of 1996 (HIPAA) contains a privacy rule, which regulates the use and disclosure of Protected Health Information (PHI). PHI is any information held by a covered entity, which concerns health status, provision of health care, or payment for health care that can be linked to an individual (HIPPA 1996).

References

  1. Ahmad A, Ghazali M (2007) Documenting requirements traceability information for small projects. In: IEEE international multitopic conference (INMIC 2007), pp 1–5. doi: 10.1109/INMIC.2007.4557711

  2. Arkley P, Riddle S (2005) Overcoming the traceability benefit problem. In: Proceedings of the 13th international requirements engineering conference (RE05), pp 385–389. doi: 10.1109/RE.2010.35

  3. Barashev D, Thomas A (2013) GanttProject: Free project scheduling and management. http://www.ganttproject.biz

  4. Basili V, Shull F, Lanubile F (1999) Building knowledge through families of experiments. IEEE Trans Softw Eng 25(4):456–473. doi: 10.1109/32.799939

    Article  Google Scholar 

  5. Box GEP, Hunter JS, Hunter WG (2005) Statistics for experimenters: design, innovation, and discovery, 2nd edn. Wiley, ISBN 978-0-471-71813-0

  6. Briand L, Falessi D, Nejati S, Sabetzadeh M, Yue T (2014) Traceability and sysml design slices to support safety inspections: a controlled experiment. ACM Trans Softw Eng Methodol (TOSEM) 23(1):1–43. doi: 10.1145/2559978

    Article  Google Scholar 

  7. Cleland-Huang J, Gotel O, Huffman Hayes J, Mäder P, Zisman A (2014) Software traceability: trends and future directions. In: Proceedings of the 36th international conference on software engineering (ICSE). Hyderabad. pp 55–69. doi: 10.1145/2593882.2593891

  8. Crawley MJ (2002) Statistical computing—an introduction to data analysis using S-plus. Wiley, ISBN 978-0-471-56040-1

  9. Cuddeback D, Dekhtyar A, Huffman Hayes J (2010) Automated requirements traceability: the study of human analysts. In: Proceedings of the 18th IEEE international requirements engineering conference (RE10), pp 231–240. doi: 10.1109/RE.2010.35

  10. Curtis B, Sheppard S B, Milliman P, Borst M A, Love T (1979) Measuring the psychological complexity of software maintenance tasks with the Halstead and McCabe metrics. IEEE Trans Softw Eng 5(2):96–104. doi: 10.1109/TSE.1979.234165

    Article  MATH  Google Scholar 

  11. De Lucia A, Oliveto R, Zurolo F, Di Penta M (2006) Improving comprehensibility of source code via traceability information: a controlled experiment. In: Proceedings of the 14th international conference on program comprehension (ICPC 2006), pp 317–326. doi: 10.1109/ICPC.2006.28

  12. De Lucia A, Oliveto R, Tortora G (2008) IR-based traceability recovery processes: an empirical comparison of “one-shot” and incremental processes. In: Proceedings of the 23rd IEEE/ACM international conference on automated software engineering (ASE 2008). IEEE Computer Society, pp 39–48. doi: 10.1109/ASE.2008.14

  13. Dömges R, Pohl K (1998) Adapting traceability environments to project-specific needs. Commun ACM 41(12):54–62. doi: 10.1145/290133.290149

    Article  Google Scholar 

  14. Dzidek WJ, Arisholm E, Briand LC (2008) A realistic empirical evaluation of the costs and benefits of UML in software maintenance. IEEE Trans Softw Eng 34(3):407–432. doi: 10.1109/TSE.2008.15

    Article  Google Scholar 

  15. Eclipse Metrics plugin (2013) Eclipse metrics plugin, http://sourceforge.net/projects/metrics/

  16. Egyed A, Graf F, Gr¨unbacher P (2010) Effort and quality of recovering requirements-to-code traces: two exploratory experiments. In: Proceedings of the 18th IEEE international requirements engineering conference (RE), pp 221–230. doi: 10.1109/RE.2010.34

  17. Glass R L (2002) Facts and fallacies of software engineering. Addison-Wesley Professional, Boston. ISBN 978-0321117427

    Google Scholar 

  18. Gotel O, Finkelstein A (1994) An analysis of the requirements traceability problem. In: Proceedings of the 1st international conference on requirements engineering ICRE94. Colorado Springs, pp 94–101. doi: 10.1109/ICRE.1994.292398

  19. Gotel O, Cleland-Huang J, Huffman Hayes J, Zisman A, Egyed A, Grünbacher P, Dekhtyar A, Antoniol G, Maletic J, Mäder P (2012) Traceability fundamentals. In: Cleland-Huang J, Gotel O, Zisman A (eds) Software and systems traceability. Springer, London, pp 3–22. doi: 10.1007/978-1-4471-2239-5_1

    Chapter  Google Scholar 

  20. HIPPA (1996) Health insurance portability and accountability act (HIPPA)—wikipedia, the free encyclopedia. http://en.wikipedia.org/w/index.php?title=Health_Insurance_Portability_and_Accountability_Act&oldid=593856731. Accessed 14 Feb 2014

  21. ISO (2011) ISO:26262-6:2011 Road vehicles—functional safety—part 6: product development at the software level

  22. Leffingwell D (1997) Calculating your return on investment from more effective requirements management. Tech. rep., Available online at http://www.ibm.com/developerworks/rational/library/347.html

  23. Lindvall M, Sandahl K (1996) Practical implications of traceability. Softw Pract Exp 26(10):1161–1180. doi: 10.1002/(SICI)1097-024X(199610)26:10<1161::AID-SPE58>3.0.CO;2-X

    Article  Google Scholar 

  24. Mäder P, Cleland-Huang J (2010) A visual traceability modeling language. In: Model driven engineering languages and systems—13th international conference, MODELS 2010, Oslo, Norway, October 3–8, 2010, Proceedings, Springer, vol LNCS 6394, pp 226–240. doi: 10.1007/978-3-642-16145-2_16

  25. Mäder P, Cleland-Huang J (2013) A visual language for modeling and executing traceability queries. Softw Syst Model 12(3):537–553. doi: 10.1007/s10270-012-0237-0

    Article  Google Scholar 

  26. Mäder P, Egyed A (2012) Assessing the effect of requirements traceability for software maintenance. In: Proceedings of the 28th IEEE international conference on software maintenance (ICSM). IEEE Computer Society, pp 171–180. doi: 10.1109/ICSM.2012.6405269

  27. Mäder P, Gotel O (2012) Towards automated traceability maintenance. J Syst Softw 85(10):2205–2227. doi: 10.1016/j.jss.2011.10.023

    Article  Google Scholar 

  28. Mäder P, Gotel O, Philippow I (2008) Rule-based maintenance of post-requirements traceability relations. In: 16th IEEE international requirements engineering conference (RE’08), 8–12 September 2008, Barcelona, pp 23–32. doi: 10.1109/RE.2008.24

  29. Mäder P, Gotel O, Philippow I (2009) Motivation matters in the traceability trenches. In: Proceedings of the 17th international requirements engineering conference (RE09), Atlanta, pp 143–148. doi: 10.1109/RE.2009.23

  30. Mäder P, Jones PL, Zhang Y, Cleland-Huang J (2013) Strategic traceability for safety critical projects. IEEE Softw 30(3):58–66. doi: 10.1109/MS.2013.60

    Article  Google Scholar 

  31. Omoronyia I, Sindre G (2011) Exploring a bayesian and linear approach to requirements traceability. Inf Softw Technol 53(8):851–871. doi: 10.1016/j.infsof.2011.03.001

    Article  Google Scholar 

  32. Pohl K (1996a) PRO-ART: enabling requirements pre-traceability. In: Proceedings of the 2nd international conference on requirements engineering (ICRE). IEEE Computer Society, pp 76–84. doi: 10.1109/ICRE.1996.491432

  33. Pohl K (1996b) Process-centered requirements engineering. Research Studies Press, ISBN 0-86380-193-5

  34. R Project (2013) The R project for statistical computing. http://www.r-project.org

  35. Ramesh B, Jarke M (2001) Toward reference models of requirements traceability. IEEE Trans Softw Eng 27(1):58–93. doi: 10.1109/32.895989

    Article  Google Scholar 

  36. Ramesh B, Powers T, Stubbs C, Edwards M (1995) Implementing requirements traceability: a case study. In: Proceedings of the 2nd IEEE international symposium on requirements engineering (ISRE), pp 89–95. doi: 10.1109/ISRE.1995.512549

  37. Rempel P, Mäder P, Kuschke T (2013) An empirical study on project-specific traceability strategies. In: Proceedings of the 21st international requirements engineering conference (RE13). Rio de Janeiro, pp 195–204. doi: 10.1109/RE.2013.6636719

  38. Rempel P, Mäder P, Kuschke T, Cleland-Huang J (2014) Mind the gap: assessing the conformance of software traceability to relevant guidelines. In: Proceedings of the 36th international conference on software engineering (ICSE14). Hyderabad, pp 943–954. doi: 10.1145/2568225.2568290

  39. Ricca F, Di Penta M, Torchiano M, Tonella P, Ceccato M (2010) How developers’ experience and ability influence web application comprehension tasks supported by uml stereotypes: a series of four experiments. IEEE Trans Softw Eng 36(1):96–118. doi: 10.1109/TSE.2009.69

    Article  Google Scholar 

  40. RTCA/EUROCAE (2011) DO-178C/ED-12C: software considerations in airborne systems and equipment certification

  41. Williams L, Meneely A, Smith S, Hayward L, Smith B, King J (2013) iTrust: role-based healthcare. http://agile.csc.ncsu.edu/iTrust/

Download references

Acknowledgments

We would like to thank all participants for their dedicated work and the developers of Gantt and iTrust for making their work publicly available. This work was funded by the Austrian Science Fund (FWF) M1268-N23, P23115-N23, the German Research Foundation (DFG) Ph49/8-1, and the German Ministry of Education and Research (BMBF) 16V0116.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Patrick Mäder.

Additional information

Communicated by: Massimiliano Di Penta and Jonathan Maletic

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 1.27 MB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mäder, P., Egyed, A. Do developers benefit from requirements traceability when evolving and maintaining a software system?. Empir Software Eng 20, 413–441 (2015). https://doi.org/10.1007/s10664-014-9314-z

Download citation

Keywords

  • Requirements traceability
  • Software traceability
  • Software maintenance
  • Software evolution
  • Traceability usage
  • Traceability effect
  • Traceability benefit
  • Empirical software engineering
  • Controlled experiment
  • Study