When NFR Templates Pay Back? A Study on Evolution of Catalog of NFR Templates
[Context] Failures in management of non-functional requirements (NFRs) (e.g., incomplete or ambiguous NFRs) are frequently identified as one of the root causes of software failures. Recent studies confirm that using a catalog of NFR templates for requirements elicitation positively impacts the quality of requirements. However, practitioners are afraid of templates as the return on investment in this technique is still unknown.
[Aim] Our aim was to investigate how the usefulness of catalog of NFR templates and its maintenance costs change over time.
[Method] Using 41 industrial projects with 2,231 NFRs, we simulated 10,000 different random evolutions of a catalog of NFR templates. It allowed us to examine the distribution of catalog value, maintenance effort, catalog utilization over a sequence of projects (a counterpart of elapsing time).
[Results] From the performed analysis it follows that after considering about 40 projects we can expect catalog value of 75% or more and maintenance effort of 10% or less. Then one could expect about 400 templates in the catalog, but only about 10% of them would be used by a single project (on average).
[Conclusions] Usefulness and maintenance costs of catalog of NFR templates depend on the number of projects used to develop it. A catalog of high usefulness and low maintenance effort need to be developed from about 40 projects. Since high variability of studied projects, this number in practice might be lower. From the perspective of a large or medium software company it seems not a big problem.
KeywordsNon-functional requirements Templates Catalog Empirical study Simulation
We would like to thank the companies that shared their data with us especially ATREM S.A., Consdata Sp. z o.o., Currency One S.A., IT Department of Poznan City Hall, Roche Sp. z o.o., TALEX S.A.
- 1.Website of NoRTs. http://norts.cs.put.poznan.pl
- 2.Adolph, S., Bramble, P., Cockburn, A., Pols, A.: Patterns for Effective Use Cases. Addison-Wesley, Boston (2002)Google Scholar
- 4.Berry, D.M., Kamsties, E., Krieger, M.M.: From Contract Drafting to Software Specification: Linguistic Sources of Ambiguity. A Handbook. Ver 1.0. https://cs.uwaterloo.ca/~dberry/handbook/ambiguityHandbook.pdf. Accessed 07 Sept 2015
- 7.Cleland-Huang, J., Mazrouee, S., Liguo, H., Port, D.: Open-Science teraPROMISE repository. http://openscience.us/repo/requirements/other-requirements/nfr. (2010). Accessed 26 June 2017
- 8.Denger, C., Berry, D.M., Kamsties, E.: Higher quality requirements specifications through natural language patterns. In: IEEE International Conference on Software: Science, Technology and Engineering, pp. 80–90 (2003)Google Scholar
- 9.Doerr, J., Paech, B., Koehler, M.: Requirements engineering process improvement based on an information model. In: 2004 Proceedings of 12th IEEE International Requirements Engineering Conference, pp. 70–79. IEEE (2004)Google Scholar
- 10.Eckhardt, J., Vogelsang, A., Femmer, H., Mager, P.: Challenging incompleteness of performance requirements by sentence patterns. In: International Requirements Engineering Conference (RE), pp. 46–55. IEEE (2016)Google Scholar
- 11.Gilb, T.: Competitive Engineering: A Handbook for Systems and Software Engineering Management Using Planguage. Butterworth-Heinemann, Oxford (2005)Google Scholar
- 12.Hull, E., Jackson, K., Dick, J.: Requirements Engineering (2005)Google Scholar
- 13.ISO/IEC: ISO/IEC 25010 - Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models. Technical report, ISO/IEC (2010)Google Scholar
- 14.Kopczynska, S., Nawrocki, J.: Using non-functional requirements templates for elicitation: a case study. In: IEEE International Workshop Requirements Patterns (2014)Google Scholar
- 16.Kopczyńska, S., Ochodek, M., Nawrocki, J.: On importance of non-functional requirements in agile software projects—a survey. In: Jarzabek, S., Poniszewska-Marańda, A., Madeyski, L. (eds.) Integrating Research and Practice in Software Engineering. SCI, vol. 851, pp. 145–158. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-26574-8_11CrossRefGoogle Scholar
- 17.Lam, W., McDermid, T., Vickers, A.: Ten steps towards systematic requirements reuse. In: Intenational Symposium on Requirements Engineering, pp. 6–15. IEEE (1997)Google Scholar
- 19.Mavin, A., Wilkinson, P.: Big Ears (The Return of “Easy Approach to Requirements Engineering”). In: Requirements Engineering Conference, pp. 277–282 (2010)Google Scholar
- 23.Pohl, K., Rupp, C.: Requirements Engineering Fundamentals. Rocky Nook, San Rafael (2011)Google Scholar
- 24.R Documentation: Box Plots. www.rdocumentation.org/packages/graphics/versions/3.5.1/topics/boxplot. Accessed 28th Sept 2018
- 26.Renault, S., Méndez Bonilla, Ó., Franch Gutiérrez, J., Quer Bosor, M.C., et al.: A pattern-based method for building requirements documents in call-for-tender processes. IJCSA 6(5), 175–202 (2009)Google Scholar
- 28.Robertson, S., Robertson, J.: Mastering the Requirements Process: Getting Requirements Right, 3rd edn. Addison-Wesley, Boston (2012)Google Scholar
- 29.Sommerville, I., Sawyer, P.: Requirements Engineering: A Good Practice Guide. Wiley, Hoboken (1997)Google Scholar
- 30.Withall, S.: Software Requirement Patterns (Developer Best Practices). Microsoft Press, Redmond (2007)Google Scholar