Engineering Requirements with Desiree: An Empirical Evaluation

  • Feng-Lin LiEmail author
  • Jennifer Horkoff
  • Lin Liu
  • Alex Borgida
  • Giancarlo Guizzardi
  • John Mylopoulos
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9694)


The requirements elicited from stakeholders suffer from various afflictions, including informality, vagueness, incompleteness, ambiguity, inconsistencies, and more. It is the task of the requirements engineering process to derive from these a formal specification that truly captures stakeholder needs. The Desiree requirements engineering framework supports a rich collection of refinement operators through which an engineer can iteratively transform stakeholder requirements into a specification. The framework includes an ontology, a formal representation for requirements, as well as a tool and a systematic process for conducting requirements engineering. This paper reports the results of a series of empirical studies intended to evaluate the effectiveness of Desiree. The studies consist of three controlled experiments, where students were invited to conduct requirements analysis using textbook techniques or our framework. The results of the experiments offer strong evidence that with sufficient training, our framework indeed helps users conduct more effective requirements analysis.


Requirements problem Controlled experiment Hypothesis testing Effect size 



This research has been funded by the ERC advanced grant 267856 “Lucretius: Foundations for Software Evolution” (April 2011–March 2016). It has also been supported by the Key Project of National Natural Science Foundation of China (no. 61432020), and the Key Project in the National Science & Technology Pillar Program during the Twelfth Five-year Plan Period (No. 2015BAH14F02). Jennifer is supported by an ERC Marie Skodowska-Curie Intra European Fellow-ship (PIEFGA - 2013 - 627489) and by a Natural Sciences and Engineering Research Council of Canada Postdoctoral Fellowship (September 2014–August 2016).


  1. 1.
  2. 2.
  3. 3.
  4. 4.
    Al-Subaie, H.S., Maibaum, T.S.: Evaluating the effectiveness of a goal-oriented requirements engineering method. In: CERE 2006, pp. 8–19. IEEE (2006)Google Scholar
  5. 5.
    Baader, F.: The Description Logic Handbook: Theory, Implementation, and Applications. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  6. 6.
    Coe, R.: It’s the effect size, stupid: what effect size is and why it is important (2002)Google Scholar
  7. 7.
    Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. Academic Press, New York (2013)zbMATHGoogle Scholar
  8. 8.
    Committee, I., Board, I.S.S.: IEEE Recommended Practice for Software Requirements Specifications. IEEE (1998)Google Scholar
  9. 9.
    Davis, A.M.: Software Requirements: Objects, Functions, and States. Prentice-Hall, Inc., Englewood Cliffs (1993)zbMATHGoogle Scholar
  10. 10.
    Estrada, H., Rebollar, A.M., Pastor, Ó., Mylopoulos, J.: An empirical evaluation of the \(i^*\) framework in a model-based software generation environment. In: Martinez, F.H., Pohl, K. (eds.) CAiSE 2006. LNCS, vol. 4001, pp. 513–527. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Fabbrini, F., Fusani, M., Gnesi, S., Lami, G.: The linguistic approach to the natural language requirements quality: benefit of the use of an automatic tool. In: IEEE/NASA SEW-26, pp. 97–105. IEEE (2001)Google Scholar
  12. 12.
    Guizzardi, R., Li, F.L., Borgida, A., Guizzardi, G., Horkoff, J., Mylopoulos, J.: An ontological interpretation of non-functional requirements. In: FOIS 2014, vol. 267, pp. 344–357. IOS Press (2014)Google Scholar
  13. 13.
    Horkoff, J., Aydemir, F.B., Li, F.-L., Li, T., Mylopoulos, J.: Evaluating modeling languages: an example from the requirements domain. In: Yu, E., Dobbie, G., Jarke, M., Purao, S. (eds.) ER 2014. LNCS, vol. 8824, pp. 260–274. Springer, Heidelberg (2014)Google Scholar
  14. 14.
    Jackson, M., Zave, P.: Deriving specifications from requirements: an example. In: ICSE 1995, pp. 15–24. ACM (1995)Google Scholar
  15. 15.
    Kamalrudin, M., Hosking, J., Grundy, J.: Improving requirements quality using essential use case interaction patterns. In: ICSE 2011, pp. 531–540. ACM (2011)Google Scholar
  16. 16.
    Krogstie, J.: A semiotic approach to quality in requirements specifications. In: Liu, K., Clarke, R.J., Andersen, P.B., Stamper, R.K., Abou-Zeid, E.-S. (eds.) Organizational Semiotics. IFIP, vol. 94, pp. 231–249. Springer, New York (2002)CrossRefGoogle Scholar
  17. 17.
    LeBlanc, D.C.: Statistics: Concepts and Applications for Science, vol. 2. Jones & Bartlett Learning, Burlington (2004)Google Scholar
  18. 18.
    Li, F.-L., Horkoff, J., Borgida, A., Guizzardi, G., Liu, L., Mylopoulos, J.: From stakeholder requirements to formal specifications through refinement. In: Fricker, S.A., Schneider, K. (eds.) REFSQ 2015. LNCS, vol. 9013, pp. 164–180. Springer, Heidelberg (2015)Google Scholar
  19. 19.
    Li, F.L., Horkoff, J., Mylopoulos, J., Guizzardi, R.S., Guizzardi, G., Borgida, A., Liu, L.: Non-functional requirements as qualities, with a spice of ontology. In: RE 2014, pp. 293–302. IEEE (2014)Google Scholar
  20. 20.
    Matulevičius, R., Heymans, P.: Comparing goal modelling languages: an experiment. In: Sawyer, P., Heymans, P. (eds.) REFSQ 2007. LNCS, vol. 4542, pp. 18–32. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  21. 21.
    McDonald, J.H.: Handbook of Biological Statistics, vol. 2. Sparky House Publishing, Baltimore (2009)Google Scholar
  22. 22.
    Menzies, T., Caglayan, B., He, Z., Kocaguneli, E., Krall, J., Peters, F., Turhan, B.: The PROMISE Repository of empirical software engineering data, June 2012Google Scholar
  23. 23.
    Wiegers, K., Beatty, J.: Software Requirements. Pearson Education, Harlow (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Feng-Lin Li
    • 1
    Email author
  • Jennifer Horkoff
    • 2
  • Lin Liu
    • 3
  • Alex Borgida
    • 4
  • Giancarlo Guizzardi
    • 5
  • John Mylopoulos
    • 1
  1. 1.University of TrentoTrentoItaly
  2. 2.City UniversityLondonUK
  3. 3.Tsinghua UniversityBeijingChina
  4. 4.Rutgers UniversityNew BrunswickUSA
  5. 5.Federal University of Espirito SantoVitoriaBrazil

Personalised recommendations