An Empirical Study of the Software Development Process, Including Its Requirements Engineering, at Very Large Organization: How to Use Data Mining in Such a Study

  • Colin M. WernerEmail author
  • Daniel M. Berry
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 809)


Very Large Organization (VLO) develops and manufacturers hardware and software products, with each product being developed in its own project. Each project, from its inception, maintains a database that contains a wealth of data pertaining to its software development lifecycle. To empirically study VLO’s software development process, the authors mined the data from seven consecutive VLO projects to determine whether the data exhibit any anomalies and whether these anomalies can help assess a project’s level of success. Some anomalies provide evidence of what VLO does well, while other anomalies highlight possible areas of improvement. Through the anomalies in the mined data, the organization can direct additional focus and research to specific areas of the development process, particularly its requirements engineering, to improve the likelihood of success for future projects.

While describing the results of the empirical study, the paper also shows how such a study can be conducted even when the mined data are not very detailed.


Code repositories Industry Project management Prototyping Requirements engineering Software engineering Empirical case study 



Daniel Berry’s work was supported in part by a Canadian NSERC grant NSERC-RGPIN227055-15.


  1. 1.
    Hassan, A.E.: The road ahead for mining software repositories. In: Frontiers of Software Maintenance (FOSM), pp. 48–57 (2008)Google Scholar
  2. 2.
    Werner, C.M.: A case study of a very large organization. Master’s thesis, University of Waterloo, Waterloo (2012)Google Scholar
  3. 3.
    Agrawal, K., Aschauer, M., Thonhofer, T., Bala, S., Rogge-Solti, A., Tomsich, N.: Resource classification from version control system logs. In: IEEE 20th International Enterprise Distributed Object Computing Workshop (EDOCW), pp. 1–10 (2016)Google Scholar
  4. 4.
    Graves, T.L., Karr, A.F., Marron, J.S., Siy, H.: Predicting fault incidence using software change history. IEEE Trans. Software Eng. 26(7), 653–661 (2000)CrossRefGoogle Scholar
  5. 5.
    Engler, D., Chen, D.Y., Hallem, S., Chou, A., Chelf, B.: Bugs as deviant behavior: a general approach to inferring errors in systems code. In: 18th ACM Symposium on Operating Systems Principles (SOSP), pp. 57–72 (2001)Google Scholar
  6. 6.
    Charette, R.N.: Why software fails [software failure]. IEEE Spectr. 42(9), 42–49 (2005)CrossRefGoogle Scholar
  7. 7.
    Ruhe, G., Greer, D.: Quantitative studies in software release planning under risk and re- source constraints. In: International Symposium on Empirical Software Engineering (IS-ESE), pp. 262–270 (2003)Google Scholar
  8. 8.
    Hooper, J.W., Hsia, P.: Scenario-based prototyping for requirements identification. SIG- SOFT Softw. Eng. Notes 7(5), 88–93 (1982)CrossRefGoogle Scholar
  9. 9.
    Davis, A.M.: Operational prototyping: a new development approach. IEEE Softw. 9(5), 70–78 (1992)CrossRefGoogle Scholar
  10. 10.
    Li, P.L., Kivett, R., Zhan, Z., Jeon, S.-e., Nagappan, N., Murphy, B., Ko, A.J.: Characterizing the differences between pre- and post- release versions of software. In: 33rd International Conference on Software Engineering (ICSE), pp. 716–725 (2011)Google Scholar
  11. 11.
    Coldrick, S., Lawson, C.P., Ivey, P.C., Lockwood, C.: A decision framework for R&D project selection. In: IEEE International Engineering Management Conference (IEMC), vol. 1, pp. 413–418 (2002)Google Scholar
  12. 12.
    Bowers, J., Pycock, J.: Talking through design: requirements and resistance in cooperative prototyping. In: SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 299–305 (1994)Google Scholar
  13. 13.
    DeMarco, T.: All late projects are the same. IEEE Softw. 28(6), 104 (2011)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Ferens, D.V.: The conundrum of software estimation models. IEEE Aerosp. Electron. Syst. Mag. 14(3), 23–29 (1999)CrossRefGoogle Scholar
  15. 15.
    Grimstad, S.: Understanding of estimation accuracy in software development projects. In: 11th IEEE International Software Metrics Symposium (METRICS), pp. 2–42 (2005)Google Scholar
  16. 16.
    Moloekken-Oestvold, K., Joergensen, M., Tanilkan, S.S., Gallis, H., Lien, A.C., Hove, S.W.: A survey on software estimation in the norwegian industry. In: International Symposium on Software Metrics, pp. 208–219 (2004)Google Scholar
  17. 17.
    Ellis, K., Berry, D.M.: Quantifying the impact of requirements definition and management process maturity on project outcome in large business application development. Requirements Eng. J. 18(3), 223–249 (2013)CrossRefGoogle Scholar
  18. 18.
    Damian, D., Chisan, J., Vaidyanathasamy, L., Pal, Y.: Requirements engineering and down- stream software development: findings from a case study. Empirical Softw. Eng. 10(3), 255–283 (2005)CrossRefGoogle Scholar
  19. 19.
    Blyth, A.J.C., Chudge, J., Dobson, J.E., Strens, M.R.: A framework for modelling evolving requirements. In: 17th Annual International Computer Software and Applications Conference (COMPSAC), pp. 83–89 (1993)Google Scholar
  20. 20.
    Liou, Y.I., Chen, M.: Integrating group support systems, joint application development, and computer-aided software engineering for requirements specification. In: 26th Hawaii International Conference on System Sciences (HICSS), vol. 3, pp. 4–12 (1993)Google Scholar
  21. 21.
    Bowles, J.B.: Better software reliability by getting the requirements right. In: Annual Reliability and Maintainability Symposium (RAMS), pp. 110–115 (2006)Google Scholar
  22. 22.
    Moloekken-Oestvold, K., Joergensen, M.: A comparison of software project overruns — flexible versus sequential development models. IEEE Trans. Software Eng. 31(9), 754–766 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Cheriton School of Computer ScienceUniversity of WaterlooWaterlooCanada

Personalised recommendations