Empirical Software Engineering

, Volume 23, Issue 6, pp 3073–3128 | Cite as

System requirements-OSS components: matching and mismatch resolution practices – an empirical study

  • Claudia AyalaEmail author
  • Anh Nguyen-Duc
  • Xavier Franch
  • Martin Höst
  • Reidar Conradi
  • Daniela Cruzes
  • Muhammad Ali Babar


Developing systems by integrating Open Source Software (OSS) is increasingly gaining importance in the software industry. Although the literature claims that this approach highly impacts Requirements Engineering (RE) practices, there is a lack of empirical evidence to demonstrate this statement. To explore and understand problems and challenges of current system requirement–OSS component matching and mismatches resolution practices in software development projects that integrate one or more OSS components into their software products. Semi-structured in-depth interviews with 25 respondents that have performed RE activities in software development projects that integrate OSS components in 25 different software development companies in Spain, Norway, Sweden, and Denmark. The study uncovers 15 observations regarding system requirements-OSS components matching and mismatch resolution practices used in industrial projects that integrate OSS components. The assessed projects focused mainly on pre-release stages of software applications that integrate OSS components in an opportunistic way. The results also provide details of a set of previously unexplored scenarios when solving system requirement–OSS component mismatches; and clarify some challenges and related problems. For instance, although licensing issues and the potential changes in OSS components by their corresponding communities and/or changes in system requirements have been greatly discussed in the RE literature as problems for OSS component integration, they did not appear to be relevant in our assessed projects. Instead, practitioners highlighted the problem of getting suitable OSS component documentation/information.


Open Source software OSS Requirements engineering Empirical study Qualitative study Survey 



We would like to thank all of the people who participated in piloting an early version of the interview guide and the interview participants who took time from their workdays to participate in our interviews. This work has been partially supported by the Spanish project ref. TIN2016-79269-R.


  1. Alspaugh TA, Scacchi W (2013) Ongoing Software Development without Classical Requirements. RE, p 165–174Google Scholar
  2. Alves CF, Finkelstein A (2003) Investigating Conflicts in Cots Decision-Making. Int J Softw Eng Knowl Eng 13(5):473–493CrossRefGoogle Scholar
  3. Ameller D, Ayala CP, Cabot J, Franch X (2013) Non-functional Requirements in Architectural Decision Making. IEEE Softw 30(2):61–67CrossRefGoogle Scholar
  4. Aminat S, Selamat A, Sahibudin S (2008) Open Source Integration into Business Strategies: A Review. Commun IBIMA 2(17):122–128Google Scholar
  5. Aversano L, Tortorella M (2013) Quality Evaluation of FLOSS projects: Application to ERP systems. Inf Softw Technol 55:1260–1276CrossRefGoogle Scholar
  6. Ayala CP, Hauge Ø, Conradi R, Franch X, Li J (2011) Selection of Third-Party Software in Off-The-Shelf-based Software Development - An interview Study with Industrial Practitioners. J Syst Softw 84(4):620–637CrossRefGoogle Scholar
  7. Badampudi D, Wohlin C, Petersen K (2016) Software Component Decision-Making: In-house, OSS, COTS or Outsourcing - A Systematic Literature Review. J Syst Softw 121:105–124CrossRefGoogle Scholar
  8. Berntsson-Svensson R, Gorschek T, Regnell B, Torkar R, Shahrokni A, Feldt R (2012) Quality Requirements in Industrial Practice - An Extended Interview Study at Eleven Companies. IEEE Trans Softw Eng 38(4):923–935CrossRefGoogle Scholar
  9. Boehm BW (2000) Requirements That Handle IKIWISI, COTS, and Rapid Change. Computer 33(7):99–102CrossRefGoogle Scholar
  10. Boehm BW, Bhuta J (2008) Balancing Opportunities and Risks in Component-Based Software Development. IEEE Softw 25(6):56–63CrossRefGoogle Scholar
  11. Bonaccorsi A, Rossi C (2006) Comparing Motivations of Individual Programmers and Firms to take part in the Open Source Movement. Knowl Technol Policy 18(4):40–64CrossRefGoogle Scholar
  12. Brownsword L, Oberndorf T, Sledge CA (2000) Developing New Processes for COTS-Based Systems. IEEE Softw 17(4):48–55CrossRefGoogle Scholar
  13. Capra E, Francalanci C, Merlo F (2008) An Empirical Study on the Relationship between Software Design Quality, Development Effort and Governance in Open Source Projects. IEEE Trans Softw Eng 34(6):765–782CrossRefGoogle Scholar
  14. Carvallo JP, Franch X, Quer C (2006) Managing Non-technical Requirements in COTS Components Selection. RE, p 323–326Google Scholar
  15. Chen W, Li J, Ma J, Conradi R, Ji J, Liu C (2008) An Empirical Study on Software Development with Open Source Components in the Chinese Software Industry. Softw Process Improv Pract 13(1):89–100CrossRefGoogle Scholar
  16. Ciokolwski M, Soto M (2008) Towards a Comprehensive Approach for Assessing Open Source Projects. IWSM/Metrikon/Mensura, p 316–330Google Scholar
  17. Crowston K, Wei K, Howison J, Wiggins A (2012) Free/Libre Open-Source Software Development: What We Know and What We Do Not Know. ACM Comput Surv 44(2):7CrossRefGoogle Scholar
  18. Cruzes DS, Dybå T (2011) Research Synthesis in Software Engineering: A Tertiary Study. Inf Softw Technol 53(5):440–455CrossRefGoogle Scholar
  19. Cruzes DS, Dybå T, Runeson P, Höst M (2015) Case Studies Synthesis: a Thematic, Cross-case, and Narrative Synthesis Worked Example. Empir Softw Eng J 20(6):1634–1665CrossRefGoogle Scholar
  20. Dagdeviren H, Juric R, Kassana TA (2005) An Exploratory Study for Effective COTS and OSS Product Marketing. ITI, p 644–649Google Scholar
  21. Daneva M, Damian D (2014) A. marchetto, O. Pastor: Empirical Research Methodologies and Studies in Requirements Engineering: How Far Did We Come? J Syst Softw 95:1–9CrossRefGoogle Scholar
  22. Daneva M, Herrmann A, Buglione L (2015) Coping with Quality Requirements in Large, Contract-Based Projects. IEEE Softw 32(6):84–91CrossRefGoogle Scholar
  23. Data Mining Software in Java, ver 3 (n.d.) Available at:
  24. Del Bianco V, Lavazza L, Morasca S, Taibi D (2009) Quality of Open Source Software: The QualiPSo Trustworthiness Model. OSS, p 199–212Google Scholar
  25. Dillon WR, Goldstein M (1984) Multivariate Analysis Methods and Applications. Wiley, HobokenzbMATHGoogle Scholar
  26. Driver M (2013) Hype Cycle for Open-Source Software. Technical Report, GartnerGoogle Scholar
  27. Dybå T (2013) Contextualizing Empirical Evidence. IEEE Softw 30(1):81–83CrossRefGoogle Scholar
  28. Dybå T, Sjøberg DIK, Cruzes DS (2012) What Works for Whom, Where, When, and Why?: On the Role of Context in Empirical Software Engineering. ESEM, p 19–28Google Scholar
  29. Erdogmus H (2010) How important is evidence, really? IEEE Softw 27(3):2–5CrossRefGoogle Scholar
  30. Franch X (2004) Do We Need Requirements in COTS-Based Software Development? ICCBSS, p 11–12CrossRefGoogle Scholar
  31. Golden B (2004) Succeeding with Open Source, Addison-Wesley ProfessionalGoogle Scholar
  32. Hand DJ (1996) Statistics and Theory of Measurements. J Royal Stat Soc 159(3):445–492CrossRefGoogle Scholar
  33. Hansen S, Lyytinen K (2010) Challenges in Contemporary Requirements Practice. HICSS, p 1–11Google Scholar
  34. Hauge Ø, Ayala CP, Conradi R (2010) Adoption of Open Source Software in Software-Intensive Organizations - A Systematic Literature Review. Inf Softw Technol 52(11):1133–1154CrossRefGoogle Scholar
  35. Hoving R, Slot G, Jansen S (2013) Python: Characteristics Identification of a Free Open Source Software Ecosystem. DEST, p 13–18Google Scholar
  36. IBM developerWorks tutorial (n.d.):
  37. International Organization for Standarization (2001) ISO Standard 9126: Software Engineering – Product Quality, part 1. International Organization for StandarizationGoogle Scholar
  38. Jadhav AS, Sonar RM (2009) Evaluating and Selecting Software Packages: A Review. Inf Softw Technol 51(3):555–563CrossRefGoogle Scholar
  39. Jansen S, Brinkkemper S, Hunink I, Demir C (2008) Pragmatic and Opportunistic Reuse in Two Innovative start-up Companies. IEEE Softw Spec Issue Oppor Softw Syst Dev 25(6):42–49Google Scholar
  40. Jarke M, Loucopoulos P, Lyytinen K, Mylopoulos J, Robinson WN (2011) The Brave New World of Design Requirements. Inf Syst 36(7):992–1008CrossRefGoogle Scholar
  41. Kitchenham BA, Dyba T, Jorgensen M (2004) Evidence-Based Software Engineering. ICSE, p 273–281Google Scholar
  42. Kohl RJ (2005) Requirements Engineering Changes for COTS-Intensive Systems. IEEE Softw 22(4):63–64MathSciNetCrossRefGoogle Scholar
  43. Kusumo DS, Staples M, Zhu L, Zhang H, Jeffery R (2012) Risks of Off-The-Shelf-based Software Acquisition and Development: A Systematic Mapping Study and A Survey. EASE, p 233–242Google Scholar
  44. Land R, Sundmark D, Lüders F, Krasteva I, Causevic A (2009) Reuse with Software Components - A Survey of Industrial State of Practice. ICSR p 150–159Google Scholar
  45. Li J, Conradi R, Slyngstad OPN, Torchiano M, Morisio M, Bunse C (2008) A State-of-the-Practice Survey of Risk Management in Development with Off-the-Shelf Software Components. IEEE Trans Softw Eng 34(2):271–286CrossRefGoogle Scholar
  46. Li J, Conradi R, Bunse C, Torchiano M, Slyngstad OPN, Morisio M (2009) Development with Off-The-Shelf Components: 10 Facts. IEEE Softw 26(2):80–87CrossRefGoogle Scholar
  47. Lincoln YS, Guba EG (1985) Naturalistic inquiry. SAGE Publications, Inc., Beverly HillsCrossRefGoogle Scholar
  48. López L, Costal D, Ayala C, Franch X, Annosi M, Glott R, Haaland C (2015) Adoption of OSS components: a Goal-Oriented Approach. Data Knowl Eng 99:17–38CrossRefGoogle Scholar
  49. Mahmood S, Lai R, Kim YS (2007) Survey of Component-Based Software Development. IET Softw 1(2):57–66CrossRefGoogle Scholar
  50. Maiden NAM, Ncube C (1998) Acquiring COTS Software Selection Requirements. IEEE Softw 15(2):46–56CrossRefGoogle Scholar
  51. Majchrowski M, Deprez JC (2008) An Operational Approach for Selecting Open Source Components in a Software Development Project. EuroSPI, p 176-188Google Scholar
  52. Méndez-Fernández D, Wagner S, Lochmann K, Baumann A, de Carne H (2012) Field Study on Requirements Engineering: Investigation of Artefacts, Project Parameters, and Execution Strategies. Inf Softw Technol 54(2):162–178CrossRefGoogle Scholar
  53. Merilinna J, Matinlassi M (2006) State of the Art and Practice of Open-Source Component Integration. EUROMICRO, p 170–177Google Scholar
  54. Mockus A, Fielding RT, Herbsleb JD (2002) Two Case Studies of Open Source Software Development: Apache and Mozilla. ACM Trans Softw Eng Methodol 11(3):309–346CrossRefGoogle Scholar
  55. Mohamed A, Ruhe G, Eberlein A (2007) COTS Selection: Past, Present and Future. ECBS, p 103–114Google Scholar
  56. Mohamed A, Ruhe G, Eberlein A (2011) Mismatch Handling for COTS Selection: a Case Study. J Softw Maint 23(3):145–178CrossRefGoogle Scholar
  57. Morad S, Kuflik T (2005) Conventional and Open Source Software reuse at Orbotech – an Industrial Experience. SWSTE, p 110–117Google Scholar
  58. Morisio M (ed) (2006) Reuse of Off-The-Shelf Components. ICSR2006Google Scholar
  59. Ncube C, Oberndorf P, Kark AW (2008) Opportunistic Software Systems Development: Making Systems from What's Available. IEEE Softw 25(6):38–41CrossRefGoogle Scholar
  60. Nguyen AD, Cruzes DS, Conradi R, Höst M, Franch X, Ayala CP (2012) Collaborative Resolution of Requirements Mismatches When Adopting Open Source Components. REFSQ, p 77–93Google Scholar
  61. Noll J (2008) Requirements Acquisition in Open Source Development: Firefox 2.0. OSS, p 69–79Google Scholar
  62. Noll J, Beecham S, Seichter D (2011) A Qualitative Study of Open Source Software Development: The Open EMR Project. ESEM, p 30–39Google Scholar
  63. Nvivo (2017) tool, further information:
  64. Open Source (2005) Initiative:
  65. OpenBRR (2005) Business Readiness Rating for Open Source a Proposed Open Standard to Facilitate Assessment and Adoption of Open Source Software. _2005RFC1.pdf, Request for Comments
  66. Paech B, Reuschenbach B (2006) Open Source Requirements Engineering. RE, p 252-259Google Scholar
  67. Perrone V (2004) A Wish List for Requirements Engineering for COTS-Based Information Systems. ICCBSS, p 146–158Google Scholar
  68. Raw data from this study (n.d.) is available at:
  69. Robson C (2002) Real World Research: A Resource for Social Scientists and Practitioner-researchers. Second Edition. Blackwell Publishers Inc, OxfordGoogle Scholar
  70. Ross DT, Schoman KE Jr (1977) tructured Analysis for Requirements Definition. IEEE Trans Softw Eng 3(1):6–15CrossRefGoogle Scholar
  71. Rubython A, Maiden N (2014) The Effect of Variability Modeling on Requirements Satisfaction for the Configuration and Implementation of Off-The-Shelf Software Packages. RE, p 394–401Google Scholar
  72. Runeson P, Höst M (2009) Guidelines for Conducting and Reporting Case Study Research in Software Engineering. Empir Softw Eng 14(2):131–164CrossRefGoogle Scholar
  73. Scacchi W (2002) Understanding the Requirements for Developing Open Source Software Systems. IEE Proc Softw 149(1):24–39CrossRefGoogle Scholar
  74. Scacchi W, Feller J, Fitzgerald B, Hissam SA, Lakhani K (2006) Understanding Free/Open Source Software Development Processes. Softw Process Improv Pract 11(2):95–105CrossRefGoogle Scholar
  75. Schuwer R, van Genuchten M, Hatton L (2015) On the Impact of Being Open. IEEE Softw 32(5):81–83CrossRefGoogle Scholar
  76. Semeteys R, Pilot O, Baudrillard L, Le Bouder G, Pinkhardt W (2006) Method for Qualification and Selection of Open Source software (QSOS) version 1.6, Technical report, Atos OriginGoogle Scholar
  77. Sen R, Subramaniam C, Nelson ML (2011) Open Source Software Licenses: Strong-Copyleft, Non-Copyleft, or Somewhere in Between? Decis Support Syst 52(1):199–206CrossRefGoogle Scholar
  78. Sen R, Singh SS, Borle S (2012) Open Source Software Success: Measures and Analysis. Decis Support Syst 52(2):364–372CrossRefGoogle Scholar
  79. ISO/IEC TR 19759 (2005) Software Engineering - Guide to the Software Engineering Body of Knowledge (SWEBOK). Available at:
  80. Spinellis D, Giannikas V (2012) Organizational Adoption of Open Source Software. J Syst Softw 85(3):666–682CrossRefGoogle Scholar
  81. Stewart KJ, Ammeter AP, Maruping LM (2006) Impact of License Choice and Organizational Sponsorship on Success in Open Source Software Development Projects. Inf Syst Res 17(2):126–144CrossRefGoogle Scholar
  82. Stol K, Ali Babar M (2010) Challenges in Using Open Source Software in Product Development: a Review of the Literature. Workshop on Emerging Trends in FLOSS Research and Development, p 17–22Google Scholar
  83. Stol K-J, Fitzgerald B (2015) Inner Source- Adopting Open Source Development Practices in Organizations: A Tutorial. IEEE Softw 32(4):60–67CrossRefGoogle Scholar
  84. Torchiano M, Morisio M (2004) Overlooked Aspects of COTS-Based Development. IEEE Softw 21(2):88–93CrossRefGoogle Scholar
  85. Vale T, Crnkovic I, Santana de Almeida E, da Mota Silveira Neto PA, Cerqueira Cavalcanti Y, Romero de Lemos Meira S (2016) Twenty-eight years of Component-Based Software Engineering. J Syst Softw 111:128–148CrossRefGoogle Scholar
  86. Ven K, Verelst J, Mannaert H (2008) Should You Adopt Open Source Software? IEEE Softw 25(3):54–59CrossRefGoogle Scholar
  87. Wesselius JH (2008) The Bazaar Inside the Cathedral: Business Models for Internal Markets. IEEE Softw 25(3):60–66CrossRefGoogle Scholar
  88. Wohlin C, Runeson P, Host M, Ohlsson MC, Regnell B, Wesslen A (2012) Experimentation in Software Engineering. Springer, BerlinCrossRefGoogle Scholar
  89. Yang Y, Bhuta J, Boehm BW, Port DN (2005) Value-Based Processes for COTS-Based Applications. IEEE Softw 22(4):54–62CrossRefGoogle Scholar
  90. Yin RK (1994) Case Study Research. Design and Methods. Second edition. Sage, Thousand OaksGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Claudia Ayala
    • 1
    Email author
  • Anh Nguyen-Duc
    • 2
  • Xavier Franch
    • 1
  • Martin Höst
    • 3
  • Reidar Conradi
    • 2
  • Daniela Cruzes
    • 4
  • Muhammad Ali Babar
    • 5
  1. 1.Technical University of Catalunya (UPC)BarcelonaSpain
  2. 2.Norwegian University of Science and Technology (NTNU)TrondheimNorway
  3. 3.University of LundLundSweden
  4. 4.SINTEF ICTTrondheimNorway
  5. 5.University of AdelaideAdelaideAustralia

Personalised recommendations