An Empirical Evaluation of Open Source in Telecommunications Software Development: The Good, the Bad, and the Ugly

  • Rolando P. Reyes Ch.
  • Efraín R. Fonseca C.
  • John W. Castro
  • Hugo Pérez Vaca
  • Manolo Paredes Calderón
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 721)


Software development for the communication networks ‘monitoring is usually based on Open Source software components, as an effective and low cost technological option. However, when we evaluated a product developed with Open Source components, we found that its efficiency is less than other similar Open Source software developed with proprietary tools; which is unusual or at least it isn’t expected. To the best of our knowledge this phenomenon has not been reported in the literature. Hence, our aim was to identify the circumstances that explain why the efficiency of Open Source software applications tends to be less than Open Source applications developed with proprietary software tools. A controlled experiment was performed at Universidad de las Fuerzas Armadas ESPE of Ecuador to compare the performance of two software tools for communication networks’ monitoring. A post hoc analysis reveled that some causal relationships that could explain the unexpected behavior of compared applications’ efficiency. From the statistical perspective, there is no significant difference in effectiveness between Open Source and proprieta- ry applications for communication networks’ monitoring. Efficiency of the Open Source tools depends on a large extent of software components used for their integration, which apparently is not considered in the development of this kind of applications.


Open Source Software engineering Experimentation Empirical evaluation Monitoring software Communication networks 



This work was supported by the “Laboratorio Industrial en Ingeniería del Software Empírica (LI2SE)”research project of the Universidad de las Fuerzas Armadas ESPE of Ecuador.


  1. 1.
    Adams, B., Kavanagh, R., Hassan, A.E., German, D.M.: An empirical study of integration activities in distributions of open source software. Empirical Softw. Eng. 21(3), 960–1001 (2016)CrossRefGoogle Scholar
  2. 2.
    Antichi, G., Shahbaz, M., Geng, Y., Zilberman, N., Covington, A., Bruyere, M., McKeown, N., Feamster, N., Felderman, B., Blott, M., et al.: OSNT: open source network tester. IEEE Netw. 28(5), 6–12 (2014)CrossRefGoogle Scholar
  3. 3.
    Bastian, M., Heymann, S., Jacomy, M., et al.: Gephi: an open source software for exploring and manipulating networks. In: ICWSM, vol. 8, pp. 361–362 (2009)Google Scholar
  4. 4.
    Chiarani, M.C., Pianucci, I.G., Lucero, M.M.: Criterios de evaluación de plataformas virtuales de código abierto para ambientes de aprendizajes colaborativos. In: VI Workshop de Investigadores en Ciencias de la Computación (2004)Google Scholar
  5. 5.
    Chowdhury, S.R., Bari, M.F., Ahmed, R., Boutaba, R.: Payless: a low cost network monitoring framework for software defined networks. In: 2014 IEEE Network Operations and Management Symposium (NOMS), pp. 1–9. IEEE (2014)Google Scholar
  6. 6.
    DiBona, C., Ockman, S.: Open Sources: Voices from the Open Source Revolution. O’Reilly Media Inc., Sebastobol (1999)zbMATHGoogle Scholar
  7. 7.
    Frakes, W.B., Kang, K.: Software reuse research: status and future. IEEE Trans. Software Eng. 31(7), 529–536 (2005)CrossRefGoogle Scholar
  8. 8.
    Gaffney, J.E., Durek, T.A.: Software reuse key to enhanced productivity: some quantitative models. Inf. Softw. Technol. 31(5), 258–267 (1989)CrossRefGoogle Scholar
  9. 9.
    Hahn, J., Moon, J.Y., Zhang, C.: Emergence of new project teams from open source software developer networks: impact of prior collaboration ties. Inf. Syst. Res. 19(3), 369–391 (2008)CrossRefGoogle Scholar
  10. 10.
    Hauge, Ø., Cruzes, D., Conradi, R., Velle, K., Skarpenes, T.: Risks and risk mitigation in open source software adoption: bridging the gap between literature and practice. Open Source Software: New Horizons, pp. 105–118 (2010)Google Scholar
  11. 11.
    Jaaksi, A.: Experiences on product development with open source software. International Federation for Information Processing. IFIP, vol. 234, pp. 85–96 (2007)Google Scholar
  12. 12.
    Li, H., Tesfatsion, L.: Development of open source software for power market research: the ames test bed. J. Energy Markets 2(2), 111 (2009)CrossRefGoogle Scholar
  13. 13.
    Meneely, A., Williams, L.: Secure open source collaboration: an empirical study of Linus’ law. In: Proceedings of the 16th ACM Conference on Computer and Communications Security, pp. 453–462. ACM (2009)Google Scholar
  14. 14.
    Mockus, A., Fielding, R.T., Herbsleb, J.D.: Two case studies of open source software development: apache and mozilla. ACM Trans. Softw. Eng. Methodol. (TOSEM) 11(3), 309–346 (2002)CrossRefGoogle Scholar
  15. 15.
    Oetiker, T.: Monitoring your it gear: the mrtg story. IT Prof. 3(6), 44–48 (2001)CrossRefGoogle Scholar
  16. 16.
    Paulson, J.W., Succi, G., Eberlein, A.: An empirical study of open-source and closed-source software products. IEEE Trans. Software Eng. 30(4), 246–256 (2004)CrossRefGoogle Scholar
  17. 17.
    Stol, K.J., Babar, M.A., Avgeriou, P., Fitzgerald, B.: A comparative study of challenges in integrating open source software and inner source software. Inf. Softw. Technol. 53(12), 1319–1336 (2011)CrossRefGoogle Scholar
  18. 18.
    Szyperski, C.: Emerging component software technologies: a strategic comparison. Softw.-Concepts Tools 19(1), 2–10 (1998)CrossRefGoogle Scholar
  19. 19.
    Van Adrichem, N.L., Doerr, C., Kuipers, F.A.: Opennetmon: network monitoring in openflow software-defined networks. In: 2014 IEEE Network Operations and Management Symposium (NOMS), pp. 1–8. IEEE (2014)Google Scholar
  20. 20.
    Ven, K., Mannaert, H.: Challenges and strategies in the use of open source software by independent software vendors. Inf. Softw. Technol. 50(9), 991–1002 (2008)CrossRefGoogle Scholar
  21. 21.
    Ven, K., Verelst, J., Mannaert, H.: Should you adopt open source software? IEEE Softw. 25(3), 54–59 (2008)Google Scholar
  22. 22.
    Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer, Heidelberg (2012)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Rolando P. Reyes Ch.
    • 1
    • 4
  • Efraín R. Fonseca C.
    • 2
  • John W. Castro
    • 3
  • Hugo Pérez Vaca
    • 1
  • Manolo Paredes Calderón
    • 4
  1. 1.Departamento de Seguridad y DefensaUniversidad de las Fuerzas Armadas ESPESangolquíEcuador
  2. 2.Departamento de Ciencias de la ComputaciónUniversidad de las Fuerzas Armadas ESPESangolquíEcuador
  3. 3.Departamento de Ingeniería Informática y Ciencias de la ComputaciónUniversidad de AtacamaCopiapóChile
  4. 4.Centro de Investigación Científica y Tecnológica del EjércitoUniversidad de las Fuerzas Armadas ESPESangolquíEcuador

Personalised recommendations