Comparing OpenBRR, QSOS, and OMM Assessment Models

  • Etiel Petrinja
  • Alberto Sillitti
  • Giancarlo Succi
Part of the IFIP Advances in Information and Communication Technology book series (IFIPAICT, volume 319)


The objective of this study was to investigate the quality and usability of three Free/Libre Open Source Software assessment models: the Open Business Readiness Rating (OpenBRR), the Qualification and Selection of Open Source software (QSOS), and the QualiPSo OpenSource Maturity Model (OMM). The study identified the positive and negative aspects of each of them. The models were used to assess two Free/Libre Open Source Software projects: Firefox and Chrome (Chromium). The study is based on a set of controlled experiments in which the participants performed the assessment using only one model each. The model used and the Free/Libre Open Source Software project assessed were randomly assigned to the participants. The experiment was conducted in a controlled environment with defined tasks to be performed in a given time interval. The results revealed that the three models provided comparable assessments for the two assessed projects. The main conclusion was that all the three models contain some questions and proposed answers that are not clear to the assessors, therefore should be rewritten or explained better. The critical aspects of each model were: Functionality and Quality for OpenBRR; Adoption, Administration/Monitoring, Copyright owners, and Browser for QSOS ; and Quality of the Test Plan, and the Technical Environment for OMM. Participants perceived the quality and usability of the three models of comparable level.


FLOSS Assessment Model Quality Criteria Software Quality FLOSS Development Process 


  1. 1.
    Amoroso, E., Watson, J., Marietta, M., Weiss, J.: A process-oriented methodology for assessing and improving software trustworthiness. In: Proceedings of the 2nd ACM Conference on Computer and communications security, pp. 39–50. ACM Press, New York (1994)CrossRefGoogle Scholar
  2. 2.
    Origin, A.: Method for Qualification and Selection of Open Source Software (QSOS), (Last visited: December 2009)
  3. 3.
    Duijnhouwer, F.-W., Widdows, C.: Capgemini Expert Letter Open Source Maturity Model, Capgemini (2003)Google Scholar
  4. 4.
    Hannay, J.E., Hansen, O., By Kampenes, V., Karahasanovic, A., Liborg, N., Rekdal, A.: A Survey of Controlled Experiments in Software Engineering. IEEE Trans. Softw. Eng. 31(9), 733–753 (2005), CrossRefGoogle Scholar
  5. 5.
    Kitchenham, B.A., Pfleeger, S.L., Pickard, L.M., Jones, P.W., Hoaglin, D.C., Emam, K.E., Rosenberg, J.: Preliminary guidelines for empirical research in software engineering. IEEE Trans. Softw. Eng. 28(8), 721–734 (2002), CrossRefGoogle Scholar
  6. 6.
    Navica Inc.: The Open Source Maturity Model is a vital tool for planning open source success, (Last visit: December 2009)
  7. 7.
    Petrinja, E., Nambakam, R., Sillitti, A.: Introducing the OpenSource Maturity Model. In: Workshop on Emerging Trends in Free/Libre/Open Source Software Research and Development collocated with 31st International Conference on Software Engineering (ICSE 2009), Vancouver, Canada (2009)Google Scholar
  8. 8.
    Qualipso Consortium: QualiPSo - Quality Platform for Open Source Software, (Last visit December 2007)
  9. 9.
    Taibi, D., Lavazza, L., Morasca, S.: OpenBQR: Aframework for the assessment of OSS. In: Open Source Software 2007, Limerick (June 2007)Google Scholar
  10. 10.
    Wasserman, M.P., Chan, C.: Business Readiness Rating Project, BRR Whitepaper 2005 RFC 1, (Last visited: December 2009)
  11. 11.
    Wittmann, M., Nambakam, R., Ruffati, G., Oltolina, S., Petrinja, E., Ortega, F.: Deliverable A6.D1.6.3: CMM-like model for OSS, (last visited December 2009)
  12. 12.
    Deprez, J.-C., Alexandre, S.: Comparing Assessment Methodologies for Free/Open Source Software: OpenBRR and QSOS, June 2008. LNCS, pp. 189–203. Springer, Berlin (2008)Google Scholar

Copyright information

© IFIP 2010

Authors and Affiliations

  • Etiel Petrinja
    • 1
  • Alberto Sillitti
    • 1
  • Giancarlo Succi
    • 1
  1. 1.Free University of BolzanoItaly

Personalised recommendations