Education and Information Technologies

, Volume 22, Issue 6, pp 3207–3229 | Cite as

An empirical verification of a-priori learning models on mailing archives in the context of online learning activities of participants in free\libre open source software (FLOSS) communities

  • Patrick MukalaEmail author
  • Antonio Cerone
  • Franco Turini


Free\Libre Open Source Software (FLOSS) environments are increasingly dubbed as learning environments where practical software engineering skills can be acquired. Numerous studies have extensively investigated how knowledge is acquired in these environments through a collaborative learning model that define a learning process. Such a learning process, identified either as a result of surveys or by means of questionnaires, can be depicted through a series of graphical representations indicating the steps FLOSS community members go through as they acquire and exchange skills. These representations are referred to as a-priori learning models. They are Petri net-like workflow nets (WF-net) that provide a visual representation of the learning process as it is expected to occur. These models are representations of a learning framework or paradigm in FLOSS communities. As such, the credibility of any models is estimated through a process of model verification and validation. Therefore in this paper, we analyze these models in comparison with the real behavior captured in FLOSS repositories by means of conformance verification in process mining. The purpose of our study is twofold. Firstly, the results of our analysis provide insights on the possible discrepancies that are observed between the initial theoretical representations of learning processes and the real behavior captured in FLOSS event logs, constructed from mailing archives. Secondly, this comparison helps foster the understanding on how learning actually takes place in FLOSS environments based on empirical evidence directly from the data.


A-priori learning models Conformance checking Learning activities in FLOSS communities Open source software Process mining 


  1. Alur, R., Henzinger, T. A., & Vardi, M. Y. (2015). Theory in practice for system design and verification. ACM Siglog News, 2(1), 46–51.Google Scholar
  2. Anderson, T. (2004). Teaching in an online learning context. Theory and practice of online learning, 273.Google Scholar
  3. Arbaugh, J. B. (2007). An empirical verification of the Community of Inquiry Framework. Journal of Asynchronous Learning Networks, 11(1), 73–85.Google Scholar
  4. Bartoletti, M., Degano, P., Ferrari, G. L., & Zunino, R. (2015). Model checking usage policies. Mathematical Structures in Computer Science, 25(03), 710–763.MathSciNetCrossRefzbMATHGoogle Scholar
  5. Bojanowski, C. (2016). Verification, validation and optimization of finite element model of bus structure for rollover test.Google Scholar
  6. Carson, J. S. (2002, December). Model verification and validation. In Simulation Conference, 2002. Proceedings of the Winter (Vol. 1, pp. 52–58). IEEE.Google Scholar
  7. Cerone, A. (2011). Learning and activity patterns in OSS communities and their impact on software quality. In Proc. of the 5th International Workshop on Foundations and Techniques for Open Source Software Certification. (OpenCert 2011). Vol 48 of electronic communications of the EASST. The European Association of Software Science and Technology.Google Scholar
  8. Cerone, A. K., & Sowe, S. K. (2010). Using Free/Libre Open Source Software Projects as E-learning Tools. In Proc. of the 4th International Workshop on Foundations and Techniques for Open Source Software Certification. (OpenCert 2010), Vol. 33 of Electronic Communications of the EASST. The European Association of Software Science and Technology.Google Scholar
  9. Elliott, M. S., & Scacchi, W. (2005). Free software development: cooperation and conflict in. Free/Open Source Software Development, 152Google Scholar
  10. Fernandes, S., Cerone, A., & Barbosa, L. S. (2013). FLOSS Communities as Learning Networks. Int. Journal of Information and Education Technology, 3(2), pages 278–281, IACSIT Press, April 2013.Google Scholar
  11. Fernandes, S., Cerone, A., & Barbosa, L. S. (2014a). Analysis of FLOSS Communities as Learning Contexts. Proc. of the 7th International Workshop on Foundations and Techniques for Open Source Software Certification. (OpenCert 2013), Vol. 8368 of Lecture Notes in Computer Science, pages 405–416, Springer.Google Scholar
  12. Fernandes, S., Cerone, A., Barbosa, L. S., & Papadopoulos, P. M. (2014b). Proc. of the 1st International Symposium on Innovation and Sustainability in Education (InSuEdu 2012), Vol. 7991 of Lecture Notes in Computer Science, pages 121–132. Springer.Google Scholar
  13. Glott, R., Meiszner, A., & Sowe, S. K. (2007). FLOSSCom phase 1 report: analysis of the informal learning environment of FLOSS communities. FLOSSCom Project.Google Scholar
  14. Glott, R., SPI, A. M., Sowe, S. K., Conolly, T., Healy, A., Ghosh, R., ... & West, D. (2011)FLOSSCom-Using the Principles of Informal Learning Environments of FLOSS Communities to Improve ICT Supported Formal Education.Google Scholar
  15. Greller, W., & Drachsler, H. (2012). Translating learning into numbers: a generic framework for learning analytics. Educational Technology & Society, 15(3), 42–57.Google Scholar
  16. Hicks, J. L., Uchida, T. K., Seth, A., Rajagopal, A., & Delp, S. L. (2015). Is my model good enough? Best practices for verification and validation of musculoskeletal models and simulations of movement. Journal of Biomechanical Engineering, 137(2), 020905.CrossRefGoogle Scholar
  17. Kwiatkowska, M., Norman, G., & Parker, D. (2011, July). PRISM 4.0: Verification of probabilistic real-time systems. In International Conference on Computer Aided Verification (pp. 585–591). Springer Berlin Heidelberg.Google Scholar
  18. Lakhani, K. R., & Von Hippel, E. (2003). How open source software works:“free” user-to-user assistance. Research Policy, 32(6), 923–943.CrossRefGoogle Scholar
  19. Larson, B. (2012). Delivering business intelligence with Microsoft SQL server 2012. McGraw-Hill Osborne Media.Google Scholar
  20. Meiszner, A., Glott, R., & Sowe, S. K. (2008). Free/libre open source software (FLOSS) communities as an example of successful open participatory learning ecosystems. UPGRADE, The European Journal for the Informatics Professional, 9(3), 62–68.Google Scholar
  21. Meiszner, A., Stamelos, I., & Sowe, S. K. (2009). 1st international workshop on:‘designing for participatory Learning’Building from open source success to develop free ways to share and learn. In Open Source Ecosystems: Diverse Communities Interacting (pp. 355–356). Springer Berlin Heidelberg.Google Scholar
  22. Mukala, P., Cerone, A., & Turini, F. (2014a). An Abstract State Machine (ASM) Representation of Learning Process in FLOSS Communities. Proc. of the 3rd International Symposium on Modelling and Knowledge Management for Sustainable Development (MoKMaSD 2014), Springer.Google Scholar
  23. Mukala, P., Cerone, A., & Turini, F. (2014b). Ontolifloss: An ontology of learning in FLOSS. Proc. of the 3rd International Symposium on Modelling and Knowledge Management for Sustainable Development (MoKMaSD 2014), Springer. In PressGoogle Scholar
  24. Mukala, P., Cerone, A., & Turini, F. (2015a). An exploration of learning processes as process maps in FLOSS repositories. In Internal technical reports 2015, Department of Computer science. Pisa: University of.Google Scholar
  25. Mukala, P., Buijs, J.C.A.M., Leemans, M., van der Aalst, W.M.P. (2015b): Learning analytics on Coursera event data: a process mining approach. In proceedings of 5th SIMPDA, pp. 18–32 (2015)Google Scholar
  26. Murdock, J., & Allen, C. (2015, January). Visualization Techniques for Topic Model Checking. In AAAI (pp. 4284–4285).Google Scholar
  27. OpenStack. (2014, November 3). In Wikipedia, The Free Encyclopedia. Retrieved 15:48, November 10, 2014, from
  28. Ozkan, S., & Koseler, R. (2009). Multi-dimensional students’ evaluation of e-learning systems in the higher education context: an empirical investigation. Computers & Education, 53(4), 1285–1296.CrossRefGoogle Scholar
  29. Papadopoulos, P. M., Stamelos, I. G., & Meiszner, A. (2013). Enhancing software engineering education through open source projects: four years of students’ perspectives. Education and Information Technologies, 18, 381.CrossRefGoogle Scholar
  30. Paszkiewicz, Z. (2013, January). Process Mining Techniques in Conformance Testing of Inventory Processes: An Industrial Application. In Business Information Systems Workshops (pp. 302–313). Springer Berlin HeidelbergGoogle Scholar
  31. Poncin, W., Serebrenik, A., & van den Brand, M. (2011, March). Process mining software repositories. The 15th European Conference on Software Maintenance and Reengineering ({CSMR} 2011) took place in Oldenburg, Germany from 1-4 March 2011.Google Scholar
  32. Rozinat, A., & van der Aalst, W. M. (2008). Conformance checking of processes based on monitoring real behavior. Information Systems, 33(1), 64–95.CrossRefGoogle Scholar
  33. Shea, P., & Bidjerano, T. (2009). Community of inquiry as a theoretical framework to foster “epistemic engagement” and “cognitive presence” in online education. Computers & Education, 52(3), 543–553.CrossRefGoogle Scholar
  34. Singh, V., Nichols, D. M., & Twidale, M. B. (2009). Users of open source software: how do they get help? Washington, DC: IEEE Computer Society.Google Scholar
  35. Sowe, S. K., & Stamelos, I. G. (2007). Involving software engineering students in open source software projects: experiences from a pilot study. Journal of Information Systems Education, 18(4), 425.Google Scholar
  36. Sowe, S. K., & Stamelos, I. (2008). Reflection on Knowledge Sharing in F/OSS Projects. In Open Source Development, Communities and Quality (pp. 351–358). Springer US.Google Scholar
  37. Steehouder, M. F. (2002). Beyond technical documentation: users helping each other. In Professional Communication Conference, 2002. IPCC 2002. Proceedings. IEEE International (pp. 489–499). IEEEGoogle Scholar
  38. Thacker, B. H., Doebling, S. W., Hemez, F. M., Anderson, M. C., Pepin, J. E., & Rodriguez, E. A. (2004). Concepts of model verification and validation (no. LA-14167). Los Alamos: Los Alamos National Lab.CrossRefGoogle Scholar
  39. van der Aalst, W. M. (2005). Business alignment: using process mining as a tool for Delta analysis and conformance testing. Requirements Engineering, 10(3), 198–211.CrossRefGoogle Scholar
  40. van der Aalst, W. M., & de Medeiros, A. K. A. (2005). Process mining and security: detecting anomalous process executions and checking process conformance. Electronic Notes in Theoretical Computer Science, 121, 3–21.CrossRefzbMATHGoogle Scholar
  41. Walton, M., Boley, L., Keenan, L., Kerstman, E., Shah, R., Young, M., et al. (2015). Integrated medical model (IMM) project verification, validation, and credibility (VVandC). Washington, DC: NASA.Google Scholar
  42. Wilson, G., & Stacey, E. (2004). Online interaction impacts on learning: teaching the teachers to teach online. Australasian Journal of Educational Technology, 20(1), 33–48.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of PisaPisaItaly
  2. 2.School of Science and TechnologyNazarbayev UniversityAstanaKazakhstan

Personalised recommendations