MASECO: A Multi-agent System for Evaluation and Classification of OERs and OCW Based on Quality Criteria

  • Gabriela Moise
  • Monica Vladoiu
  • Zoran Constantinescu
Part of the Studies in Computational Intelligence book series (SCI, volume 528)


Finding effectively open educational resources and open courseware that are the most relevant and that have the best quality for a specific user’s need, in a particular context, becomes more and more demanding. Hence, even though teachers and learners (enrolled students or self-learners as well) get to a greater extent support in finding the right educational resources, they still cannot rely on support for evaluating their quality and relevance, and, therefore, there is a stringent need for effective search and discovery tools that are able to locate high quality educational resources. We propose here a multi-agent system for evaluation and classification of open educational resources and open courseware (called MASECO) based on our socio-constructivist quality model. MASECO supports learners and instructors in their quest for the most appropriate educational resource that fulfills properly their educational needs in a given context. Faculty, educational institutions, developers, and quality assurance experts may also benefit from using it.


Evaluation and classification of open educational resources and open courseware Multi-agent system for classification of open educational resources and open courseware Socio-constructivist quality model for open educational resources and open courseware 



The authors are very grateful to both the editors and the anonymous reviewers for their valuable comments and ideas to improve this chapter.


  1. 1.
    Atkins D., Seely Brown J., Hammonds A.: A review of the open educational resources (OER) movement: achievements, challenges, and new opportunities. (2007)
  2. 2.
    OECD: Giving knowledge for free—the emergence of open educational resources (2007)Google Scholar
  3. 3.
    Hylén, J.: Open educational resources: opportunities and challenges, pp. 49–63, Utah State University, Logan, UT. (2006)
  4. 4.
    Kernohan, D., Thomas A: Open educational resources—a historical perspective.
  5. 5.
    Albright, P.: Discussion highlights, pp. 61-83. In: D’Antoni, S. (ed.) OERs—conversations in cyberspace, UNESCO, Paris.…/OER_Full_Book.pdf (2009)
  6. 6.
  7. 7.
    Vlădoiu, M.: Quality criteria for open courseware and open educational resources. In: 11th International Conference on Web based Learning 2012 (ICWL 2012), Workshops—2nd International Symposium on Knowledge Management and E-Learning (KMEL 2012), LNCS 7697, Sinaia, Romania (2012)Google Scholar
  8. 8.
    Vladoiu, M., Constantinescu, Z.: Evaluation and comparison of three open courseware based on quality criteria. In: Grossniklaus, M., Wimmer, M. (eds.) 12th International Conference on Web Engineering 2012 (ICWE 2012) Workshops—3rd Workshop on Quality in Web Engineering 2012 (QWE 2012), LNCS vol. 7703, pp. 204–215. Springer, Heidelberg (2012)Google Scholar
  9. 9.
    Vlădoiu, M.: Towards assessing quality of open courseware. In: 11th International Conference on Web Based Learning 2012 (ICWL 2012) Workshops—2nd International Workshop on Creative Collaboration through Supportive Technologies in Education (CCSTED 2012), LNCS 7697, Sinaia, Romania (2012)Google Scholar
  10. 10.
    Bloom, B.S., Englehart, M.D., Furst, E.J., Hill, W.H., Krathwohl, D.R.: Taxonomy of Educational Objectives: The Classification of Educational Goals. David McKay, New York (1956)Google Scholar
  11. 11.
    Custard, M., Sumner, T.: Using machine learning to support quality judgments. D-Lib Mag. 11(10), 1082.
  12. 12.
    Nikoi, S., Armellini, A.: The OER mix in higher education: purpose, process, product, and policy. Distance Educ. 33(2), 165–184 (2012)CrossRefGoogle Scholar
  13. 13.
    Wiley, D.: On the sustainability of open educational resource initiatives in higher education, OECD-CERI. (2007)
  14. 14.
    Geser, G.: Open educational practices and resources. OLCOS Roadmap 2012 Recommendations, Salzburg Research, EduMedia Group. Salzburg. (2007)
  15. 15.
    Geser, G.: Open educational practices and resources. OLCOS Roadmap 2012, Revista de Universidad y Sociedad del Conoscimiento 4(1), 4–13 (2007)Google Scholar
  16. 16.
    Downes, S.: Models for sustainable open educational resources. Interdisc. J. Knowl. Learn. Objects 3, 29–44 ( (2007)
  17. 17.
    Bethard, S., Wetzer, P., Butcher, K., Martin, J. H., Sumner, T.: Automatically characterizing resource quality for educational digital libraries. In: 9th ACM/IEEE-CS Joint Conference on Digital Libraries, Austin, TX, USA (2009)Google Scholar
  18. 18.
    Pawlowski, J.M., Hoel, T.: Towards a global policy for open educational resources: the Paris OER Declaration and its implications, White Paper, Version 0.2, Jyväskylä, Finland (2012)Google Scholar
  19. 19.
    Larsen, K., Vincent-Lancrin, S.: The impact of the ICT on tertiary education: advances and promises. In: Kahin, B., Foray, D. (eds.) Advancing Knowledge and the Knowledge Economy. MIT Press, Cambridge (2006)Google Scholar
  20. 20.
    Taylor, P.: Quality and web-based learning objects: towards a more constructive dialogue, in Quality Conversations. In: 25th HERDSA Annual Conference, Perth, Western Australia, pp. 655–662 (2002)Google Scholar
  21. 21.
    Littlejohn, A.: Reusing Online Resources: A Sustainable Approach to E-Learning. Routledge, London (2003)Google Scholar
  22. 22.
    Richter, T., McPherson, M.: Open educational resources: education for the world. Distance Educ. 33(2), 201–219 (2012)CrossRefGoogle Scholar
  23. 23.
    UNESCO: Forum on the impact of open courseware for higher education in developing countries (final report), Paris. (2002)
  24. 24.
    Camilleri, A.F., Ehlers, U.D., Conole, G.: Mainstreaming OEP—recommendations for policy, OPAL Consortium. (2011)
  25. 25.
  26. 26.
    NLN Materials:
  27. 27.
    Kelty, C.M., Burrus, C.S., Baraniuk, R.G.: Peer review anew: three principles and a case study in postpublication quality assurance. Proc. IEEE 96(6), 1000–1011 (2008)CrossRefGoogle Scholar
  28. 28.
  29. 29.
    Rosewell, J., Ferreira, G.: QA in open educational resources (OER): open access to quality teaching resources. European Seminar on QA in e-learning, UNESCO, Paris. (2011)
  30. 30.
    Devedzic, V.: Education and the semantic web. Int. J. Artif. Intell. Educ. 14, 39–65 (2004)Google Scholar
  31. 31.
    Johnson, W.L., Rickel, J., Lester, J.C.: Animated pedagogical agents: face-to-face interaction in interactive learning environments. Int. J. Artif. Intell. Educ. 11, 47–78 (2000)Google Scholar
  32. 32.
    Hassan, S., Mihalcea, R.: Learning to identify educational materials. In: Conference in Recent Advances in Natural Language Processing, pp. 123–127, Borovets, Bulgaria(2009)Google Scholar
  33. 33.
    Hassan, S., Mihalcea, R.: Learning to identify educational materials. ACM Trans. Speech Lang. Process. 8(2), 2–18 (2011)Google Scholar
  34. 34.
    Meyer, M., Hannappel, A., Rensing, C., Steinmetz, R.: Automatic classification of didactic functions of e-learning resources. In: 15th International Conference on Multimedia’07 (MM’07), pp. 513–516, Augsburg, Germany (2007)Google Scholar
  35. 35.
    Meder, N.: Didaktische Ontologien. Globalisierung und Wissensorganisation: Neue Aspekte für Wissen,Wissenschaft und Informationssysteme, 401–416 (2000)Google Scholar
  36. 36.
    Sanz-Rodriguez, J., Dodero Beardo, J., Sánchez-Alonso, S.: Ascertaining the relevance of open educational resources by integrating various quality indicators, RUSC—Revista de Universidad y Sociedad del Conocimiento, 8(2), 211–224 (2011)Google Scholar
  37. 37.
    Han, K.: Quality rating of learning objects using Bayesian belief networks. PhD thesis, Simon Fraser University , Canada (2004)Google Scholar
  38. 38.
    Nesbit, J.C., Li, J.Z., Leacock, T.L.: Web-based tools for collaborative evaluation of learning resources. J. Systemics Cybern. Informatics, 3(5) ( (2005)
  39. 39.
    Burgos Aguilar, J.V.: Rubrics to evaluate OERs. (2011)
  40. 40.
    Kumar, V., et al.: Quality rating and recommendation of learning objects. In: Pierre, S. (ed.) E-Learning Networked Environments and Architectures—A Knowledge Processing Perspective, pp. 337–373. Springer, London (2007)CrossRefGoogle Scholar
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.
    The Saylor Foundation:
  47. 47.
    University of Washington Courses:
  48. 48.
  49. 49.
  50. 50.
  51. 51.
    ParisTech Libres Savoirs:
  52. 52.
  53. 53.
    OCW University of California, Irvine:
  54. 54.
    OCW University of Southern Queensland:
  55. 55.
    OCW Utah State University:
  56. 56.
  57. 57.
  58. 58.
  59. 59.
  60. 60.
  61. 61.
    Saylor Foundation’s Introduction to Modern Database Systems open courseware:
  62. 62.
    Stanford’s Professor Jennifer Widom Introduction to Databases open courseware:
  63. 63.
    Introduction to Database Systems courseware, Nguyen Kim Anh, in Connexions:
  64. 64.
    King Fahd University’s KFUPM OpenCourseWare on Database Systems:
  65. 65.
    University of Washington’s Introduction to Data Management open courseware:
  66. 66.
    Universidad Charlos III de Madrid’s Database Fundamentals:
  67. 67.
    Universidad Politecnica de Madrid’s Database Administration:
  68. 68.
    Vladoiu, M.: State-of-the-art in open courseware initiatives worldwide. Informatics Educ. 10(2), 271–294 (2011)Google Scholar
  69. 69.
    Vladoiu, M.: Open courseware initiatives—after 10 years. In: 10th International Conference Romanian Educational Network—RoEduNet, pp. 183–188. IEEE Press, Iasi (2011)Google Scholar
  70. 70.
    Brockbank, A., McGill, I.: Facilitating Reflective Learning in Higher Education. SRHE and Open University Press, Buckingham (1998)Google Scholar
  71. 71.
    Light, G., Cox, R.: Learning and Teaching in Higher Education. The Reflective Professional. Paul Chapman Publishing, London (2001)Google Scholar
  72. 72.
    Loughran, J.J.: Developing Reflective Practice. Learning About Teaching and Learning Through Modelling. Falmer Press, London (1996)Google Scholar
  73. 73.
    Schunk, D.H., Zimmerman, B.J.: Self-regulated Learning—from Teaching to Self-reflective Practice. Guilford Press, New York (1998)Google Scholar
  74. 74.
    Panait, L., Luke, S.: Cooperative multi-agent learning: the state of the art. Auton. Agents Multi-Agent Syst. 11(3), 387–434 (2005)CrossRefGoogle Scholar
  75. 75.
    Mitchell, T.M.: Machine Learning. McGraw-Hill Higher Education, New York (1997)MATHGoogle Scholar
  76. 76.
    Sebastiani, F.: Machine learning in automated text categorization. ACM Comput. Surv. 34(1), 1–47 (2002)CrossRefGoogle Scholar
  77. 77.
    Rao, A.S., Georgeff, M.P.: Modeling rational agents within a BDI-architecture. In: Fikes, R., Sandewall, E. (eds.) Knowledge Representation and Reasoning (KR&R-91), pp. 473–484. Morgan Kaufmann Publishers, San Mateo (1991)Google Scholar
  78. 78.
    Rao, A.S., Georgeff, M.P.: BDI agents: from theory to practice. In: International Conference on Multi-Agent Systems (ICMAS-95), San Francisco, USA (1995)Google Scholar
  79. 79.
    Müller, J.P.: The Design of Intelligent Agents: A Layered Approach, LNCS, vol. 1177: Lecture notes in artificial intelligence. Springer, Berlin (1996)Google Scholar
  80. 80.
    Moise, G.: A software system for online learning applied in the field of computer science. Int. J. Comput. Commun. Control II(1), 84–93 (2007)Google Scholar
  81. 81.
    Wooldridge, M., Jennings, N.: Intelligent agents: theory and practice. Knowl. Eng. Rev. 10(2), 115–152 (1995)CrossRefGoogle Scholar
  82. 82.
    Sycara, K.P.: Multiagent systems. AI Mag. 19(2), 79–92 (1998)Google Scholar
  83. 83.
    Young T.Y., Calvert, T.W.: Classification, Estimation, and Pattern Recognition. American Elsevier Pub. Co., Amsterdam (1974)Google Scholar
  84. 84.
    Archer, N.P., Wang, S.: Application of the back propagation neural network algorithm with monotonicity constraints for two-group classification problems. Decis. Sci. 24(1), 60–75 (1993)CrossRefGoogle Scholar
  85. 85.
    Guobin, O., Yi, LuM: Multi-class pattern classification using artificial neural networks. Pattern Recogn. 40, 4–18 (2007)CrossRefMATHGoogle Scholar
  86. 86.
    McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 115–133 (1943)MathSciNetCrossRefMATHGoogle Scholar
  87. 87.
    Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386–408 (1958)MathSciNetCrossRefGoogle Scholar
  88. 88.
    Zilouchian, A.: Intelligent control systems using soft computing methodologies. CRC Press, Boca Raton (2001)CrossRefGoogle Scholar
  89. 89.
    Moise, G., Netedu, L., Toader, F.A.: Bio-inspired E-learning systems. A simulation case: english language teaching. In: Pontes, I., Silva, A., Guelfi, A., Takeo Kofuji, S. (eds.) Methodologies, Tools and New Developments for E-Learning, p. 14. InTech, Rijeka (2012)Google Scholar
  90. 90.
    Pearl, J.: Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San Mateo (1988)Google Scholar
  91. 91.
    Bayes, T., Price, R.: An essay towards solving a problem in the doctrine of chance. Philos. Trans. R. Soc. London 53, 370–418. (1763)Google Scholar
  92. 92.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29, 131–163 (1997)CrossRefMATHGoogle Scholar
  93. 93.
    Baesens, B., et al.: Bayesian network classifiers for identifying the slope of the customer lifecycle of long-life customers. EJOR 156(2), 508–523 (2004)MathSciNetCrossRefMATHGoogle Scholar
  94. 94.
    Minsky, M.: Steps toward artificial intelligence. Proc. Inst. Radio Eng. 49(1), 8–30 (1961)MathSciNetGoogle Scholar
  95. 95.
    Piedra, N., Chicaiza, J., Tovar, E., Martinez, O.: Open educational practices and resources based on social software: UTPL experience. In: 9th IEEE International Conference on Advanced Learning Technologies—ICALT 2009, pp. 497–498, Riga (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Gabriela Moise
    • 1
  • Monica Vladoiu
    • 1
  • Zoran Constantinescu
    • 1
  1. 1.UPG University of PloiestiPloiestiRomania

Personalised recommendations