Classsourcing: Crowd-Based Validation of Question-Answer Learning Objects

  • Jakub Šimko
  • Marián Šimko
  • Mária Bieliková
  • Jakub Ševcech
  • Roman Burger
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8083)


The Web 2.0 principles reflect into learning domain and provide means for interactivity and collaboration. Student activities during learning in this environment can be utilized to gather data usable for learning corpora enrichment. It is now a research issue to examine, to what extent the student crowd is reliable in delivering useful artifacts and to bring in suitable tools to enable this. In this paper we present a method for crowd-based validation of question-answer learning objects involving interactive exercise for learners. The method utilizes students’ correctness estimations of answers provided by other students during learning. We show that aggregate student crowd estimations are to big extent comparable to teacher’s evaluations of provided answers.


crowdsourcing education question answer evaluation technology enhanced learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adamic, L.A., Zhang, J., Bakshy, E., Ackerman, M.S.: Knowledge sharing and yahoo answers: everyone knows something. In: Proc. of the 17th Int. Conf. on World Wide Web (WWW 2008), pp. 665–674. ACM, New York (2008)CrossRefGoogle Scholar
  2. 2.
    Agichtein, E., Castillo, C., Donato, D., Gionis, A., Mishne, G.: Finding high-quality content in social media. In: Proc. of the Int. Conf. on Web Search and Web Data Mining, WSDM 2008, pp. 183–194. ACM, New York (2008)CrossRefGoogle Scholar
  3. 3.
    Chen, B.C., Dasgupta, A., Wang, X., Yang, J.: Vote calibration in community question-answering systems. In: Proc. of the 35th Int. ACM SIGIR Conf. on Research and Development in Information Retrieval (SIGIR 2012), pp. 781–790. ACM, New York (2012)CrossRefGoogle Scholar
  4. 4.
    Downes, S.: E-learning 2.0. eLearn magazine 2005, vol. 10 (1). ACM, New York (2005)Google Scholar
  5. 5.
    IEEE LTS: Draft Standard for Learning Object Metadata. IEEE Standard 1484.12.1. IEEE (2002) (retrieved March 2013)Google Scholar
  6. 6.
    Ghauth, K.I., Abdullah, N.A.: The Effect of Incorporating Good Learners’ Ratings in e-Learning Content-based Recommender System. Educational Technology & Society 14(2), 248–257 (2011)Google Scholar
  7. 7.
    Golovchinsky, G., Qvarfordt, P., Pickens, J.: Collaborative information seeking. Information Seeking Support Systems (2008)Google Scholar
  8. 8.
    Kidd, J., O’Shea, P., Baker, P., Kaufman, J., Allen, D.: Student-authored Wikibooks: Textbooks of the Future? In: McFerrin, K., et al. (eds.) Proc. of Society for Information Technology & Teacher Education Int. Conf. 2008, pp. 2644–2647. AACE, Chesapeake (2008)Google Scholar
  9. 9.
    Lawson, M.: Berners-Lee on the read/write web. BBC, Technology (2005), (accessed March 31, 2013)
  10. 10.
    Quinn, A.J., Bederson, B.B.: Human computation: a survey and taxonomy of a growing field. In: Proc. of the 2011 Annual Conf. on Human Factors in Computing Systems (CHI 2011), pp. 1403–1412. ACM, New York (2011)CrossRefGoogle Scholar
  11. 11.
    Stahl, G., Koschmann, T., Suthers, D.: Computer-supported collaborative learning: An historical perspective. In: Sawyer, R.K. (ed.) Cambridge Handbook of the Learning Sciences, pp. 409–426. Cambridge University Press, Cambridge (2006)Google Scholar
  12. 12.
    Šimko, M., Barla, M., Bieliková, M.: ALEF: A Framework for Adaptive Web-based Learning 2.0. In: Reynolds, N., Turcsányi-Szabó, M. (eds.) KCKS 2010. IFIP AICT, vol. 324, pp. 367–378. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. 13.
    Šimko, M., Barla, M., Mihál, V., Unčík, M., Bieliková, M.: Supporting Collaborative Web-Based Education via Annotations. In: Proc. of W. Conf. on Educational Multimedia, Hypermedia & Telecommunications, ED-MEDIA 2011, pp. 2576–2585. AACE (2011)Google Scholar
  14. 14.
    Wheeler, S., Yeomans, P., Wheeler, D.: The good, the bad and the wiki: Evaluating student-generated content for collaborative learning. British Journal of Educational Technology 39(6), 987–995 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Jakub Šimko
    • 1
  • Marián Šimko
    • 1
  • Mária Bieliková
    • 1
  • Jakub Ševcech
    • 1
  • Roman Burger
    • 1
  1. 1.Institute of Informatics and Software EngineeringSlovak University of Technology in BratislavaBratislavaSlovakia

Personalised recommendations