Abstract
Predicting knowledge levels from user’s implicit interactions with an adaptive system is a difficult task, particularly in learning systems that are used in the context of daily work tasks. We have collected interactions of six persons working with the adaptive work-integrated learning system APOSDLE over a period of two months to find out whether naturally occurring interactions with the system can be used to predict their level of expertise. One set of interactions is based on the tasks they performed, the other on a number of additional Knowledge Indicating Events (KIE). We find that the addition of KIE significantly improves the prediction as compared to using tasks only. Both approaches are superior to a model that uses only the frequencies of events.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Brusilovsky, P., Millán, E.: User Models for Adaptive Hypermedia and Adaptive Educational Systems. In: Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.) Adaptive Web 2007. LNCS, vol. 4321, pp. 3–53. Springer, Heidelberg (2007)
Ley, T., Kump, B., Albert, D.: A methodology for eliciting, modelling, and evaluating expert knowledge for an adaptive work-integrated learning system. International Journal of Human-Computer Studies 68, 185–208 (2010)
Lindstaedt, S., Kump, B., Beham, G., Pammer, V., Ley, T., Dotan, A., de Hoog, R.: Providing varying degrees of guidance for work-integrated learning. In: Wolpers, M., Kirschner, P.A., Scheffel, M., Lindstaedt, S., Dimitrova, V. (eds.) EC-TEL 2010. LNCS, vol. 6383, pp. 213–228. Springer, Heidelberg (2010)
Lindstaedt, S.N., Beham, G., Kump, B., Ley, T.: Getting to know your user - Unobtrusive user model maintenance within work-integrated learning environments. In: Cress, U., Dimitrova, V., Specht, M. (eds.) EC-TEL 2009. LNCS, vol. 5794, pp. 73–87. Springer, Heidelberg (2009)
Kump, B., Seifert, C., Beham, G., Lindstaedt, S.N., Ley, T.: Seeing what the system thinks you know. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, LAK 2012, pp. 153–157. ACM Press, New York (2012)
Brusilovsky, P., Kobsa, A., Vassileva, J. (eds.): Adaptive hypertext and hypermedia. Kluwer Academic Publishers, Dordrecht (1998)
Poulson, M.C., Richardson, J.J. (eds.): Foundations of intelligent tutoring systems. Lawrence Erlbaum Associates, Hillsdale (1988)
Desmarais, M.C., Baker, R.S.J.: A review of recent advances in learner and skill modeling in intelligent learning environments. User Modeling and User-adapted Interaction 22, 9–38 (2011)
Anderson, J.R., Boyle, C.F., Corbett, A.T., Lewis, M.W.: Cognitive modeling and intelligent tutoring. Artificial Intelligence 42, 7–49 (1990)
Ritter, S., Anderson, J.R., Koedinger, K.R., Corbett, A.T.: Cognitive tutor: Applied research in mathematics education. Psychonomic Bulletin & Review 14, 249–255 (2007)
Mitrovic, A.: Fifteen years of constraint-based tutors: What we have achieved and where we are going. User Modeling and User-Adapted Interaction 22, 39–72 (2011)
Heller, J., Steiner, C., Hockemeyer, C., Albert, D.: Competence-based knowledge structures for personalised learning. International Journal on E-Learning 5, 75–88 (2006)
Conati, C., Gertner, A.S., VanLehn, K.: Using Bayesian Networks to Manage Uncertainty in Student Modeling. User Modeling and User-Adapted Interaction 12, 371–417 (2002)
De Bra, P., Aroyo, L., Cristea, A.: Adaptive web-based educational hypermedia. In: Web Dynamics: Adapting to Change in Content, Size, Topology and Use, pp. 387–410 (2004)
Limongelli, C., Sciarrone, F., Temperini, M., Vaste, G.: Adaptive learning with the LS-Plan dystem: A field evaluation. IEEE Transactions on Learning Technologies 2, 203–215 (2009)
De Bra, P., Smits, D., Stash, N.: The Design of AHA? In: Proceedings of the ACM Conference on Hypertext and Hypermedia, Odense, Denmark, p. 133 (2006)
Brusilovsky, P., Schwarz, E., Weber, G.: ELM-ART: An intelligent tutoring system on World Wide Web. In: Frasson, C., Gauthier, G., Lesgold, A. (eds.) ITS 1996. LNCS, vol. 1086, pp. 261–269. Springer, Heidelberg (1996)
Brusilovsky, P.: KnowledgeTree: A distributed architecture for adaptive E-Learning. In: WWW 2004, New York, USA, May 17-22, pp. 104–113 (2004)
Kay, J., Kummerfeld, B., Lauder, P.: Personis: A server for user models. In: De Bra, P., Brusilovsky, P., Conejo, R. (eds.) AH 2002. LNCS, vol. 2347, pp. 203–212. Springer, Heidelberg (2002)
Augustin, T., Hockemeyer, C., Kickmeier-Rust, M.D., Albert, D.: Individualized skill assessment in digital learning games: Basic definitions and mathematical formalism. IEEE Transactions on Learning Technologies 4, 138–148 (2011)
Linton, F., Schaefer, H.-P.: Recommender systems for learning: Building user and expert models through long-term observation of application use. User Modeling and UserAdapted Interaction 10, 181–208 (2000)
Happel, H.-J., Maalej, W.: Potentials and challenges of recommendation systems for software development. In: Proceedings of the 2008 International Workshop on Recommendation Systems for Software Engineering, RSSE 2008, vol. 11. ACM Press, New York (2008)
Ley, T., Ulbrich, A., Scheir, P., Lindstaedt, S.N., Kump, B., Albert, D.: Modelling competencies for supporting work-integrated learning in knowledge work. Journal of Knowledge Management 12, 31–47 (2008)
Ley, T., Kump, B., Gerdenitsch, C.: Scaffolding Self-directed Learning with Personalized Learning Goal Recommendations. In: De Bra, P., Kobsa, A., Chin, D. (eds.) UMAP 2010. LNCS, vol. 6075, pp. 75–86. Springer, Heidelberg (2010)
Brusilovsky, P., Cooper, D.W.: Domain, task, and user models for an adaptive hypermedia performance support system. In: Proceedings of the 7th International Conference on Intelligent User Interfaces, IUI 2002, vol. 23. ACM Press, New York (2002)
Mislevy, R.J., Riconscente, M.M.: Evidence-centered assessment design. In: Downing, S.M., Haladyna, T.M. (eds.) Handbook of Test Development, pp. 61–90. Lawrence Erlbaum Associates, Mahwah (2006)
Lepsinger, R., Lucia, A.D.: The art and science of 360 degree feedback. John Wiley & Sons (2009)
Hoffman, C., Nathan, B., Holden, L.: A comparison of validation criteria: Objective versus subjective performance measures and self- versus supervisor ratings. Personnel Psychology 44, 601–619 (1991)
Muellerbuchof, R., Zehrt, P.: Vergleich subjektiver und objektiver Messverfahren für die Bestimmung von Methodenkompetenz - am Beispiel der Kompetenzmessung bei technischem Fachpersonal. Zeitschrift für Arbeits- und Organisationspsychologie 48, 132–138 (2004)
Wild, F., Haley, D., Bülow, K.: Using latent-semantic analysis and network analysis for monitoring conceptual development. Journal for Language Technology and Computational Linguistics 26, 9–21 (2011)
Harris, M.M., Schaubroeck, J.: A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. Personell Psychology 41, 43–62 (1988)
Conway, J.M., Huffcutt, A.I.: Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings. Human Performance 10, 331–360 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ley, T., Kump, B. (2013). Which User Interactions Predict Levels of Expertise in Work-Integrated Learning?. In: Hernández-Leo, D., Ley, T., Klamma, R., Harrer, A. (eds) Scaling up Learning for Sustained Impact. EC-TEL 2013. Lecture Notes in Computer Science, vol 8095. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40814-4_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-40814-4_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40813-7
Online ISBN: 978-3-642-40814-4
eBook Packages: Computer ScienceComputer Science (R0)