Skip to main content
SpringerLink
Log in

Mouse Movement and Probabilistic Graphical Models Based E-Learning Activity Recognition Improvement Possibilistic Model

  • Research Article - Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

Automatically recognizing the e-learning activities is an important task for improving the online learning process. Probabilistic graphical models such as hidden Markov models and conditional random fields have been successfully used in order to identify a Web users’ activity. For such models, the sequences of observation are crucial for training and inference processes. Despite the efficiency of these probabilistic graphical models in segmenting and labeling stochastic sequences, their performance is adversely affected by the imperfect quality of data used for the construction of sequences of observation. In this paper, a formalism of the possibilistic theory will be used in order to propose a new approach for observation sequences preparation. The eminent contribution of our approach is to evaluate the effect of possibilistic reasoning—during the generation of observation sequences—on the effectiveness of hidden Markov models and conditional random fields models. Using a dataset containing 51 real manipulations related to three types of learners’ tasks, the preliminary experiments demonstrate that the sequences of observation obtained based on possibilistic reasoning significantly improve the performance of hidden Marvov models and conditional random fields models in the automatic recognition of the e-learning activities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aggarwal J.K., Ryoo M.S.: Human activity analysis: a review. ACM Comput. Surv. (CSUR) 43(3), 16 (2011)

    Article  Google Scholar 

  2. Natarajan, P.; Nevatia, R.: Graphical models for representation and recognition of human actions. In: Wang, L., Cheng, L., Zhao, G. (eds.) Machine Learning for Human Motion Analysis: Theory and Practice, pp. 31–54. Medical Information Science Reference, Hershey, PA (2010). doi:10.4018/978-1-60566-900-7.ch003

  3. Seo, S.W.; Kang, J.-H.; Drummond, J.; Kim, J.: Using graphical models to classify dialogue transition in online Q&A discussions. In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) Artificial Intelligence in Education, 15th International Conference, AIED 2011, Auckland, New Zealand, June 28–July 2011, pp. 550–553. Springer, Berlin, Heidelberg (2011). doi:10.1007/978-3-642-21869-9_98

  4. Karaman S., Benois-Pineau J., Dovgalecs V., Mégret R., Pinquier J., André-Obrecht R., Gaëstel Y., Dartigues J.F.: Hierarchical Hidden Markov Model in detecting activities of daily living in wearable videos for studies of dementia. Multimed. Tools Appl. 69(3), 743–771 (2014)

    Article  Google Scholar 

  5. Kulić, D.; Takano, W.; Nakamura, Y.: Representability of human motions by factorial hidden Markov models. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007. IROS 2007, pp. 2388–2393. IEEE (2007)

  6. Mahjoub, M.A.; Omri, M.N.: Non stationary DHMM based on line piece of Arabic word (PAW) recognition network. In: The 2002 International Conference on Imaging Science, Systems, and Technology (CISST’02), Las Vegas, USA (2002)

  7. Huang, Q.; Han, M.; Wu, B.; Ioffe, S.: A hierarchical conditional random field model for labeling and segmenting images of street scenes. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1953–1960. IEEE (2011)

  8. Yao, K.; Peng, B.; Zweig, G.; Yu, D.; Li, X.; Gao, F.: Recurrent conditional random field for language understanding. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4077–4081. IEEE (2014)

  9. Lu, W.; Ng, H.T.: Better punctuation prediction with dynamic conditional random fields. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 177–186 (2010)

  10. Ma, X.; Grimson, W.E.L.: Learning coupled conditional random field for image decomposition with application on object categorization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8. IEEE (2008)

  11. McKay M.D., Morrison J.D., Upton S.C.: Evaluating prediction uncertainty in simulation models. Comput. Phys. Commun. 117(1), 44–51 (1999)

    Article  MATH  Google Scholar 

  12. Dufresne, A.; Courtemanche, F.; Tep, S.P.; Senecal, S.: Physiological measures, eye tracking and task analysis to track user reactions in user generated content. In: 7th Internationcal Conference on Methods and Techniques in Behavioral Research (Measuring Behavior), p. 218 (2010)

  13. Goldberg, J.H.; Stimson, M.J.; Lewenstein, M.; Scott, N.; Wichansky, A.M.: Eye tracking in web search tasks: design implications. In: Proceedings of the 2002 Symposium on Eye Tracking Research & Applications, pp. 51–58. ACM (2002)

  14. Huang, J.; White, R.W.; Buscher, G.; Wang, K.: Improving searcher models using mouse cursor activity. In: Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 195–204. ACM (2012)

  15. Courtemanche F., Aïmeur E., Dufresne A., Najjar M., Mpondo F.: Activity recognition using eye-gaze movements and traditional interactions. Interact. Comput. 23(3), 202–213 (2011)

    Article  Google Scholar 

  16. Vetrov, D.; Kropotov, D.; Konushin, A.; Lomakina-Rumyantseva, E.: Automatic segmentation of mouse behavior using hidden Markov model. In: Spink, A.J.; Ballintijn, M.R.; Bogers, N.D.; Grieco, F.; Loijens, L.W.S.; Noldus, L.P.J.J.; Smit, G.; Zimmerman, P.H. (eds.) Proceedings of the 6th International Conference on Methods and Techniques in Behavioral Research, p. 241. Maastricht, The Netherlands (2008)

  17. Soller A.: Computational modeling and analysis of knowledge sharing in collaborative distance learning. User Model. User Adapt. Interact. 14(4), 351–381 (2004)

    Article  Google Scholar 

  18. Lafferty, J.D.; McCallum, A.; Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, pp. 282–289 (2001)

  19. Tong Y., Chen R., Gao J.: Hidden State Conditional Random Field for Abnormal Activity Recognition in Smart Homes. Entropy. 17(3), 1358–1378 (2015)

    Article  Google Scholar 

  20. Agarwal, M.; Flach, P.: Activity recognition using conditional random field. In: Proceedings of the 2nd International Workshop on Sensor-Based Activity Recognition and Interaction, p. 4. ACM (2015)

  21. Sun X., Kashima H., Ueda N.: Large-scale personalized human activity recognition using online multitask learning. IEEE Trans. Knowl. Data Eng. 25(11), 2551–2563 (2013)

    Article  Google Scholar 

  22. Nazerfard, E.; Das, B.; Holder, L.B.; Cook, D.J.: Conditional random fields for activity recognition in smart environments. In: Proceedings of the 1st ACM International Health Informatics Symposium, pp. 282–286. ACM (2010)

  23. Elbahi, A.; Omri, M.N.: Web user interact task recognition based on conditional random fields. In: Azzopardi, G., Petkov, N. (eds.) Computer Analysis of Images and Patterns, 16th International Conference, CAIP 2015, Valletta, Malta, September 2–4, 2015 Proceedings, Part I, pp. 740–751. Springer International Publishing (2015). doi:10.1007/978-3-319-23192-1_62

  24. Elbahi, A.; Mahjoub, M.A.; Omri, M.N.: Hidden Markov model for inferring user task using mouse movement. In: Information and Communication Technology and Accessibility (ICTA), IEEE, pp. 1–7 (2013)

  25. Mathe, S.; Sminchisescu, C.: Action from still image dataset and inverse optimal control to learn task specific visual scanpaths. In: Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, pp. 1923–1931. Curran Associates, Inc (2013)

  26. Fitts P.M., Jones R.E., Milton J.L.: Eye movements of aircraft pilots during instrument-landing approaches. Ergon. Psychol. Mech. Models Ergon. 3, 56 (2005)

    Google Scholar 

  27. Bateman S., Mandryk R.L., Gutwin C., Xiao R.: Analysis and comparison of target assistance techniques for relative ray-cast pointing. Int. J. Hum. Comput. Stud. 71(5), 511–532 (2013)

    Article  Google Scholar 

  28. Poole, A.; Ball, L.J.; Phillips, P.: In search of salience: a response-time and eye-movement analysis of bookmark recognition. In: People and Computers XVIII-Design for Life: Proceedings of HCI 2004. Springer Science & Business Media, vol. 18, pp. 363–378 (2004)

  29. Elbahi, A.; Omri, M.N.: Conditional random fields for web user task recognition based on human computer interaction. In: 23rd International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, (WSCG), pp. 59–63 (2015)

  30. Spehr, J.: On Hierarchical Models for Visual Recognition and Learning of Objects, Scenes, and Activities. Studies in Systems, Decision and Control 11, pp. 7–20. Springer International Publishing, Switzerland (2015). doi:10.1007/978-3-319-11325-8_1

  31. Markov, A.A.: The theory of algorithms (translated from the Russian by J.J. Schorrkon), U.S. Dept. of Commerce, Office of Technical Services, No. OTS 60-5108 (1953)

  32. Kushner, B.A.: The constructive mathematics of A. A. Markov. Am. Math. Mon. 113(6), 559–566 (2006). doi:10.2307/27641983

  33. Rabiner L., Juang B.H.: An introduction to hidden Markov models. ASSP Mag. IEEE 3(1), 4–16 (1986)

    Article  Google Scholar 

  34. Ji X., Wang C., Li Y.: Hidden Markov model-based human action recognition using mixed features. J. Comput. Inf. Syst. 9, 3659–3666 (2013)

    Google Scholar 

  35. Pentland A., Liu A.: Modeling and prediction of human behavior. Neural Comput. 11(1), 229–242 (1999)

    Article  Google Scholar 

  36. Baum, L.E.; Petrie, T.: Statistical inference for probabilistic functions of finite state Markov chains. Ann. Math. Stat. 37(6), 1554–1563 (1966)

  37. Baum L.E., Petrie T., Soules G., Weiss N.: A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Stat. 41, 164–171 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  38. Davis R.I., Lovell B.C., Caelli T.: Improved estimation of hidden markov model parameters from multiple observation sequences. Proc. 16th Int. Conf. Pattern Recognit. 2, 168–171 (2002)

    Google Scholar 

  39. Sutton C., McCallum A.: An introduction to conditional random fields. Mach. Learn. 4(4), 267–373 (2011)

    Article  MATH  Google Scholar 

  40. Shannon C.E.: A mathematical theory of communication. ACM SIGMOBILE Mob. Comput. Commun. Rev. 5(1), 3–55 (2001)

    Article  MathSciNet  Google Scholar 

  41. Cappé, O.; Moulines, E.; Rydén, T.: Inference in hidden markov models. In: Proceedings of EUSFLAT Conference, pp. 14–16 (2009)

  42. Kelarestaghi, M.; Slimane, M.; Vincent, N.: Introduction of fuzzy logic in the Hidden Markov models. In: Proceedings of EUSFLAT Conference, pp. 14–16 (2001)

  43. Zhang, X.; Naghdy, F.: Human motion recognition through fuzzy hidden markov model. In: International Conference on Computational Intelligence for Modelling, Control and Automation, 2005 and International Conference on Intelligent Agents, Web Technologies and Internet Commerce, vol. 2, pp. 450–456 (2005)

  44. Zeng J., Liu Z.Q.: Type-2 fuzzy hidden Markov models and their application to speech recognition. IEEE Trans. Fuzzy Syst. 14(3), 454–467 (2006)

    Article  MathSciNet  Google Scholar 

  45. Ramasso, E.: Contribution of belief functions to hidden Markov models with an application to fault diagnosis. In: IEEE International Worshop on Machine Learning for Signal Processing, MLSP’09, pp. 1–6 (2009)

  46. Salmane, H.; Ruichek, Y.; Khoudour, L.: Using hidden markov model and dempster-shafer theory for evaluating and detecting dangerous situations in level crossing environments. In: Batyrshin, I., González Mendoza, M. (eds.) Advances in Artificial Intelligence, 11th Mexican International Conference on Artificial Intelligence, MICAI 2012, San Luis Potosí, Mexico, October 27–November 4, 2012. Revised Selected Papers, Part I, pp. 131–145. Springer, Berlin, Heidelberg (2013). doi:10.1007/978-3-642-37807-2_12

  47. Cho P.C., Chen W.H., Liu S.I.: A comparative study of fuzzy conditional random fields for bathroom activity recognition. Adv. Sci. Lett. 13(1), 427–430 (2012)

    Article  Google Scholar 

  48. Luo J., Li Y.: Intrusion detection method based on fuzzy conditional random fields. J. Comput. Inf. Syst. 9(20), 8361–8370 (2013)

    Google Scholar 

  49. Dubois, D.; Prade, H.: Possibility theory and its applications: where do we stand?. In: Kacprzyk, J., Pedrycz, W. (eds.) Springer Handbook of Computational Intelligence, pp. 31–60. Springer, Berlin, Heidelberg (2015). doi:10.1007/978-3-662-43505-2_3

  50. Bocheński, I.M.: La Logique de Théophraste (Librairie de l’Université de Fribourg en Suisse, Fribourg) (1947)

  51. Chellas B.F.: Modal Logic, An Introduction. Cambridge University Press, Cambridge (1980)

    Book  MATH  Google Scholar 

  52. Shackle Decision G.L.S.: Order and Time in Human Affairs, 2nd edn. Cambridge University Press, London (1961)

    Google Scholar 

  53. Lewis D.L.: Counterfactuals. Basil Blackwell, Oxford (1973)

    MATH  Google Scholar 

  54. Cohen L.J.: The Probable and the Provable. Clarendon, Oxford (1977)

    Book  Google Scholar 

  55. Zadeh L.A.: Fuzzy sets. Inf. Control. 8, 338–353 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  56. Zadeh L.A.: Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets Syst. 1, 3–28 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  57. Gaines, B.R.; Kohout, L.J.: Possible automata. In: Proceedings of International Symposium on Multiple-Valued Logic, pp. 183–196 (1975)

  58. Dubois D., Prade H.: Representation and combination of uncertainty with belief functions and possibility measures. Comput. Intell. 4, 244–264 (1988)

    Article  Google Scholar 

  59. Dubois D., Prade H.: Possibility Theory. Wiley, New York (1988)

    Book  MATH  Google Scholar 

  60. Jenhani, I.; Benferhat, S.; Elouedi, Z.: Possibilistic similarity measures. In: Bouchon-Meunier, B., Magdalena, L., Ojeda-Aciego, M., Verdegay, J.-L., Yager, R.R. (eds.) Foundations of Reasoning under Uncertainty, pp. 99–123. Springer, Berlin, Heidelberg (2010). doi:10.1007/978-3-642-10728-3_6

  61. Dubois, D.; Prade, H.: Possibility theory: qualitative and quantitative aspects. In: Smets, P. (ed.) Quantified Representation of Uncertainty and Imprecision, pp. 169–226. Springer, Netherlands (1998). doi:10.1007/978-94-017-1735-9_6

  62. Garrouch, K.; Omri, M.N.; ElAyeb, B.: Pertinent information retrieval based on possibilistic bayesian network: origin and possibilistic perspective. In: International Forum for Exploring e-Business and e-Government Research, Applications, and Technologies (TIGERA) (2008)

  63. Garrouch, K.; Omri, M.N.; Kouzana, A.: A new information retrieval model based on possibilistic Bayesian network. In: Proceedings of the International Conference on Computer Related Knowledge, Tunisia, pp. 79–88 (2012)

  64. Omri, M.N.: Fuzzy knowledge representation, learning and optimization with bayesian analysis in fuzzy semantic networks. In: 6th International Conference on Neural Information Processing, 1999. Proceedings. (ICONIP’99), vol. 1, pp. 412–417 (1999)

  65. Omri, M.N.; Tijus, C.; Poitrenaud, S.; Bouchon-Meunier, B.: Fuzzy sets and semantic nets for on line assistance. In: 11th Conference on Artificial Intelligence for Applications, pp. 374–379 (1995)

  66. Laurent, L.; Guillaume, S.; Charnomordic, B.: Extraction de zones floues sur des cartes. Rencontres LFA francophones sur la Logique Floue et ses Applications (LFA). Cépaduès Editions (2006)

  67. Bezdek J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York (1981)

    Book  MATH  Google Scholar 

  68. Liu, Y.; Shriberg, E.; Stolcke, A.; Harper, M.P.: Comparing HMM, maximum entropy, and conditional random fields for disfluency detection. In: INTERSPEECH, pp. 3313–3316 (2005)

Download references

Author information

Authors and Affiliations

  1. Department of computer sciences, Faculty of sciences of Monastir, Research Unit MARS, Monastir, Tunisia

    Anis Elbahi, Mohamed Nazih Omri & Kamel Garrouch

  2. National Engineering School of Sousse, Research Unit SAGE, Sousse, Tunisia

    Mohamed Ali Mahjoub

Authors
  1. Anis Elbahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Nazih Omri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Ali Mahjoub
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kamel Garrouch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anis Elbahi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Elbahi, A., Omri, M.N., Mahjoub, M.A. et al. Mouse Movement and Probabilistic Graphical Models Based E-Learning Activity Recognition Improvement Possibilistic Model. Arab J Sci Eng 41, 2847–2862 (2016). https://doi.org/10.1007/s13369-016-2025-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-016-2025-6

Keywords

Search

Navigation