Skip to main content
SpringerLink
Log in

Engagement Analysis of Students in Online Learning Environments

  • Conference paper
  • First Online:
Machine Learning and Big Data Analytics (Proceedings of International Conference on Machine Learning and Big Data Analytics (ICMLBDA) 2021) (ICMLBDA 2021)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 256))

Included in the following conference series:

Abstract

Engagement rate is considered a metric that measures the extent of engagement a particular content is receiving from the audience. In e-learning settings, educators want to observe the level of interest of learners to appropriately modify their courses and make the educational process more effective. In this paper, an ensemble approach is proposed to detect student engagement levels while watching an e-learning video. The ensemble model consists of a deep convolutional neural network (DCNN) for facial expression recognition and a deep recurrent neural network (DRNN) for establishing a relationship between eye-gaze and engagement intensity. OpenFace 2.0 toolbox abilities are leveraged for feature extraction. Experimental results on the test datasets give an accuracy of 55.64% on DAiSEE and an MSE of 0.0598 on Engagement in the Wild Dataset.

Equal contribution–All authors

Proceedings of the \(38^{\mathrm{th}}\) International Conference on Machine Learning, PMLR 139, 2021. Copyright 2021 by the author(s).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Baltrusaitis, T., Robinson, P., Morency, L.: Constrained local neural fields for robust facial landmark detection in the wild. In: 2013 IEEE International Conference on Computer Vision Workshops, pp. 354–361 (2013)

    Google Scholar 

  • Baltrusaitis, T., Zadeh, A., Lim, Y.C., Morency, L.: Openface 2.0: facial behavior analysis toolkit. In: 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), pp. 59–66 (2018)

    Google Scholar 

  • BaltruÅ¡aitis, T., Mahmoud, M., Robinson, P.: Cross-dataset learning and person-specific normalisation for automatic action unit detection. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), vol. 06, pp. 1–6 (2015)

    Google Scholar 

  • Bidwell, J., Fuchs, H.: Classroom analytics: measuring student engagement with automated gaze tracking, November 2011

    Google Scholar 

  • Burch, G.F., Heller, N.A., Burch, J.J., Freed, R., Steed, S.A.: Student engagement: developing a conceptual framework and survey instrument. J. Educ. Bus. 90(4), 224–229 (2015). https://doi.org/10.1080/08832323.2015.1019821

    Article  Google Scholar 

  • Chang, C., Zhang, C., Chen, L., Liu, Y.: An ensemble model using face and body tracking for engagement detection. In: Proceedings of the 20th ACM International Conference on Multimodal Interaction, ICMI 2018, pp. 616–622. Association for Computing Machinery, New York (2018). ISBN 9781450356923. https://doi.org/10.1145/3242969.3264986

  • Chatterjee, P., Nath, A.: Massive open online courses (MOOCs) in higher education – unleashing the potential in India. In: 2014 IEEE International Conference on MOOC, Innovation and Technology in Education (MITE), pp. 256–260 (2014)

    Google Scholar 

  • Correa, E., Jonker, A., Ozo, M., Stolk, R.: Emotion recognition using deep convolutional neural networks. Tech. Rep. IN4015 (2016)

    Google Scholar 

  • Dewan, M.A.A., Murshed, M., Lin, F.: Engagement detection in online learning: a review. Smart Learn. Environ. 6(1), 1–20 (2019). https://doi.org/10.1186/s40561-018-0080-z

    Article  Google Scholar 

  • Dhall, A.: Emotiw 2019: automatic emotion, engagement and cohesion prediction tasks. In: 2019 International Conference on Multimodal Interaction, ICMI 2019, pp. 546–550. Association for Computing Machinery, New York (2019). ISBN 9781450368605. https://doi.org/10.1145/3340555.3355710

  • Dhall, A., Kaur, A., Goecke, R., Gedeon, T.: Emotiw 2018: audio-video, student engagement and group-level affect prediction. In: Proceedings of the 20th ACM International Conference on Multimodal Interaction, ICMI 2018, pp. 653–656. Association for Computing Machinery, New York (2018). ISBN 9781450356923. https://doi.org/10.1145/3242969.3264993

  • Goodfellow, I.J.: Challenges in representation learning: a report on three machine learning contests. In: Lee, M., Hirose, A., Hou, Z.-G., Kil, R.M. (eds.) Neural Information Processing, pp. 117–124. Springer, Heidelberg (2013). ISBN 978-3-642-42051-1

    Google Scholar 

  • Gudi, A.: Recognizing semantic features in faces using deep learning. ArXiv. arXiv:abs/1512.00743 (2015)

  • Gupta, A., Jaiswal, R., Adhikari, S., Balasubramanian, V.: DAISEE: dataset for affective states in e-learning environments. CoRR. arXiv:abs/1609.01885 (2016)

  • He, W.: Examining students’ online interaction in a live video streaming environment using data mining and text mining. Comput. Hum. Behav. 29(1), 90–102 (2013). ISSN 0747-5632. https://doi.org/10.1016/j.chb.2012.07.020. Including Special Section Youth, Internet, and Wellbeing

  • Hernandez, J., Zicheng Liu, Hulten, G., DeBarr, D., Krum, K., Zhang, Z.: Measuring the engagement level of tv viewers. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–7 (2013)

    Google Scholar 

  • Huang, M., Shu, H., Jiang, J.: An algorithm of key-frame extraction based on adaptive threshold detection of multi-features. In: 2009 International Conference on Test and Measurement, vol. 1, pp. 149–152 (2009)

    Google Scholar 

  • Kamath, A., Biswas, A., Balasubramanian, V.: A crowdsourced approach to student engagement recognition in e-learning environments. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–9 (2016)

    Google Scholar 

  • Kaur, A.: Attention network for engagement prediction in the wild. In: Proceedings of the 20th ACM International Conference on Multimodal Interaction, ICMI 2018, pp. 516–519. Association for Computing Machinery, New York (2018). ISBN 9781450356923. https://doi.org/10.1145/3242969.3264972

  • Kaur, A., Mustafa, A., Mehta, L., Dhall, A.: Prediction and localization of student engagement in the wild. In: 2018 Digital Image Computing: Techniques and Applications (DICTA), pp. 1–8 (2018)

    Google Scholar 

  • Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates, Inc. (2012)

    Google Scholar 

  • Nezami, O.M., Dras, M., Hamey, L., Richards, D., Wan, S., Paris, C.: Automatic recognition of student engagement using deep learning and facial expression (2018)

    Google Scholar 

  • Nguyen, T.P., Chew, M.T., Demidenko, S.: Eye tracking system to detect driver drowsiness. In: 2015 6th International Conference on Automation, Robotics and Applications (ICARA), pp. 472–477 (2015)

    Google Scholar 

  • Niu, X., Han, H., Zeng, J., Sun, X., Shan, S., Huang, Y., Yang, S., Chen, X.: Automatic engagement prediction with gap feature. In: Proceedings of the 20th ACM International Conference on Multimodal Interaction, ICMI 2018, pp. 599–603. Association for Computing Machinery, New York (2018). ISBN 9781450356923. https://doi.org/10.1145/3242969.3264982

  • Thomas, C., Jayagopi, D.B.: Predicting student engagement in classrooms using facial behavioral cues. In: Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education, MIE 2017, pp. 33–40. Association for Computing Machinery, New York (2017). ISBN 9781450355575. https://doi.org/10.1145/3139513.3139514

  • Thong Huynh, V., Kim, S.-H., Lee, G.-S., Yang, H.-J.: Engagement intensity prediction with facial behavior features. In: 2019 International Conference on Multimodal Interaction, ICMI 2019, pp. 567–571. Association for Computing Machinery, New York (2019). ISBN 9781450368605. https://doi.org/10.1145/3340555.3355714

  • Whitehill, J., Serpell, Z., Lin, Y., Foster, A., Movellan, J.R.: The faces of engagement: automatic recognition of student engagement from facial expressions. IEEE Trans. Affect. Comput. 5(1), 86–98 (2014)

    Article  Google Scholar 

  • Wilson, P.I., Fernandez, J.: Facial feature detection using HAAR classifiers. J. Comput. Sci. Coll. 21(4), 127–133 (2006). ISSN 1937-4771

    Google Scholar 

  • Wood, E., Baltrusaitis, T., Zhang, X., Sugano, Y., Robinson, P., Bulling, A.: Rendering of eyes for eye-shape registration and gaze estimation (2015)

    Google Scholar 

  • Zadeh, A., Lim, Y., Baltrusaitis, T., Morency, L.: Convolutional experts constrained local model for 3D facial landmark detection. In: 2017 IEEE International Conference on Computer Vision Workshop (ICCVW), Los Alamitos, pp. 2519–2528. IEEE Computer Society, October 2017. https://doi.org/10.1109/ICCVW.2017.296

  • Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23(10), 1499–1503 (2016). ISSN 1558-2361. https://doi.org/10.1109/lsp.2016.2603342

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, National Institute of Technology Karnataka, Mangalore, India

    Sharanya Kamath, Palak Singhal, Govind Jeevan & B. Annappa

Authors
  1. Sharanya Kamath
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Palak Singhal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Govind Jeevan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. B. Annappa
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Patna, Patna, India

    Rajiv Misra

  2. Indian Institute of Technology Bombay, Mumbai, India

    Rudrapatna K. Shyamasundar

  3. Indian Institute of Technology (BHU), Varanasi, India

    Amrita Chaturvedi

  4. Cardiff University, Cardiff, UK

    Rana Omer

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kamath, S., Singhal, P., Jeevan, G., Annappa, B. (2022). Engagement Analysis of Students in Online Learning Environments. In: Misra, R., Shyamasundar, R.K., Chaturvedi, A., Omer, R. (eds) Machine Learning and Big Data Analytics (Proceedings of International Conference on Machine Learning and Big Data Analytics (ICMLBDA) 2021). ICMLBDA 2021. Lecture Notes in Networks and Systems, vol 256. Springer, Cham. https://doi.org/10.1007/978-3-030-82469-3_4

Download citation

Publish with us

Policies and ethics

Search

Navigation