Abstract
Since COVID-19, there have been significant advancements made in the area of online teaching and learning. To provide their pupils with more resources, academic institutions are going digital. Students now have more options for learning at their speed and developing their skills. There has been a shift in favor of online tests for evaluations. AI-assisted proctoring solutions are in great demand as online proctoring services grow in popularity. We provide a method for doing away with the need for a human proctor to be present during the test by creating a multi-modal system. To get footage, we used a camera and active window capture. To infer the test taker’s emotions, his face is recognized. To establish his head position, his feature points are calculated. The surroundings of the examinee can be picked up on, such as a phone, a book, or the presence of another person. Additionally, our system also keeps track of the examinee’s mouth opening and face spoofing. An intelligent rule-based inference system that can determine whether or not there was examination fraud is produced by the combination of these models.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Data will be available on request.
References
Atoum Y, Chen L, Liu AX, Hsu SD, Liu X (2017) Automated online exam proctoring. IEEE Trans Multimed 19(7):1609–1624
Balamurali K, Chandru S, Razvi MS, Kumar VS (2021) Face spoof detection using vgg-face architecture, In: Journal of Physics: Conference Series, Vol. 1917, pp 012010
Bouvier C, Benoit A, Caplier A, Coulon P-Y (2008) Open or closed mouth state detection: static supervised classification based on log-polar signature, In: international conference on advanced concepts for intelligent vision systems, Springer, pp 1093–1102
Chen Y, Lin Q, Wei W, Ji J, Wong K-C, Coello CAC (2022) Intrusion detection using multi-objective evolutionary convolutional neural network for internet of things in fog computing. Knowl-Based Syst 244:108505
Clarke NL, Dowland P, Furnell SM (2013) e-invigilator: A biometric-based supervision system for e-assessments, In: international conference on information society (i-Society 2013), IEEE, pp 238–242
Fares M, Moufarrej A, Jreij E, Tekli J, Grosky W (2019) Unsupervised word-level affect analysis and propagation in a lexical knowledge graph. Knowl-Based Syst 165:432–459
Gadekar A, Oak S, Revadekar A, Nimkar AV (2022) Mmap: A multi-modal automated online proctor, in: Machine Learning and Big Data Analytics (Proceedings of international conference on machine learning and big data analytics (ICMLBDA) 2021), Springer, pp 314–325
Ganidisastra AHS, Bandung Y (2021) An incremental training on deep learning face recognition for m-learning online exam proctoring, in: 2021 IEEE Asia Pacific Conference on Wireless and Mobile (APWiMob), IEEE, pp 213–219
Hou Z, Du Y, Li W, Hu J, Li H, Li X, Chen X (2022) C-bdclstm: a false emotion recognition model in micro blogs combined char-cnn with bidirectional dilated convolutional lstm. Appl Soft Comput 130:109659
Irfan M, Aslam M, Maraikar Z, Jayasinghe U, Fawzan M (2021) Ensuring academic integrity of online examinations, in: 2021 IEEE 16th international conference on industrial and information systems (ICIIS), IEEE, pp 295–300
Kaddoura S, Popescu DE, Hemanth JD (2022) A systematic review on machine learning models for online learning and examination systems. PeerJ Comput Sci 8:e986
Khaireddin Y, Chen Z (2021) Facial emotion recognition: state of the art performance on fer2013, arXiv preprint arXiv:2105.03588
King DL, Case CJ (2014) E-cheating: incidence and trends among college students., Issues in Information Systems 15 (1)
Kuin A (2018) Fraud detection in video recordings of exams using convolutional neural networks, Inception-v4, inception-resnet and the impact of residual connections on learning
Labayen M, Vea R, Flórez J, Aginako N, Sierra B (2021) Online student authentication and proctoring system based on multimodal biometrics technology. IEEE Access 9:72398–72411
Lee S, Kim H, Lieu QX, Lee J (2020) Cnn-based image recognition for topology optimization. Knowl-Based Syst 198:105887
Liu S, Lee I (2021) Sequence encoding incorporated cnn model for email document sentiment classification. Appl Soft Comput 102:107104
Liu Z-Y, Lomovtseva N, Korobeynikova E (2020) Online learning platforms: reconstructing modern higher education. Int J Emerg Technol Learn (iJET) 15(13):4–21
Pramerdorfer C, Kampel M (2016) Facial expression recognition using convolutional neural networks: state of the art, arXiv preprint arXiv:1612.02903
Prathish S, Bijlani K, et al., (2016) An intelligent system for online exam monitoring, In: 2016 international conference on information science (ICIS), IEEE, pp 138–143
Raj RV, Narayanan SA, Bijlani K (2015) Heuristic-based automatic online proctoring system, in: 2015 IEEE 15th international conference on advanced learning technologies, IEEE, pp 458–459
Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. https://arxiv.org/abs/1804.02767
Research W (2018) FER-2013, from the Wolfram Data Repository,
Sasikala N, Sundaram BM, Kumar VN, Sumanth J, Hrithik S (2022) Face recognition based automated remote proctoring platform, in: 2022 second international conference on artificial intelligence and smart energy (ICAIS), IEEE, pp 1753–1760
Tiong LCO, Kim ST, Ro YM (2020) Multimodal facial biometrics recognition: dual-stream convolutional neural networks with multi-feature fusion layers. Image Vis Comput 102:103977
Vulpe-Grigoraşi A, Grigore O (2021) Convolutional neural network hyperparameters optimization for facial emotion recognition, in, 12th international symposium on advanced topics in electrical engineering (ATEE). IEEE 2021:1–5
Wang W, Xu K, Niu H, Miao X (2020) Emotion recognition of students based on facial expressions in online education based on the perspective of computer simulation. Complexity 2020:1–9
Yang J, Zhang L, Tang X, Han M (2022) Codnnet: A lightweight cnn architecture for detection of covid-19 infection. Appl Soft Comput 130:109656
Zhang Y, Lin H, Yang X, Long W (2021) Combining expert weights for online portfolio selection based on the gradient descent algorithm. Knowl-Based Syst 234:107533
Zhang Y, Lu Z, Wang S (2021) Unsupervised feature selection via transformed auto-encoder. Knowl-Based Syst 215:106748
Funding
The authors declare that no funds, grants, or other supports were received during the preparation of this manuscript.
Author information
Authors and Affiliations
Contributions
All the authors have contributed equally to the study, conception, design, material preparation, and analysis. The whole draft of the manuscript was written jointly by all the authors.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
This research does not involve any human participants or animals.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Verma, P., Malhotra, N., Suri, R. et al. Automated smart artificial intelligence-based proctoring system using deep learning. Soft Comput 28, 3479–3489 (2024). https://doi.org/10.1007/s00500-023-08696-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-023-08696-7