Multimodal Group Activity State Detection for Classroom Response System Using Convolutional Neural Networks

Abstract

Human–Computer Interaction is a crucial and emerging field in computer science. This is because computers are replacing humans in many jobs to provide services. This has resulted in the computer being needed to interact with the human in the same way as the human does with another. When humans talk to each other, they gain feedback based on how the other person responds non-verbally. Since computers are now interacting with humans, they need to be able to detect these facial cues and accordingly adjust their services based on this feedback. Our proposed method aims at building a Multimodal Group Activity State Detection for Classroom Response System which tries to recognize the learning behavior of a classroom for providing effective feedback and inputs to the teacher. The key challenges dealt here are to detect and analyze as many students as possible for a non-biased evaluation of the mood of the students and classify them into three activity states defined: active, passive, and inactive.