Early Prediction of Learners At-Risk of Failure in Online Professional Training Using a Weighted Vote

Abstract

Professional training involves the acquisition of knowledge, skills, and expertise required to perform specific job roles. It can take various forms, including classroom instruction, practical experience, and online training. While online training has the potential to reach a wider audience, a lack of interaction and support may lead to low engagement and high risk of failure. In this paper, we propose a solution to predict, as early as possible, the risk of failure in online professional training. To achieve this, we use a dataset of 13,719 observations, including 912 learners and 182 online courses delivered through an LMS from 2017 to 2022. Our data analysis led us to define the risk of failure based on three classes: low risk, medium risk, and high risk. The objective is to predict the class of each observation when the learner reaches the halfway point in each course. Initially, we tested nine predictive models, which revealed discrepancies in their results across the three classes. To address this gap, we propose a new solution that employs a weighted vote to improve classifications. This solution applies the Borda method to rank the nine models based on their predictive performance in each class. Then, the weight assigned to each model is calculated by considering both its rank and its F1 score in each risk of failure class. Finally, we apply a weighted majority vote involving the nine models. On average, and compared to the baseline models, our solution improves the F1 score by 1.2% for the three risk classes.