Introduction

The Coronavirus infectious disease 2019 (COVID-19) pandemic, which began in October 2019, had a profound impact on various sectors, including tourism, healthcare, and education (International Labor Organization et al., 2020)). As a response to the need for social distancing to curb the spread of the virus, educational institutions, including medical schools, transitioned from traditional face-to-face instruction to online learning, particularly in the preclinical years (Ahmed et al., 2020).

Cheating in Remote Online Examination

Assessment plays a critical role in the educational process alongside knowledge delivery (Tuah & Naing, 2021). The COVID-19 pandemic prompted a transformation in assessment methods from onsite to remote online formats. However, ensuring the validity and reliability of remote online education outcomes has become a significant challenge. To address this issue, several techniques have been developed to enable appropriate and accurate evaluation of cognitive processes at each level, based on the revised Bloom's taxonomy.

Similar to other fields of study, medical education has shifted to a remote online format. Onsite activities such as lectures, small group discussions, and problem-based learning have been organised using real-time communication software (e.g., Cisco WebEx and Zoom), supplemented with recorded videos and web boards. Practical sessions in preclinical years have been converted into online laboratories using simulation, while those in clinical years have been cancelled or postponed. Real-time formative assessments have been integrated into lectures, and certain types of summative evaluations, such as objective structured clinical examination (OSCE) and viva voce, have been conducted via remote online proctoring (Blythe et al., 2021). However, multiple-choice question (MCQ)-type summative examinations face various challenges, especially academic dishonesty, which has been on the rise since the beginning of the COVID-19 pandemic, not only among Science, Technology, Engineering, and Mathematics (STEM) students but also among medical students, where honesty is expected (Abdelrahim, 2021; Comas-Forgas et al., 2021; Hill et al., 2021; Lancaster & Cotarlan, 2021).

Academic cheating is generally defined as the act or attempt of taking advantage of academic output and education by undermining the value of integrity (Anderman et al., 2009; Davis et al., 2009). A recent systematic review has highlighted several ways cheating can be conducted during remote online examinations (Noorbehbahani et al., 2022). Cheating can occur when examinees use forbidden materials as individuals, which can be detected when examinees frequently look at suspicious points where they hide study materials, such as under the table and above the monitor. Additionally, group cheating can occur through collaboration, such as phone calls, screen sharing, and using unauthorised extra devices (Noorbehbahani et al., 2022). Considering the shift to online education due to the COVID-19 pandemic, this study aims to investigate the prevalence of academic dishonesty among medical students in online examinations at Ramathibodi Medical School. Additionally, we will utilise hierarchical cluster analysis and visualisation techniques to explore patterns of academic dishonesty and compare the accuracy of such analysis with self-reported confessions.

Methods

This was a retrospective study of academic honesty during online exams. Initially, qualitative data on examinees’ behaviour during the examination were collected after the routine review of recorded video footage of every remote online exam. The reviewing was done by five of the institute’s educators. Suspicious activities, such as looking at an unusual place where study materials could be placed, during the exam of each examinee were observed and recorded independently by each educator. The exam responses of each student were obtained from Moodle (an online learning management system) for hierarchical clustering analysis.

Source of Information

Hierarchical clustering methods were used to identify cheating on online examinations for three basic medical science modules for first-year medical students. This study used data that had already been collected and stored in Moodle within the Safe Exam Browser for the midterm examination.

In addition, after the head of the preclinical academic unit announced suspicions of cheating to students, some students submitted anonymous self-disclosure reports using a Google form. The Google form was launched by the undergraduate education office as a student engagement method. The research protocol was reviewed and approved by the Institutional Research Board (COA. MURA2021/813).

Online Examination Procedure

A total of 204 preclinical medical students were enrolled in three mandatory basic medical modules, and they all participated in an online examination in April 2021. Questions were in the form of MCQ and short-answer questions (SAQ). For MCQ, questions and options were not shuffled, and the order and wording of the questions were identical for all students. For each examination, students were divided into four groups (50–51 students per group), and each group was assigned to four separate online examination rooms using the Cisco WebEx programme for proctoring. One lecturer and one educator were assigned as online invigilators for each online examination room (two invigilators per 50–51 examinees). Moodle was used to provide the environment for the remote online examination and store students’ responses (selected answers and changed answers), time (starting time, finishing time and individual response time), and a number of changes. Moodle stored correct and incorrect answers for the students’ selected answers.

Detection of Academic Misconduct

The loss of integrity of our students was identified when several students sent an email to the course moderator confessing their misconduct during the online examination. In response to students’ confession, a retrospective investigation of suspicious behaviours was conducted as follows:

For confidentiality, the responses of each student in each exam paper were only obtained from the Moodle system by an assigned educator. The names and the identification numbers of the students were redacted and replaced by a study code before the analyses. The similarity between students’ responses from each student was analysed using a similarity matrix-based hierarchical cluster. Therefore, an M X N table was constructed using the anonymised data provided. Each row contained the exam data of each student, while the first column contained the study code, and students’ answers to each question were stored in the subsequent columns. To observe the correlation among students’ answers, the prepared table was used to construct a hierarchical clustering heatmap using hclust() and heatmap() in R (R Core Team, 2022). An in-depth review of physical behaviours and changes in responses during the examinations was conducted on students, who self-confessed and those who raised suspicions from the heatmap. The video clips recorded during the examination’s proctoring on the WebEx recording system were reviewed by the educator team to investigate the examinees’ behaviours. Suspicious activities during the examinations were noted, including eyes glancing beyond exam devices, hand movements to ears or mouth, and mouth movement.

Question response patterns, especially revision patterns, can indicate possible cheating during examinations. Response revision-based indicators include revision, change of response and time needed for the response (Ranger et al., 2020). In this study, we focused on the pattern of response changes into either correct responses or incorrect responses and the time needed for the response revision. Similarity of response patterns could be associated with cheating, especially in the final response in an MCQ exam. Misconduct is highly likely if two or more students changed their response to the same option within one minute (Ranger et al., 2020).

Results

Prevalence of Cheating on Examination by Confession and Eeasons

Based on anonymous undisclosed confessions submitted to the head of the preclinical academic unit, the prevalence of cheating in the exams of this study was 13.7% (28 out of 204). Only 15 examinee self-confessed prior to the announcement of the cheating suspicion.

When we reviewed the proctoring process in this examination, we found that two invigilators had to monitor approximately 25 examinees in one screen, and they had to switch to another screen for the remaining 25 examinees in the WebEx application. According to a previous study, ineffective proctoring or lack of proctoring can increase cheating in online examinations (Dendir & Maxwell, 2020). Upon reviewing video footage recorded during the examination, several suspicious behaviours were observed. Specifically, certain students were seen on camera raising their hands to cover their mouths, moving their lips as though speaking, looking up at something above their exam device, and staring down at something below their desk. Our findings agreed with previous studies on online examinations that specific behaviours during an examination could indicate an attempt at cheating, including using spoken communication and forbidden materials, which were all found in our study (Noorbehbahani et al., 2022). Forbidden materials identified in our study included short notes, lecture handouts, and wireless in-ear headphones. Taking these together, remote online proctoring coupled with a low invigilator-to-examinee ratio may increase the chances of cheating in remote online examinations.

To understand possible motivations for cheating more comprehensively, reasons for cheating were obtained and extracted from students’ self-confession report forms.

These reasons could be categorised into three themes.

Theme 1: Check Their Responses with Peers

Some examinees’ explanations demonstrated that there was no intention for misconduct. These students needed to be more confident about their answers. The following statements exemplified this theme:

“I did not intend to cheat on the exam. I just wanted to check my answers with my friends. I did it via phone calling during the exam. I am sorry.” (Examinee A).

Theme 2: Retention of Good GPA

Many students announced this theme reported. Students are concerned about post-graduate clinical training, which requires a high GPAX for acceptance. Moreover, these students noticed their peers cheating in other modules. This was reflected in the following statement:

“Everyone did it in the other modules. I was afraid that if I had not done it, my GPA would not have been great. I need a good GPA to secure my residency training in high-quality training institutes.” (Examinee B).

Theme 3: Lack of Care in Proctoring

This theme was raised by several students noticing that live online proctoring was not strict due to the high student-to-invigilator ratio. Some students evidenced the lack of proctoring by cheating without being detected. This behaviour was observed by other examinees, as stated:

“I saw my classmates do that without being noticed by the invigilators, and they kept doing it. I, therefore, did the same.” (Examinee C).

Heatmap and Hierarchical Clustering Methods and Results

A hierarchical heatmap was constructed to visualise the clustering of examinees, using the final responses selected by the examinees, which were stored on the Moodle learning management system. The clustering was based on the correlation of response patterns between each pair of examinees. The colour gradient on the heatmap defined the highest correlation value as red and the lowest as blue (Fig. 1). The heatmap showed that examinees were clustered into three main groups based on their response pattern and correlation value. The correlation of examinees to one another in cluster C was low (Fig. 1). Cluster A showed a low-to-moderate correlation (Fig. 1). Interestingly, the correlation amongst examinees in cluster B was moderate-to-high, as illustrated in yellow and red (Fig. 1). By focusing on the examinees with high correlation values (≥ 0.75) in cluster B, a total number of 27 examinees were observed and clustered into 13 groups based on the dendrogram (the left panel in Fig. 1). When comparing the study code of the suspicious examinees guided by the heatmap (n = 27) and the study code of the examinees who confessed (n = 28), only 15 examinees were identified from both lists (53.6% of examinees who confessed and 55.6% of suspicions raised by the heatmap). All of them were in the cluster B. This showed the concordance between hierarchical clustering and students’ confessions.

Fig. 1
figure 1

A hierarchical heatmap showing similar response patterns was obtained from a remote online examination. Scale represents similarity index (0–1) between data (1 = 100% similarity; 0 = no similarity)

Full size image

Discussion

Academic dishonesty problems rose in educational institutions worldwide during the COVID-19 pandemic. Several tools have been utilised to detect dishonesty, such as exam cheating detection and plagiarism detection (Ranger et al., 2020). In this paper, using a retrospective approach, the authors shared experiences of detecting academic dishonesty during remote online examinations taken by preclinical medical students during the COVID-19 pandemic.

Our findings emphasised that data science-based analysis may help educators identify academic dishonesty, as previously reported (Kamalov et al., 2021). In our case, detection of cheating using hierarchical clustering was used to narrow down and identify the suspicious groups of students so that educators could then pursue the detection with other measures, including inspecting recorded video footage and response times.

In general, our work demonstrated the practice of routine post-examination inspection, done by educator staff, into research at the office of medical education. In addition, hierarchical clustering was applied to illustrate the clustering of students’ responses. Our findings suggested that the grouping of examinees with similar response patterns was concordant with the group of students who confessed (Fig. 1). With guidance from the heatmap, educators could investigate and observe suspicious behaviours further via recorded video clips.

A study conducted amongst medical students in Pakistan demonstrated that exam cheating can be done by various techniques, including exchanging answers with others on mobile during exams, passing information to others, and taking unauthorised materials into the exam (Dar & Khan, 2021). Our observation reported that the cheating type used by examinees during the remote online examinations was voice calls and the use of forbidden study materials. They used short notes, lecture handouts, and wireless in-ear headphones to facilitate their cheating. A systematic review showed that more than ten cheating methods could be used in the online examinations (Noorbehbahani et al., 2022). Cheating could be done either individually or in collaboration with others. For individual cheating, the use of forbidden materials, such as textbooks, lecture notes, and web browsing, was found to be the most common method. For collaborative cheating, the examinee could use various methods, such as impersonation, voice calls, abnormal movement, and remote control screens (Noorbehbahani et al., 2022).

There is a need to raise awareness amongst medical educators to design the curriculum and assessment methods to provide students with the required knowledge and ensure the assessment is a reliable measure of their learning. Interestingly, the most common reasons students gave for cheating were to check their exam responses with their classmates to ensure their high marks because they wanted to receive an “A”. The reasons for cheating could vary depending on the value and situation of each program. A previous study demonstrated that the most frequent reasons for cheating were “time constraints” and “to help friends” (Yardley et al., 2009). It is widely acknowledged that in Thailand, the most popular undergraduate programmes for high achievers are in medical schools because doctors are highly respectable and can earn a high salary with health care benefits for their families (Mei et al., 2022). More importantly, a high GPAX in a medical school transcript usually favours good positions in postgraduate clinical training. Taken together, the motivation for cheating could likely be to get a high GPA to pursue postgraduate training in top-ranked institutes.

Inadequate proctoring was considered a factor that allowed examinees to cheat on exams. In our study, the invigilator-to-examinee ratio was 1:25, which could be too high for one invigilator to observe 25 examinees. Unfortunately, the institution provided details of the online exam rooms, and the list of examinees in each room and the invigilators in the same announcement. This would have let examinees know who were in the same online room. During live proctoring, invigilators had to watch the screen and concentrate on examinees’ behaviour. The invigilators had to move their eyes continuously to look at all 25 examinees. This meant that not all student behaviours could be noticed by the invigilators, and examinees could have a chance to cheat in the exam without being detected.

Limitation

The purpose of this study was to demonstrate the retrospective analysis of a possible cheating event using hierarchical clustering and adjunctive review of video footage of a remote online examination. There are some significant limitations to be declared.

The main limitation of this study is that responses from MCQ-type examinations can be similar among 252 examinees. Clustering from a heatmap alone can mislead invigilators into falsely suspecting cheating. Second, it would have been useful to conduct student focus groups to analyse the root causes of the motivations behind exam cheating. By taking into account these limitations, a more carefully designed future study should improve analysis and findings.

Conclusion

The COVID-19 pandemic necessitated a shift towards online learning in education. Medical curricula adopted online classrooms and examinations in response to social distancing policies aimed at preventing the spread of SARS-CoV2. This study aimed to determine the prevalence of academic dishonesty in remote online examinations using multiple approaches, including anonymised confessions and reviewing recorded video footage. The findings from this study indicate that cheating occurs during online examinations, as evidenced by anonymised confessions, reviewing recorded videos, and high correlations in response patterns. This finding should lead to more careful design of assessments for medical students under any future physical distancing requirements.