A tele-OSCE was designed and implemented by involved faculty academicians for semester 2 2019/2020 of the Master’s in Clinical Pharmacy program in response to the call for restriction of face-to-face TL activities as an effort to curb the COVID-19 pandemic.
The University subscribed to two main learning management systems to support TL activities, Moodle® and Microsoft Office 365®. Microsoft Office 365® was selected to conduct tele-OSCE because it features the Microsoft Teams® application that supports virtual videoconferencing and real-time sharing of documents. Microsoft Teams® can be downloaded and accessed using all smart devices, including tablets and smartphones. Several roles and tasks were established to ensure the smooth conduct of the tele-OSCE (Table 1).
Content experts on OSCE and clinical pharmacy led the discussion and workshops on tele-OSCE development. OSCE tasks were aimed to assess the psychomotor and affective domains of learning in outcome-based education, which was possible to be assessed using tele-OSCE . The teaching team decided to exclude the assessment of the cognitive/knowledge domain of learning in this tele-OSCE because it was extensively tested in other forms of assessment such as quizzes and the final written examination. OSCE tasks were designed to assess understanding and skills on the following topics taught in the second semester of study: critical care, musculoskeletal and rheumatology, oncology, and special populations. The setting and the tasks required for completion at each OSCE station were decided based on the complexity of the tasks: whether it could be successfully attempted within 15 min and whether it can be tested and assessed effectively via tele-OSCE.
The focus of this tele-OSCE was clinical rationalisation and problem solving, communication skills, and professionalism which were in accordance with the course learning outcomes. An overview of the tele-OSCE tasks to be undertaken at each of the six stations is tabulated in Table 2.
For each planned tele-OSCE station, question providers with adequate background in the area were tasked to create questions, scripts for simulated actors, and marking rubric for assessors in word processor format. The questions and supporting documents were subjected to rigorous online vetting processes by the faculty academic at the department level and subsequently at a higher-level vetting. Afterward, the final version of the answer schemes was converted into online assessment forms using the Microsoft Forms® software for assessors’ use during the actual examination. The tele-OSCE manager conducted a final vetting of the transcribed online rubric with the Microsoft Word document to identify and rectify errors during transcription.
Two duplicates of tele-OSCE stations were created to maximise time and efficiency of tele-OSCE conduction. The decision on number of duplicates was based on both the number of students and the number of academic faculty with adequate background to perform the assessment of learning in clinical pharmacy. For a total number of 17 Master’s in Clinical Pharmacy students, 6 assessors who were clinical pharmacy lecturers and 6 simulated actors per OSCE station were appointed.
Two mock sessions of tele-OSCE were trialled prior to OSCE day. The first session was conducted to test the operational flow for each of the four roles involved in each tele-OSCE station: tele-OSCE manager, assessors, simulated actors, and students. The second session was conducted to assure the quality of online communication between assessors, simulated actors, and students, and finalising the tele-OSCE flow. Written feedbacks were informally obtained through emails and iterations to the final tele-OSCE operational flow were made. A briefing session with assessors and standardised actors was also conducted to ensure standardised assessments by the assessors and standardised delivery of performance by the standardised actors during the tele-OSCE to minimise assessment bias. Before the actual tele-OSCE, all involved faculty members and student candidates signed a declaratory form to preserve academic integrity as a deterrent for cheating.
On tele-OSCE day, the OSCE manager, assessors, simulated actors, and students logged online into Microsoft Teams® and entered a virtual room created and titled “Master’s in Clinical Pharmacy Tele-OSCE” 30 min ahead of OSCE start time. All individuals are required to use their University Microsoft Outlook account for this procedure to ease communication and online tracking which was harder to achieve when using an external email (guest) account. Upon instruction, the OSCE manager, assessors, and simulated actors adjourned into an online conference call at designated “stations” which were virtual breakout rooms created on Microsoft Teams®. As scheduled, students received an invitation on Microsoft Teams® to enter the station at predetermined times. Students successfully entering into the OSCE station marked the start of the 15 min allocated for each OSCE task.
The first 2 min required the student to verbally affirm viewing of the tele-OSCE question on the shared screen and for students to prepare necessary information to complete the task. After 2 min have elapsed, the assessor announces the start of interaction with the simulated actor. An indicator was given at minute 8 via the “raise hand” function and an alarm sound. At 12 min, the assessor instructed the student to complete their final sentence and end the task. Verbal formative feedback on the student’s performance of the tasks immediately ensued prior to adjourning. The student was removed from the call at the end of 15 min and the cycle was repeated with the next students according to rotation.
Feedback and Performance Analyses
At the end of the session, online links to two sets of surveys were uploaded to the Microsoft Teams®. Students were asked to complete the survey on experience, ease of use, and acceptability of the tele-OSCE. The academic faculty and simulated actors were asked to complete the survey on ease of participation in the tele-OSCE, and ease in assessing or responding to the students. The survey employed a 5-point Likert scale (1 = strongly agree, 2 = agree, 3 = neutral, 4 = disagree, 5 = strongly disagree). The settings on the online survey form were adjusted to disallow submission if the survey was incompletely filled. Students’ performance during the tele-OSCE was taken as the scores obtained from each station and converted into percentages (%). The mean scores were calculated for each station and compared with scores attained during face-to-face OSCE in the previous semester. An independent t-test was conducted using IBM Statistical Package for the Social Sciences (SPSS) version 24.0 after normality testing revealed a normal data distribution.
This study was a part of the end of semester routine course evaluation undertaken to enhance teaching–learning activities and was exempt from requiring approval from the institutional research ethics committee. Prior to attempting the online survey, respondents were made aware that completion of the survey constitutes giving informed consent on the use of information and future intended publication. All data were treated confidentiality and anonymously.