Introduction

Basic surgical skills are essential for physicians in their clinical practice. Medical schools are responsible for teaching and ensuring proficiency of these skills in their undergraduates while students are responsible for self-directed practice [1]. In our institute, a 2-h small-group teaching session, containing skill demonstration and practice under supervision in part-task models, has been used. Students may gain additional experience in real patients in the clinic, operating room, and emergency room. Further self-directed practice is encouraged by providing a take-home “suture practicing kit”. The skills would be assessed as a part of the end-of-rotation summative Objective Structured Clinical Examination (OSCE). Scores of 65–75% were seen in the past few years, which were unsatisfactory. Explanation may be inadequate learning or inadequate practice. Firstly, we failed to reach the optimal teacher to student ratio for motor skill training of 1:4 [2]. This is a limitation in most medical schools. Secondly, inadequate further self-directed practice was likely due to a lack of students’ external motivation. Students might internally be motivated to be a good and skillful doctor but the drive to repeatedly practice the skills might not be steadily high enough.

The critical step that could be added during this self-directed psychomotor skill practice period is “feedback” [3,4,5,6]. “Feedback” is a teaching strategy which specific information about learner’s performance, as compared to the established standard, is given to the learner with aims to increase learner’s motivation and improve learner’s performance [3]. For procedural skills, feedback is one of Gagne’s critical learning conditions [4] and a part of Ericsson’s deliberate practice condition to acquire expert performance [5]. Classically, feedback is given verbally right after direct observation of skills. Other ways which feedback can be conveyed are via paper, video, audio and digital platform [7]. Among these, video-recorded practice and feedback (VDO-feedback) is attractive because (1) asynchronous assessment and feedback was possible [7, 8] when there is a limited number of experts available. (2) individualized feedback making teacher to student ratio 1:1. (3) self-review is possible (4) Performing task without presence of assessor would reduce stress and anxiety [9] and (5) Ease of production of self-recording VDO using smartphone or tablet [9]. Another role of feedback is to improve students’ motivation in learning [10, 11]. According to self-determination theory, to enhance intrinsic motivation, 3 innate psychological needs, namely competence, autonomy and relatedness, should be satisfied [12]. Students may be intrinsically motivated by a need to achieve physician’s surgical skill competency which extrinsically related them to be a part of medical society. Self-directed practice may provide a feeling of autonomy. Feedback by a surgical teacher about their skill would support their competence and feeling of relatedness. To assess learning, which is a relatively permanent change in a person’s capability to perform a skill [13], 2 measures could be used; (1) comparison of the “entering behavior” to the “post-training behavior” [4], and (2) assessment of “skill retention” which reflects changes in behavior over time [14, 15].

Whether VDO-feedback is as effective as standard face-to-face(F2F) verbal feedback, in terms of improvement of surgical skill is not definitely conclusive. This study aimed to compare two types of feedback in improving surgical skill. Hypotheses of this study were (1) there are differences in performance score improvements after surgical skill practice with F2F- and with VDO- feedback (2) There are differences in retention scores between students who had surgical skill practice with F2F- and with VDO-feedback (3) there are differences in changes of motivation and learning strategies scores between students received different feedbacks, and (4) there are differences in satisfaction scores after surgical skill practice with F2F-and VDO-feedback.

Materials and methods

Subjects

The 4th-year medical students, just before the start of surgical rotation; and the 3rd−-year students, just before the start of clinical years, were voluntarily enrolled via advertisement. The period of study was from 28th December 2020 to 9th April 2021. Students who were interested were given a simple, clear, and thorough explanation about the purposes of the study, roles and benefits of participants, and additional inquiry was allowed until satisfied. They were free to deny, participate, or drop out of the study at any time. Informed consents were obtained prior to participation.

Half of the 4th and half of the 3rd year students were randomly assigned into Group 1: face-to-face feedback and Group 2: video feedback.

Flow of the study

The study was divided into 2 phases: skill learning phase and intervention phase.

Skill learning phase

During the period of 4 or 6 weeks (for 3rd and 4th year, respectively), students would participate in 2–3 sessions of small group teaching of “vertical mattress” suturing. Simulated skin pad with a 3-cm cut wound and 4/0 Polyamide threads were used.

Intervention phase

Face-to-face feedback group (F2F-feedback)

Onsite sessions with a surgical teacher (a researcher) were scheduled every other week for 2 times. During F2F-feedback session, students were asked to perform wound closure using vertical mattress technique. A teacher gave verbal feedback after a few stitches had finished. Each feedback lasted 20 min.

Video-feedback group (VDO-feedback)

Students were taught how to VDO-record their performance using their smartphone and the provided smartphone holder. They were asked to submit a VDO clip demonstrating wound closure with 3 stitches of vertical mattress technique, every other week for 2 times. The submission was done by uploading the VDO-clips to Google Drive™ and sharing them with a surgical teacher (a researcher) via university email. After reviewed, the feedback message was typed and sent back via their email within 1–2 days.

During the 6 weeks of intervention phase, students were asked not to share or discuss about skill training and feedback with their friends. They were not allowed to assess other sources of skill training including additional classroom training, skill workshop, accessing online resources other than the provided demonstration VDO.

Performance assessment

For assessment purposes, students were asked to perform 2 vertical mattress stitches to close a standard 3-cm cut wound on a simulated skin pad within 4 min. Assessment of suturing skill performance were carried out at 4 timepoints; pre-learning test, post-learning test, OSCE, and retention test. For the 4th year students, this item was one of 12 OSCE stations at the end-of-rotation. The retention test was performed 2 weeks after the end-of-rotation when there was no additional practice or teaching (Fig. 1). This point of time was selected because it was 4 weeks from the last feedback and before the students would further gained additional clinical experience.

Fig. 1
figure 1

Flow of the study

Full size image

Each performance was VDO-recorded and later evaluated using a developed performance checklist which is developed from the Objective Structured Assessment of Technical Skills (OSATS) [16]. Items to assess instrument handling, respect of tissue, suture techniques and tissue approximation were rated as perfect, partially correct, or incorrect with appropriate weighted scores. A total sum of 100 points indicated perfect expert skill. The questionnaire was validated for its content by 3 experts and was tried in the 6th year medical students (n = 61). They were asked to perform the same skill which would be VDO-recorded. Two independent surgical teachers were asked to rate their skills using the questionnaire. Internal consistency reliability (Cronbach’s alpha) was 0.66 and inter-rater reliability was 0.55 (95%CI 0.22–0.74, p = 0.002). The inter-rater reliability for the part of instrument handling was 0.62 (95%CI 0.35–9.78, p < 0.001) and for the part of vertical mattress technique was 0.66 (95%CI 0.41–0.81, p < 0.001). For the part of knot tying, the inter-rater reliability was only 0.20 (95%CI 0.39–0.53, p = 0.22).

Assessment of motivation and learning strategies

A questionnaire to assess students’ motivation was derived from publicly available Motivated Strategies for Learning Questionnaire (MSLQ) by Pintrich et al. [17]. For motivation, 3 areas were measured: (1) value (intrinsic and extrinsic orientation, task value), (2) expectancy (learning control belief, self-efficacy) and (3) affect (test anxiety). Learning strategies were also evaluated which divided into 3 domains: (1) cognitive (a. rehearsal, b. elaboration, c. organization, and d. critical thinking), (2) metacognitive (a. planning b. monitoring and c. regulating) and (3) resource management (a. managing time and study environment, b. effort management, c. peer learning and d. help-seeking). Thirty-one items were selected to measure across these topics, using a 7-point Likert scale; ranging from 1 = totally disagree to 7 = totally agree. Internal consistency reliability was 0.84. Questions were modified to match the context of undergraduate medical students. Students were asked to fill this questionnaire twice, at the start-of-course (pre-learning) and at the end-of-course (post-OSCE). Satisfaction to feedback intervention (6 mini-items, 7-point Likert scale), demographic data and practice log were also recorded. Range of scores depended on the number of items in the questionnaires, i.e., minimum was the number of items and maximum was the number of items multiply by 7.

Ethical consideration

Data obtained from participants, both by questionnaires and performance tests was kept confidential. Their identity would not be disclosed, except for the purpose of summative assessment at the end-of-surgical rotation of the 4th year medical students. By design, students in both groups benefited from the feedback intervention. This study has been approved by Institutional Review Boards of Human Research Protection Unit, Faculty of Medicine Siriraj Hospital, Mahidol University (no. 887/2563(IRB2)) and the Ethical Review Board, Faculty of Medicine, Chulalongkorn University (no.764/63). This study has been registered to Thai Clinical Trial Registry (WHO International Clinical Trial Registry Platform) on 11/07/2023 (TCTR20230711005).

Statistical analysis

SPSS for Microsoft Windows version 22. Statistical significance was set at p-value of equal or less than 0.05. Descriptive statistics were used for demographic data. Comparison of performance scores between groups at each timepoint were done by independent sample t-test. Comparison in each group between 2 different timepoints were done by pair sample t-test. Difference in performance scores among post-learning, OSCE and retention timepoints were done by repeated measure ANOVA.

Results

Of the initial 63 interested students, 5 withdrew after small-group learning, leaving 58 eligible students (4th: 3rd year 28:30) to complete the study. Half of each year were randomly assigned into Group 1: face-to-face-feedback (4th: 3rd year 14:15) and Group 2: video feedback (4th: 3rd year 14:15). There was no difference between students’ gender, mean age, grade point average (GPA), handedness, and previous skill training experience between groups (Table 1).

Table 1 Demographic data of F2F- and VDO-feedback groups
Full size table

Performance scores were not different between groups at 3 timepoints; pre-learning, OSCE and retention. However, at post-learning point, F2F-feedback group had significantly higher scores than the VDO-feedback group. This could be related to the higher initial (pre-learning) score of the F2F-feedback group (Table 2).

Table 2 Comparison of mean performance test scores between groups at 4 timepoints (mean ± SD)
Full size table

The higher mean performance scores could infer to improvement of surgical skills closer to the expert level. When scores between each pair of different timepoints were compared (Figs. 2 and 3), significant increase of score were seen after small group learning (pre-learning vs. post-post-learning), after feedback (post-learning vs. OSCE), and after small group learning + feedback (pre-learning vs. OSCE) with p < 0.05. These were found in both groups of feedback (Figs. 2 and 3). There was no difference between scores at retention and OSCE timepoints of both groups. Statistical power of the analysis was assessed by G*power 3.1.9.7 program. Effect size of 1.58 and power of 0.99 were seen. Repeated measure ANOVA with a sphericity assumed shows significant difference in the performance test scores among the post-learning point, OSCE, and retention test: F (2,144) = 42.63, p < 0.001. Significant difference were seen between post-learning and OSCE point (p < 0.001).

Fig. 2
figure 2

Barchart of mean performance test scores at 4 timepoints; prelearning, postlearning, OSCE and retention test

Full size image
Fig. 3
figure 3

Mean performance test scores at postlearning, OSCE and retention timepoints

Full size image

Motivation and learning strategies scores between pre-learning and post-OSCE timepoint were compared. Total scores and sum of scores of both motivation part and learning strategies parts were significantly increased in both groups (Table 3). There was no difference between groups in these scores. In the part of motivation, self-efficacy and test-anxiety scores were improved in both groups. Intrinsic motivation was unchanged in both group and extrinsic motivation was increased only in the VDO-feedback group. In the part of learning strategies, significant increase of cognitive and metacognitive strategies scores was improved in both groups. However, there were no different change in metacognitive self-regulation, management of time and study environment, and help-seeking strategies.

Table 3 Mean modified motivation and learning strategies questionnaire scores
Full size table

At the end of rotation (post-OSCE), satisfaction was assessed in 6 topics (Table 4). Students who participated in VDO-feedback group had slightly higher satisfaction scores towards the intervention, in terms of promotion of comprehension of instrument handling and suturing technique, time-to-feedback, convenience of access to instructor and clarity of perceived information. However, they were less likely to recommend this type of feedback to others.

Table 4 End-of-course students’ satisfaction scores
Full size table

Discussion

This study addresses the problem of lower-than-expected student’s performance of surgical skill. The proposed explanations were inadequate periodic practice and/or inadequate learning. Compared to the previous small-group teaching with informal feedback, this study had added formal feedback intervention, by either face-to-face or video- styles. Students were scheduled for 2 regular feedbacks during the period that they were supposed to carry out self-directed practice of the procedural skill. Feedback interventions during this period had been proven to improve students’ performance significantly after in-class small group learning. There was no significant difference between F2F- and VDO-feedback in this improvement. Both interventions could retain skill performance up to at least 4 weeks later without further practice. Feedback had been demonstrated to be moderately effective in simulation-based procedural skill teaching [18]. For successful outcome in behavior changing, an actionable feedback model was proposed by Hysong et al. [19]. The characteristics of effective feedback were timeliness, personalization, non-punishing, and customizability. Both feedbacks seem to have these characteristics. In terms of timeliness, F2F-feedback was immediate, but VDO-feedback was later but withing 1–2 days. Thus, this duration is likely acceptable.

Both types of feedback were shown to improve motivation and learning strategies scores. A student who is motivated would use more effective learning strategies to achieve his/her academic goal [10]. Intrinsic goal orientation (or intrinsic motivation) was found to have significant effect on learning strategies [20, 21]. Both intrinsic motivation and learning strategies directly affects learning outcome [21]. Extrinsic motivation had less effect on student’s performance than intrinsic motivation [20]. Self-efficacy in learning as shown by self-confidence level had been improved significantly after both types of feedback. Intrinsic motivation score was not changed by the feedbacks, but the extrinsic motivation was significantly increased with VDO-feedback. This may be explained by submitting VDO-record of their performance, they would get an “email” back. In the email, some compliments were added together with the instruction of how to improve which they could review many times. In the F2F-feedback, the verbal compliment and instruction might not be as “objective” as the written ones. Increased test anxiety was supposed to impair student’s performance [22]. However, there were evidences showing benefits of stress to academic [23], and physical performance [24]. Optimal amount of anxiety would drive students to practice for better performance. Although, some evidence showed that anxiety did not seem to affect students’ performance during simulation skills-based learning [25], or OSCE [26]. The absolute amount of optimal anxiety was not known. After the intervention, students had more test anxiety score which was beneficial for the students.

Cognitive and metacognitive strategies inferred to how a student memorized and elaborated information he/she learnt. These comprised rehearsal; a simple recital of procedure steps, elaboration; processes to incorporate the new skills to previously known skills, and organization; a learner’s attempt to construct pattern(s) of new knowledges. At baseline, F2F-feedback group showed significantly higher cognitive and metacognitive strategies scores. However, this difference had been eliminated after feedback intervention, and the post-intervention scores had significantly improved from pre-intervention. The same results were seen in metacognitive self-regulation scores, which is how learners manage themselves in practicing and performing skills. Improvement in these scores were found after both types of feedback and seemed to correlate with the improved performance scores.

Students who participated in VDO-feedback group had slightly higher satisfaction scores towards the intervention, in terms of promotion of comprehension of instrument handling and suturing technique, time-to-feedback, convenience of access to instructor and clarity of perceived information. However, they were less likely to recommend this type of feedback to others. Further study is needed to confirm and explore this finding. Our hypothesis was that VDO-recording is an additional workload for students. Cook et al. [27] had reported reliable association between MSLQ scores with knowledge score and satisfaction scores. This confirms our findings. With the similar efficacy of both feedback intervention on improving basic surgical skill, motivation, learning strategies and satisfaction, VDO-feedback could be an alternative to the classic F2F-feedback in the situation that there is a limited time and space for simultaneous presentation of teacher and students. For example, 1) Clinician-as-a-teacher with limited teaching schedule 2) Clinical-year-apprentice with tight schedule 3) Institute with limited official teaching space and 4) When social distancing is warranted.

Some possible drawbacks of VDO-feedback for procedural skill teaching are (1) learners cannot communicate with teachers in the real time and (2) teachers need to be good at writing communication [9]. Also, things to consider before applying the technique for procedural skill teaching are; 1) A part-task simulation which easy-to-find and easy-to-set up tools may be more suitable. 2) Before students can practice on their own, didactic and/or formal training may be required to ensure correct steps. 3) Students would be better motivated if the feedback is “on-time” and “in-time”. That is a regular interval is scheduled for VDO-submission and a teacher should review and feedback within 24–48 h. 4) Students and teachers should agree on the mode of training.

Conclusion

This study has compared effectiveness of asynchronous electronic feedback after VDO-observation and synchronous face-to-face feedback after direct observation of student’s performance of basic procedural skills. Both feedback interventions were effective in improving and retaining student’s performance skill and also improved student’s motivation and learning strategies. Students shows high satisfaction scores for both interventions. VDO-feedback could be an alternative to face-to-face feedback in teaching procedural skills in the situation when simultaneous presentation of teacher and students are limited, by time, space, and social situation. Further study of VDO-feedback in other context; more complex procedural skill, in other specific group of students, and with different feedback protocol are encouraged.