Introduction

Time and practice are necessary for an orthopaedic resident to learn and perform musculoskeletal surgery. Time constraints from work-hour regulations, time pressures on surgeon teachers, and the limited experience with using tools that many residents bring to orthopaedic programs make learning these skills more challenging [16, 18]. Accomplishing this, with both time and financial sensitivity, is a challenge that requires creative solutions, including efficient teaching models, creative and inexpensive skills modules, and motivating performance criteria [10]. The classical learning models that are presently used, especially those that teach procedural skills, are similar to traditional apprenticeship-style approaches and involve emulation, self-shaping (learning) of complex activities, and extend for the duration of the residency [1, 6]. Such models can be tedious and demotivating, resulting in slow progress. This is in part the result of teaching procedures that generally focus on what students do incorrectly.

An alternative to the classic teaching models is a teaching procedure that is based on operant learning. The foundational tenet of operant learning is that a behavior is more likely to occur if it is reinforced. Behaviors in humans have been successfully reinforced using the acoustic stimulus from a clicker, where the clicker serves as a conditioned reinforcer that marks the moment at which the criterion has been achieved in a way that is language- and judgment-free [13, 14]. An example of this method is the teaching procedure TAGteach (teaching with acoustic guidance) (TAGteach International, Indian Trail, NC, USA) that uses operant learning methodologies (including a variety of conditioned reinforcers) along with a process known as chaining [13, 17]. Chaining is the act of breaking complex activities into simpler behavior links, reinforcing those behavior links, and then stringing those learned behavior links back together into the desired complex behavior chain [13, 17]. When the learned complex behavior (the skill) becomes automatic through reinforcement, deliberate practice, and repetition, the learner’s full attention can then be focused on performing the complex task, in any environment, with an expectation of success [5, 13, 20]. Operant learning with deliberate practice is not a new concept and has been used in a variety of human athletic and performance activities successfully enhancing performance in football lineman, golfers, pole-vaulters, and dancers [7, 14, 19], but to our knowledge, a program that takes advantage of these concepts and uses them for orthopaedic skill training has not been tested before.

For this project, we compared two teaching strategies: (1) an operant learning methodology (adapted from TAGteach); and (2) a more classical approach using demonstration alone. Our hypothesis was that a group that is taught a surgical skill using an operant learning procedure will be able to more precisely perform that skill than a group that is taught by demonstration alone. To accomplish this, we identified two tasks from our surgical skills program, “tying the locking, sliding knot” and “making a low-angle drill hole,” and taught those skills to the test learners using an operant learning procedure. We compared the learners’ fluency (accuracy and time of performance) achieved with the operant learning approach with that achieved using a typical demonstration approach often used in surgical teaching.

Materials and Methods

The Albert Einstein College of Medicine institutional review board approved testing and found the project exempt because of its use of normal educational practices. Sixteen tool-specific modules (with 72 behavior-specific, submodules) were designed for and presented to the 2014 Postgraduate Year (PGY)-1 orthopaedic resident class Two of these submodules, “tying the locking, sliding knot” and “making a low-angle drill hole,” were selected for evaluation in this study. Both of the selected submodules were composed of steps with performance criteria that could be consistently measured as successful or not successful.

Tying the Locking, Sliding Knot

Both the test group and the control group were asked to learn to tie a locking, sliding knot (a Tennessee Slider variant) using a two-foot piece of one-fourth-inch braided nylon rope.

The test group was composed of six PGY-1 residents (three men and three women) and six volunteer, randomly selected, first-year and second-year medical students (five men, one woman). The control group was composed of five PGY-1 and PGY-2, nonorthopaedic, surgical residents (three men, two women) and six volunteer, first- and second-year medical students (five men, one woman).

Each test learner was taught, one-on-one, by a single senior surgeon who was trained in operant learning methods (IML), who used operant methodology and followed a precise script (Table 1); the goal, “tying the locking, sliding knot,” was identified, brief background information about the purpose of the knot was given followed by a demonstration of tying the knot. The first component step (behavior) of the complex knot-tying behavior was identified. Instructions were then given and that component behavior was demonstrated. The tag point, that is, the aspect of the behavior to be marked with acoustic feedback (a click), was named using five words or less. The learner was then asked to reproduce that behavior. A correct performance of the behavior (achieving criteria) was marked with a clicker and repeated five times. Each component behavior was learned in succession and then linked together into a behavior chain until the behavior of “tying the locking, sliding knot” was completed. The time to achieve the first correct knot was recorded. The time started with the first instruction and ended with completion of the first correctly performed knot. The behavior, “tying the locking, sliding knot,” was repeated and marked five times. The learners were then asked to practice tying the knot for 15 minutes. Once the behavior was acquired, the 15-minute practice time allotted allowed for practice of at least 100 knots. Practice was done in isolation with no verbal instructions, diagrams, or video available.

Table 1 The script for the submodule ‘tying the locking, sliding knot
Full size table

Each control learner received two demonstrations with verbal instructions of the locking, sliding knot, by a single instructor (IML). The control learners were then given an exploded diagram of the knot and asked to construct it. The time to the first correct knot was recorded. The time started with the beginning of the demonstration and ended on completion of the first correctly made knot. They were given 15 minutes (900 seconds) to complete the task. Technique was not commented on or corrected at this point. After successfully completing the knot, the learners were then asked to practice tying the knot for 15 minutes. Practice was done in isolation with no verbal instructions or video available.

Both the test and control learners were then asked to perform 10 knots. Precision of the knot-tying technique along with speed of performance (fluency) was observed and recorded by a single observer (IML), one learner with the observer at a time. To precisely perform the knot-tying technique, the learner had to successfully complete each of the critical steps. For the locking, sliding knot, there were six steps (Fig. 1A–E): (1) place one-third of the rope directly over two-thirds of the rope (Fig. 1A); (2) wrap, then go over and through (Fig. 1B); (3) wrap to a fakey pinch (the rope coming from palm side to the thumb-index pinch [Fig. 1C]); (4) maintain the loop (Fig. 1C); (5) go behind over and through (Fig. 1D); and (6) dress and slide (Fig. 1E). We determined the performance of a component step to have been completed successfully or not completed. Performance of the complex behavior was successful only if all of the component steps were performed successfully. We then timed the surgeon-instructor tying 10 knots correctly, repeating the behavior five times, to establish a best time target to use as the speed component for fluency determination.

Fig. 1A–E
figure 1

(A) The figure shows component behavior 1. The goal is to pinch one-third of the rope with your left hand, placing it directly over the two-thirds of the rope pinched with the right (“pinch” with index and thumb is a foundation behavior that has been previously taught). The TAGpoint (position to be marked with the clicker) is “over.” (B) The figure shows component behavior 2. The goal is to wrap the long end around the left palm, then over the rope pinched with the left hand, and finally between the two limbs of the rope. The TAGpoint is “through.” (C) The figure shows component behaviors 3 and 4. The goal of behavior 3 is to wrap the long end of the rope around your right palm and then hold it with a fakey pinch. (“Fakey pinch” is a foundation behavior that has been previously taught.) The TAGpoint is “fakey pinch.” The goal of behavior 4 is to maintain the loop built in behavior 3. The TAGpoint is “maintain loop.” (D) The figure shows component behavior 5: the goal is to maintain the loop, pass the long end behind and then over the left pinched rope, pulling the rope through the loop with a “backside grab.” (“Backside grab” is a foundation behavior that has been previously taught.) The TAGpoint is “backside grab.” (E) The figure shows component behavior 6. Dress, tighten, and deliver. The TAGpoint is “deliver.”

Full size image

The success of technique for tying the knot was compared for the two groups using the Wilcoxon rank-sum test for median comparison. The median time (in seconds) to the first knot between test and control groups was compared using Wilcoxon rank-sum test. The time (in seconds) needed to perform 10 knots after 15 minutes of practice was compared for the two groups using the two-sample t-test for mean comparison.

Making a Low-angle Drill Hole

For this submodule, both the test group and the control group were asked to learn to make a low-angle drill hole (for this case, a drill hole that was 45° to a tangent of a circular polyvinyl chloride [PVC] pipe) using a one-eighth-inch drill bit and a 12-volt cordless drill (DeWALT Industrial Tool Co, Baltimore, MD, USA) drilled into a two-foot piece of ¾-inch PVC pipe marked with lines along the longitudinal axis of the pipe that were spaced 120° apart.

The low-angle drill hole test group was composed of eight first- and second-year medical students (five men and three women) who were randomly selected from a group of 16 who volunteered. For the low-angle drill hole control group, eight first- and second-year medical students (seven men and one woman) were randomly selected from the same group.

Each test learner was taught, one-on-one, by a single senior surgeon who was trained in operant learning methods (IML), used an operant methodology, and followed a precise script (Table 2); the goal, “making a low-angle drill hole,” was identified, brief background information about the purpose of the drill hole was given followed by a demonstration of making the low-angle drill hole. The first component step (behavior) of “making a low-angle drill hole” was identified. Instructions were then given and that component behavior was demonstrated. The aspect of the behavior to be marked (the tag point) was named with five words or less. The learner was then asked to reproduce that behavior. A correct performance of the behavior (the criteria were achieved) was marked (with a clicker) and repeated five times. Each of the six component behaviors were taught, learned in succession, and linked together into a behavior chain until the behavior of “making a low-angle drill hole” was completed. The behavior “making a low-angle drill hole” was repeated and marked five times. The learners were then asked to practice the drill hole for 10 minutes on 12 inches of marked ¾-inch PVC. Learners averaged 20 practice holes and were limited by the available space on the practice pipe. Practice was done in isolation with no verbal instructions, diagrams, or video available.

Table 2 The script for the submodule ‘making a low-angle drill hole’
Full size table

Each control learner received two demonstrations with verbal instructions for making a low-angle drill hole by a single instructor (IML). The control learners were then tested.

Both the test and control groups were asked to make 10 low-angle drill holes in PVC pipe. Starting points and targets were 120° apart and were marked so the drill holes were identical for all tests. Precision of the drilling technique along with speed of performance were observed and recorded by a single observer (IML). To precisely perform the drilling technique, the learner had to successfully complete each of the critical steps. There were six steps for “making a low-angle drill hole”: (1) grip the drill; (2) locate the drill point; (3) stabilize the drill tip perpendicular to the surface; (4) make a dimple while running the drill at low speed; (5) in the dimple, angle the drill; and (6) drill the hole and back the drill out at full speed. Drill point slips occurring while stabilizing the drill, making a dimple, or angling the drill were recorded. The component step performance was determined to be “completed successfully” or “not completed successfully.” Accuracy was evaluated and determined to be successful if the drill exited within a target zone. The boundaries of the target zone were the two drawn longitudinal lines, 120° apart. A drill exiting within the confines of the lines or piercing the lines was considered within the target zone. Any drill piercing at greater than 120° was considered a failure. Performance of the complex behavior was considered successful only if all of the component steps were performed successfully with no slips and with the drill emerging in the target zone. The surgeon-instructor was then asked to drill 10 holes precisely and repeat the behavior five separate times to establish a best time target to use as the speed component for fluency determination.

The median number of technique successes for drilling a low-angle hole was compared for the two groups using the Wilcoxon rank-sum test for median comparison. The mean time (in seconds) to create 10 drill holes was compared for the test and control groups using a two-sample t-test for mean comparison.

Results

Tying the Locking, Sliding Knot

The operant learning (test) group achieved better precision tying the locking sliding knot when compared with the control group but took longer to acquire the first knot. Twelve of the 12 test group learners tied the knot precisely 10 times compared with only four of the 11 in the control group (Table 3). The median number of test group learners who tied correct knots was 10 (range, 10–10) and the median number of control group learners who tied correct knots was 0 (range, 0–10). Using the Wilcoxon rank-sum test for median comparison, the test group was determined to be more precise (p = 0.004) (Table 4). The odds ratio could not be estimated accurately as a result of the small sample size and the 100% success rate in the test group.

Table 3 ‘Tying the locking, sliding knot’--time to first knot, time for 10 knots after practice, and number of knots with the behavior criteria achieved for the test and control groups
Full size table
Table 4 ‘Tying the locking, sliding knot’: time comparisons between test and control groups
Full size table

The median “time to the first knot” for the test group (median 271 seconds [range, 184–626 seconds]) and control groups (median 163 seconds [range, 93–900 seconds]) was compared. Using the Wilcoxon rank-sum test, the test group was determined to take longer than the control group to achieve the first knot (p = 0.017) (Tables 3, 4).

After the test, the control group learner who was unable to construct the knot after 900 seconds was taught to construct the knot using the operant learning methodology in 201 seconds.

The 12 test group members averaged 95 seconds ± SD = 15 (range, 67–120 seconds) to complete the 10-knot task and the 10 control group members averaged 95 seconds ± SD = 28 (range, 62–139 seconds) to complete the 10-knot task (Tables 3, 4). Using the two-sample t-test for mean comparison, it was determined that there was no difference between the mean time for tying 10 knots for the test group and control groups after 15 minutes of practice (p = 0.996) (Table 4).

No learner reached the mean fluency ceiling (precision and speed of performance) for successfully performing 10 knots established by the surgeon-instructor (mean 61 seconds ± SD = 4.9 (range, 56–69 seconds).

Making the Low-angle Drill Hole

The operant learning group achieved better precision but averaged longer time to perform the low-angle drill task than did the control group. The technique of the test group more consistently achieved the ideal drilling behavior compared with the control group (p = 0.006 for the median number of technique success comparison with an odds ratio of 82.3 [29.1–232.8] [for a 95% confidence interval {CI}]). It is important to note that three members of the control group damaged their drill bits during the 10 low-angle drill hole tests (Table 5).

Table 5 Drill time and technique success comparison between test and control groups
Full size table

The mean time to drill 10 low-angle drill holes for the test group was 193 seconds (range, 153–222 seconds) and the mean time to drill 10 low-angle drill holes for the control group was 146 seconds (range, 91–294 seconds) (Table 5). With the numbers available, there was no difference between the test group and control group in terms of the time it took to complete 10 low-angle drill holes (test group mean was 193 seconds ± SD = 26 and the control group mean was 146 seconds ± SD = 63, p = 0.084) (Tables 5, 6). No learner reached the mean fluency ceiling, to perform 10 drill holes successfully, established by the surgeon-instructor (mean 94 seconds [range, 84–104 seconds]).

Table 6 ‘Making a low-angle drill hole’–time for 10 holes, number of slips, and number of precise drilling behaviors
Full size table

Discussion

Substantial effort has been made by the surgical and orthopaedic communities to create learning platforms and use effective teaching methodologies to train surgical procedural skills [1]. To acquire fluency with these complex (composite) skills, a learner needs to be both accurate and fast (fluent) with the component (foundation) behaviors of those skills [2, 3]. Operant learning methodologies effectively shape new behaviors, incrementally, by using a high rate of reinforcement, event markers to isolate and identify the desired physical movement, and by connecting or “chaining” foundation behaviors into the desired complex behaviors. Instruction, demonstration, expert feedback, video recording, and computer analysis have all been used to improve technical skill performance and have met with varied success. However, these procedures often use criticism, reprimand, public correction, and other punitive methods, which can interrupt and delay learning. In addition, overestimating simulation realism (fidelity) may result in missing the desired outcome as a result of cognitive and sensory overload [4]. By using an operant learning procedure, these punitive approaches can be avoided. Operant learning procedures have been used successfully with a variety of human athletes and performers (football linemen, golfers, pole-vaulters, and dancers), but to our knowledge, a program that takes advantage of these concepts and uses them for orthopaedic skill training has not been tested before [7, 14, 19]. We used an operant learning model for two specific complex behaviors, “tying a locking, sliding knot” and “making a low-angle drill hole.” Behaviors were established using precise scripts to generate those behaviors and clickers to mark achievement of the behavior in a nonjudgmental manner. For the learning model presented here, the operant test group more consistently achieved precision of performance than when only demonstration was used. However, the time to first success and the time of performance were faster in the demonstration/control group. Reinforcement and the resulting behavior can take longer to build and longer to perform but the precision of that behavior is undoubtedly worth the time.

An important limitation of this study is the control comparison. We have attempted to emulate approaches that we commonly see, in which the skill is demonstrated (but not necessarily broken into its component parts) or where the learner is allowed to self-shape critical aspects of the skill. However, to more completely validate our operant approach, we need to compare other learning methods that more rigorously teach the component elements but do not use positive reinforcement. Also, although our expectation is that operant learning will be effective over the entire skill learning landscape, the present study cannot make such claims. In this study, we have only evaluated two submodules of 72 (Table 7). (Future studies will need to evaluate these submodules to determine to what degree an operant teaching method is effective in disseminating other relevant skills and psychomotor tasks.

Table 7 Fourteen tools were taught in 16 modules*
Full size table

Another area of potential concern is the composition of the test and control groups. Both were made up of combinations of residents and medical students and for one control group, nonorthopaedic surgical residents. None of the residents or medical students had any prior experience with the technical challenges they were presented with and none had any experience with operant learning. The medical students were part of the medical school’s orthopaedic interest group and were motivated to be successful because they were performing in front of the orthopaedic residency program director. The nonorthopaedic surgical group was used for the knot-tying challenge and demonstrated no differences in knot-tying skill acquisition. Finally, the fact that in both tests, the surgeon-instructor was able to outperform the test and control groups suggests that the learners may not have achieved their performance ceiling in 15 or 10 minutes of practice. In future studies, extending the practice period for longer durations may be necessary to evaluate if fluency has been achieved.

Orthopaedic residents are adult learners and as such a teaching methodology needs to be centered on the learner, the first steps of which should be modeling, coaching, and scaffolding [15]. Surgical skill instructors have attempted to do this. Expert instruction enhances performance of simple surgical procedures, specifically joint injection and knot-tying [8, 9, 12]. Evidence also suggests that the use of computer-based video instruction can be effective in teaching learners surgical skills such as knot-tying [11, 21]. However, whether it is laparoscopic simulation, knot-tying and suturing, using power tools, or doing carpal tunnel surgery, the present teaching models are generally taught as a demonstration/correction or a self-directed/correction couple. Correction is demotivating. In addition, unsupervised repetition of incompletely formed behaviors can build in mistakes or habits that will increase effort and reduce efficiency of performance. Only when students can perform at half the proficiency skill are they likely to participate in and benefit from independent practice [3].

In contrast, operant learning occurs as the behavior is constructed and is highly reinforced with the result measured, not in the time saved, but in the ultimate outcome. In this study, the behavior is built step by step and precisely formed with the aid of an instructor who uses a precise script and acoustic markers to indicate the behavior criteria have been achieved. The students taught using operant methods were precise, whereas our control students had a much higher rate of error; they tended to omit crucial elements of the skill resulting in failure of performance or even damage to the tool (bent drill bits). Our findings suggest that the use of operant learning to teach the foundation behaviors necessary to use the tools of the operating room is effective and can be expected to create learners that use those tools with precision. What is the disadvantage? No doubt, to accomplish this will take effort on the part of teachers to learn the language and methods of behavioral science and operant methodology; as the new tool skills need to be learned, so will the new strategies for teaching them. In the end, our goal is to define a methodology for teaching our orthopaedic residents the fundamental tool skills in a way that is positive, time-efficient, reproducible and motivating, and allows the learner to pass on the acquired knowledge in the same precise way he or she learned it.