A randomized controlled trial was carried out to determine whether Perk Tutor, a computerized training platform that displays an ultrasound image and real-time needle position in a three-dimensional (3D) anatomical model, would benefit residents learning ultrasound-guided lumbar puncture (LP) in simulation phantoms with abnormal spinal anatomy.
Twenty-four residents were randomly assigned to either the Perk Tutor (P) or the Control (C) group and asked to perform an LP with ultrasound guidance on part-task trainers with spinal pathology. Group P was trained with the 3D display along with conventional ultrasound imaging, while Group C used conventional ultrasound only. Both groups were then tested solely with conventional ultrasound guidance on an abnormal spinal model not previously seen. We measured potential tissue damage, needle path in tissue, total procedure time, and needle insertion time. Procedural success rate was a secondary outcome.
The needle tracking measurements (expressed as median [interquartile range; IQR]) in Group P vs Group C revealed less potential tissue damage (39.7 [21.3-42.7] cm2 vs 128.3 [50.3-208.2] cm2, respectively; difference 88.6; 95% confidence intervals [CI] 24.8 to 193.5; P = 0.01), a shorter needle path inside the tissue (426.0 [164.9-571.6] mm vs 629.7 [306.4-2,879.1] mm, respectively; difference 223.7; 95% CI 76.3 to 1,859.9; P = 0.02), and lower needle insertion time (30.3 [14.0-51.0] sec vs 59.1 [26.0-136.2] sec, respectively; difference 28.8; 95% CI 2.2 to 134.0; P = 0.05). Total procedure time and overall success rates between groups did not differ.
Residents trained with augmented reality 3D visualization had better performance metrics on ultrasound-guided LP in pathological spine models.
Une étude randomisée contrôlée a été menée afin de déterminer si la plateforme de formation informatisée Perk Tutor, qui affiche une image échoguidée et la position de l’aiguille en temps réel sur un modèle anatomique tridimensionnel (3D), serait utile pour les résidents apprenant à réaliser une ponction lombaire (PL) échoguidée sur des fantômes de simulation avec une anatomie anormale de la colonne.
Vingt-quatre résidents ont été aléatoirement répartis en deux groupes, soit le groupe Perk Tutor (P) et le groupe témoin (T). On leur a demandé de réaliser une PL échoguidée sur des simulateurs de tâches partielles présentant une pathologie rachidienne. Le groupe P a été formé avec l’afficheur 3D en plus de l’échoguidage conventionnel, alors que le groupe T n’a utilisé que l’échoguidage conventionnel. Les deux groupes ont ensuite été testés uniquement à l’aide de l’échoguidage conventionnel sur un modèle rachidien anormal qu’ils n’avaient pas étudié auparavant. Nous avons évalué les lésions tissulaires potentielles, le chemin de l’aiguille dans les tissus, le temps d’intervention total et le temps d’insertion de l’aiguille. Le taux de réussite de l’intervention était l’un des critères de recherche secondaire.
Les mesures de cheminement de l’aiguille (exprimées en tant que médiane [écart interquartile; EIQ]) dans le groupe P vs le groupe T ont révélé une probabilité moindre de lésions tissulaires potentielles (39,7 [21,3-42,7] cm2 vs 128,3 [50,3-208,2] cm2, respectivement; différence 88,6; intervalles de confiance [IC] 95 % 24,8 à 193,5; P = 0,01), un chemin plus court de l’aiguille dans les tissus (426,0 [164,9-571,6] mm vs 629,7 [306,4-2879,1] mm, respectivement; différence 223,7; IC 95 % 76,3 à 1859,9; P = 0,02) et un temps réduit d’insertion de l’aiguille (30,3 [14,0-51,0] sec vs 59,1 [26,0-136,2] sec, respectivement; différence 28,8; IC 95 % 2,2 à 134,0; P = 0,05) dans le groupe P. Le temps total de l’intervention et les taux de réussite globaux étaient semblables dans les deux groupes.
Les résidents formés avec une visualisation 3D augmentée en temps réel ont obtenu de meilleures mesures de la performance en matière de PL échoguidée sur des modèles rachidiens pathologiques.
Use of ultrasound imaging to guide invasive procedures is increasingly becoming the standard of practice in keeping with the goals to improve procedural success rates and decrease complications. Real-time ultrasound imaging allows visualization of the needle tip at the target as well as important adjacent structures. For neuraxial procedures, such as lumbar puncture (LP) and epidural needle insertion,1,2 ultrasound can help identify the midline of the spine, the vertebral interspace, and the optimal angle and depth of needle insertion to reach the spinal canal.3 Despite these potential benefits, a survey on ultrasound-guided spinal needle interventions4 concluded that a significant disadvantage of ultrasound-guided techniques is the long learning curve, requiring expertise in both image acquisition and interpretation. In addition, it requires the ability to guide the needle appropriately while continuously visualizing the needle tip for successful performance.5 Furthermore, interpretation of ultrasound images in obese patients and in those with a diseased spine is even more challenging. It is precisely in these types of patients where traditional landmarks often fail to guide the procedure and where acquiring the skill to visualize the target would be most helpful.
The simulated environment is increasingly being used for learning procedural skills, including LP,6 because it permits repetitive practice while avoiding undue risk to patients. It provides a safe environment to learn and practice technical skills with the advantages of reducing both time pressures and fear of harming the patient.7 It also allows for deliberate practice so that individual trainees can develop both skills and confidence prior to performing the procedure on patients. The simulated environment would therefore seem ideal for learning the complex skills of ultrasound image acquisition and interpretation with simultaneous needle manipulation.
A training platform for ultrasound-guided procedures, the Perk Tutor, has recently been developed to help learners acquire these skills8 in the simulated setting. The Perk Tutor is designed to display not only an ultrasound image but also a real-time three-dimensional (3D) graphic image of the spatial positions of a needle relative to anatomic landmarks. In our view, the ability to visualize the position of the needle in the tissues while navigating the tip of the needle with ultrasound could improve the efficiency and effectiveness of learning. The Perk Tutor can also track trainee performance by computing motion economy parameters and time spent in each workflow step. In previous studies, the Perk Tutor was shown to be beneficial in teaching novices to use ultrasound to guide spinal facet joint injections.9,10
We designed a randomized controlled trial to determine if Perk Tutor could help residents learn to use ultrasound to guide LP in simulation phantoms with abnormal spinal anatomy. Real-time needle position tracking was used as a measure of needle movement within tissue. Specifically, our outcomes were potential tissue damage, needle path in tissue, total procedure time, and needle insertion time. We also recorded success or failure of LP.
Study design and participants
This single-centre trial was performed from March 2013 to July 2014 following approval by the Health Sciences Review Ethics Board (HSREB) of Queen’s University (Kingston, ON, Canada). Teaching staff of the School of Medicine, Queen’s University invited 52 residents to participate in the study through a recruitment letter sent to all residents in the anesthesia and surgery programs. Twenty-four of all invited residents consented to participate and also completed the study. Written informed consent was obtained from all of the residents before participation. The protocol for this trial was not registered.
Participants of this randomized controlled trial were assigned to either the Perk Tutor group (Group P) or the Control group (Group C). Randomization was stratified by number of years spent in training and training program to ensure similar demographics and previous experience between the two groups. To ensure uniform baseline knowledge, all participants received a short presentation on spinal anatomy, the basics of ultrasound, and how to use ultrasound to assist in LP. All participants were then instructed to perform an LP with ultrasound guidance on the three different lumbar spine models (Fig. 2). Training sessions of ten minutes each were dedicated to the healthy spine model (model 1) and the first degenerative model (model 2). During their training sessions, Group P had access to the 3D visualization aid along with the ultrasound image, whereas Group C had only the conventional ultrasound display. The second degenerative spine model (model 3) was used for testing. During the testing phase, both groups had access to only the conventional ultrasound display and not the Perk Tutor, regardless of their group assignment. Although palpation of the phantoms was allowed during testing, participants were informed to expect a degenerative spine model, and they were advised to follow the ultrasound-guided technique. We recorded their needle trajectories in the testing session at a sampling rate of 10 Hz. Testing lasted for a maximum of ten minutes or until positive fluid backflow was observed at the needle hub. Participants used the lumbar interspace of their choice based on landmark and ultrasound information.
Perk Tutor system
Lumbar Puncture Simulator II, a commercial LP phantom (Kyoto Kagaku, Kyoto, Japan), was used as a part-task trainer for the simulated procedure. Computed tomography (CT) images of three real patients were used to create three custom spine inserts for the simulator. The first spine model represented healthy and normal anatomy, the second had severely narrowed intervertebral spaces and mild scoliosis, and the third had severe scoliosis with narrowed intervertebral spaces. The characteristics of the lumbar spine were not visually identifiable from outside the phantom model; however, palpation did reveal potential abnormalities similar to those in a real patient.
The Perk Tutor1 training system consists of a conventional ultrasound machine (SonixTablet, Ultrasonix, Richmond, BC, Canada) equipped with an electromagnetic position tracker (GPS extension, Ultrasonix, Richmond, BC, Canada). The tracker sensors are connected to the spine of the phantom model, the ultrasound transducer, and the needle (Fig. 1). The Perk Tutor displays 3D representations of the ultrasound image and the needle with respect to the anatomical spine model. The computer models of the spine were generated by manual contouring CT images obtained from patients who were challenging cases for spinal anesthesia in the past. The freely available 3D Slicer software was used for contouring and generating the computer models and for spatial registration to the training phantom.11 The Perk Tutor display was positioned behind the spine phantom so the trainee could conveniently view it during needle insertions. The Perk Tutor software can operate on a wide variety of ultrasound machines and position tracking systems, and it is freely available for academic or commercial use from an open-source repository8 (www.perktutor.org). Although the Perk Tutor is not a commercial product, it can be set up and calibrated by technical personnel in a few hours. This system is therefore reproducible at minimal cost. The preparation time before each teaching session took less than five minutes and involved connecting the position sensors to the phantom and needle, filling the phantom spinal canal with water, and loading the training software.
Evaluation of recorded procedures
Recorded trajectories were imported into the Perk Evaluator software module of Perk Tutor for offline analysis. The recorded trajectories were replayed by Perk Evaluator, and performance metrics were automatically calculated as primary outcomes of our study. The research assistant operating the evaluation software was blinded to the group allocation of participants. The recorded needle trajectory consisted of path elements, each defined as a straight line linking two time points 0.1 sec apart. Needle path inside tissue was defined by the sum of all needle tip path elements that were fully inside the phantom. Potential tissue damage was calculated as the sum of phantom tissue areas that the tracked needle would cut through.12 The unit of potential tissue damage was recorded in square centimetres because it was calculated as the total area of triangles between points at the needle tip and where the needle entered the tissue at subsequent recorded times. Potential tissue damage accounted for sideways needle motion and redirection inside the tissue, while needle path depended solely on the length of needle tip motion. Total procedure time was defined from the ultrasound transducer’s initial contact with the phantom surface to the positive fluid feedback (CSF) through the needle. Needle insertion time was calculated as the sum of the periods inside the total procedure time when the needle tip was inside the phantom.
Since performance metrics did not follow a normal distribution, they were statistically compared using the Mann-Whitney test for independent samples. Bonferroni correction was applied to each P value to compensate for multiple comparisons; all reported P values are two sided. Success rate between groups was compared using Fisher’s exact test. Statistical analysis was performed using MedCalc version 9.2 (MedCalc Software, Ostend, Belgium). Measurement results are presented as median [interquartile range; IQR], as recommended for data with non-normal distribution.
Twenty-four anesthesia and surgical residents (PGY 1-5) participated in the study. There was variability in both the LP experience and ultrasound skills among the residents. None of the participants had ever used ultrasound-guidance when performing LP in their usual practice. The two groups did not differ significantly in their demographic data or experience. The characteristics of the two study groups are summarized in Table 1.
Objective performance metrics were significantly different between the groups for potential tissue damage, needle path, and needle insertion time. Needle trajectory statistical analysis revealed that the median [IQR] potential tissue damage was significantly lower (Fig. 3) in Group P than in Group C (39.7 [21.3-2.7] cm2 vs 128.3 [50.3-208.2] cm2, respectively; P = 0.01). In addition, both needle path in tissue and needle insertion time were significantly lower in Group P compared with Group C (426.0 [164.9-571.6] mm vs 629.7 [306.4-2,879.1] mm, respectively; P = 0.02) and (30.3 [14.0-51.0] sec vs 59.1 [26.0-136.2] sec, respectively; P = 0.05). Total procedure time was not significantly lower in Group P compared with Group C (203.8 [135.1-274.9] sec vs 266.9 [221.6-416.2] sec, respectively; P = 0.06). The median differences between Groups P and C in outcome measures are shown in Table 2 along with 95% confidence intervals.
The overall success of the procedure was similar in both groups. In Group C, 11 of 12 LPs were successful, and in Group P, all 12 LPs were successful within the testing time of ten minutes.
The results of our study suggest that the Perk Tutor may benefit residents learning ultrasound-guided LP on a simulated model. Participants in Group P were successful in using ultrasound guidance to maneuver the needle to the target within a shorter distance, within a shorter period of needle insertion time, and with less potential tissue damage to surrounding structures caused by needle movement. The results are encouraging, and it would be important if they translate to the real clinical setting because longer procedure times, large needle movements, and erroneous needle trajectories can result in patient discomfort and potential complications in clinical practice.
In addition to the performance differences noted between the two groups, the median absolute deviation was also lower in the metrics of Group P compared with Group C (e.g., potential tissue damage 10.4 vs 84.3, respectively). This implies more consistent outcomes with the Perk Tutor compared with wider variability in performance with ultrasound alone (Fig. 3). The variability may be the consequence of some participants struggling mentally to project the needle position and the ultrasound display onto the 3D spinal anatomy. This visuospatial coordination is reported to be the key factor in ultrasound-guided needle placement.5 The Perk Tutor system may have facilitated this skill for these participants while playing a lesser role for learners who could already mentally project the needle position relative to the ultrasound plane and the underlying spinal anatomy.
The ultimate procedural success rate between the groups was similar and may be a result of the generous testing time (ten minutes) given to complete the procedure. In a simulated model, there is no concern about patient discomfort and no real tissue injury from multiple needle insertions. The lack of obvious patient discomfort combined with the knowledge that no real trauma is occurring facilitates many needle reinsertion attempts within the allotted time period, thereby increasing the chance of success. In real patients, such factors may limit attempts and affect the overall success rate.
Our study is unique and relevant in several ways. We have shown that real-time augmented reality visualization for ultrasound-guided LP can improve the development of coordination skills when confronted with spinal pathology. This is likely due to merging the spatial positions of the needle and ultrasound images with the 3D anatomy model. This mental projection of needle position in space is an essential skill in all ultrasound-guided procedures. Residents who practiced with the Perk Tutor performed better on the assessment test, although they did not use the assistive visualization technology during the actual testing phase. Additionally, there was wide variability in our participants’ LP experience, yet all were novices in the use of ultrasound to guide the procedure. This is reflective of the characteristics of many clinicians performing spinal anesthesia. The training platform presented in this study may help clinicians who are inexperienced in ultrasound guidance gain these skills in this setting and incorporate them into their practice.
Educators could use such parameters as measuring potential tissue damage and needle in tissue motion economy for objective formative feedback during deliberate practice of procedural skills. Alternatively, these parameters could be added to the summative assessment of learners and incorporated into the discrete milestones that mark the development of procedural skills and the readiness to utilize these skills on real patients. Simulator training gives the advantage of deliberate practice in a pressure-free environment, and previous studies have proven that trainees who first learn on a simulator perform better afterwards and have a higher degree of confidence.13,14 Finally, expert educators are an extremely limited and expensive resource; therefore, if computerized methods can facilitate the acquisition of skills by providing valuable feedback on certain technical components, this could save significant resources in residency training programs. Perk Tutor software is part of the free 3D Slicer software and integrates with most available ultrasound machines and position trackers, thus allowing for low-cost replication at clinical simulation centres. Phantom models are commercially available, and spine models can be customized with recent 3D printing technology.
Our study has several limitations. As with any simulated environment, there is a loss of physical fidelity in both the setting and the patient, and this may affect participant performance. The setting was a research lab and not an actual operating room or block room typical of locations for spinal insertion. The part-task trainer phantom model suffered from artificial tissues and a lack of a whole torso. Nevertheless, both groups were subject to the same simulated conditions, and we made efforts to recreate realistic anatomy and pathological deformations by using actual patients’ spinal CT scans for our models. It is uncertain whether the abilities demonstrated in this simulated setting will translate to improved performance in the real clinical environment. Simulation-based education is generally considered effective,6 and most studies conclude that it is beneficial before transitioning to clinical procedures.15 Results of studies in laparoscopy, colonoscopy and central line insertion have shown that participants achieve better clinical performance in these skills when they are trained on a simulator beforehand.16-19 On the other hand, not all investigations have resulted in a demonstrated improvement in clinical performance after a simulation-based training session, e.g., LP simulation in infants.20
Performance of trainees as measured by the Perk Tutor should be directly compared with their clinical performance. We rely on the technical validity of the system and that it measures features in needle motion that are clinically relevant. Indeed, the risk of failure and serious complications, e.g., spinal hematoma, increases from repeated attempts.21 Perk Tutor parameters, such as motion economy and procedure time, have also been validated as indicators of general technical skills in surgery.22 Therefore, simulation-based training using Perk Tutor should be rigorously evaluated for transfer of skills to the clinical domain. Future studies should compare the performance of residents in their clinical practice with their performance in simulation training. Further follow-up of residents would provide valuable information, not only on how the skills are transferred to the clinical environment but also on skills retention, which is crucial in the design of effective training curricula.
Perhaps the most important limitation to this study is the possibility that our intervention group may have been more skilled from the outset, and the difference between groups may simply be a reflection of this superiority. Attempts were made to mitigate this difference by randomly allocating subjects within junior and senior resident blocks. Also, the demographic data included LP experience, and the uniform inexperience of all participants was confirmed when ultrasound guidance was indicated for the procedure. Furthermore, we were not able to conceal group allocation from the investigators and participants because the Perk Tutor group used the virtual reality display during training sessions. We attempted to mitigate this potential bias by relying on computerized quantitative parameters measured objectively from recorded test sessions.
In summary, our study showed that real-time position tracking and augmented reality 3D visualization may add benefit if incorporated in training for ultrasound-guided LPs, particularly when faced with anatomical spinal abnormalities. Further studies with large cohorts are needed to delineate the full potential of this unique teaching tool.
Peterson MA, Abele J. Bedside ultrasound for difficult lumbar puncture. J Emerg Med 2005; 28: 197-200.
Chin KJ, Chan VW, Ramlogan R, Perlas A. Real-time ultrasound-guided spinal anesthesia in patients with a challenging spinal anatomy: two case reports. Acta Anaesthesiol Scand 2010; 54: 252-5.
Shaikh F, Brzezinski J, Alexander S, et al. Ultrasound imaging for lumbar punctures and epidural catheterisations: systematic review and meta-analysis. BMJ 2013; 346: f1720.
Chen CP, Lew HL, Tsai WC, Hung YT, Hsu CC. Ultrasound-guided injection techniques for the low back and hip joint. Am J Phys Med Rehabil 2011; 90: 860-7.
Smith HM, Kopp SL, Johnson RL, Long TR, Cerhan JH, Hebl JR. Looking into learning: visuospatial and psychomotor predictors of ultrasound-guided procedural performance. Reg Anesth Pain Med 2012; 37: 441-7.
Shanks D, Wong RY, Roberts JM, Nair P, Ma IW. Use of simulator-based medical procedural curriculum: the learner’s perspectives. BMC Med Educ 2010; 10: 77.
Sydor DT, Naik VN. Simulation and competency-based medical education: “showing how”. In: Sherbino J, Frank JR (Eds.) Educational Design: A CanMEDS guide for the health professions. The Royal College of Physicians and Surgeons of Canada; 2011: 59-64.
Ungi T, Sargent D, Moult E, et al. Perk Tutor: an open-source training platform for ultrasound-guided needle insertions. IEEE Trans Biomed Eng 2012; 59: 3475-81.
Moult E, Ungi T, Welch M, Lu J, McGraw RC, Fichtinger G. Ultrasound-guided facet joint injection training using Perk Tutor. Int J Comput Assist Radiol Surg 2013; 8: 831-6.
Bartha L, Lasso A, Pinter C, Ungi T, Keri Z, Fichtinger G. Open-source surface mesh-based ultrasound-guided spinal intervention simulator. Int J Comput Assist Radiol Surg 2013; 8: 1043-51.
Fedorov A, Beichel R, Kalpathy-Cramer J, et al. 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magn Reson Imaging 2012; 30: 1323-41.
Yeo CT, Ungi T. U-Thainual P, Lasso A, McGraw RC, Fichtinger G. The effect of augmented reality training on percutaneous needle placement in spinal facet joint injections. IEEE Trans Biomed Eng 2011; 58: 2031-7.
McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med 2011; 86: 706-11.
Palter VN, Grantcharov TP. Simulation in surgical education. CMAJ 2010; 182: 1191-6.
Nix CM, Margarido CB, Awad IT, et al. A scoping review of the evidence for teaching ultrasound-guided regional anesthesia. Reg Anesth Pain Med 2013; 38: 471-80.
Barsuk JH, McGaghie WC, Cohen ER, O’Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med 2009; 37: 2697-701.
Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 2004; 91: 146-50.
Sedlack RE, Kolars JC. Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study. Am J Gastroenterol 2004; 99: 33-7.
Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002; 236: 458-63.
Kessler DO, Arteaga G, Ching K, et al. Interns’ success with clinical procedures in infants after simulation training. Pediatrics 2013; 131: e811-20.
Moen V, Dahlgren N, Irestedt L. Severe neurologic complications after central neuraxial blockades in Sweden 1990-1999. Anesthesiology 2004; 101: 950-9.
Datta V, Chang A, Mackay S, Darzi A. The relationship between motion analysis and surgical technical assessments. Am J Surg 2002; 184: 70-3.
This work was financially supported as a Collaborative Health Research Project (CHRP #127797), a joint initiative between the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Canadian Institutes of Health Research (CIHR). Financial support was also received from the Southeastern Ontario Academic Medical Organization (SEAMO), Educational Innovation and Research Fund. The preliminary analysis of the results in this manuscript has been presented at The Canadian Anesthesiology Society Annual Meeting in June, 2014.
Conflicts of interest
Devin Sydor contributed significant intellectual content. Tamas Ungi contributed to methodology and to the Methods section of the manuscript and designed the data acquisition system. Parvin Mousavi performed the literature review, and Daniel Borschneck contributed to participant recruitment. Gabor Fichtinger supervised the system design and implementation and contributed to the Methods and Discussion sections. Melanie Jaeger was the main contributor to study concept and design and oversaw and contributed to drafts of the article. Devin Sydor, Robert McGraw, Gabor Fichtinger, Parvin Mousavi, and Daniel Borschneck contributed to the conception and design of the study. Robert McGraw facilitated data acquisition, and Matthew Holden and Daniel Borschneck contributed to data acquisition. Gabor Fichtinger and Melanie Jaeger supervised data acquisition. Zsuzsanna Keri was the main contributor to data acquisition, drafted the article, and prepared figures. Matthew Holden implemented instrument motion metrics software for performance evaluation. Zsuzsanna Keri and Tamas Ungi statistically analyzed the collected data. Matthew Holden made a significant contribution to data interpretation. Daniel Borschneck and Gabor Fichtinger critically revised the article draft. Devin Sydor, Robert McGraw, and Parvin Mousavi critically revised the article.
About this article
Cite this article
Keri, Z., Sydor, D., Ungi, T. et al. Computerized training system for ultrasound-guided lumbar puncture on abnormal spine models: a randomized controlled trial. Can J Anesth/J Can Anesth 62, 777–784 (2015). https://doi.org/10.1007/s12630-015-0367-2
- Lumbar Puncture
- Spine Model
- Deliberate Practice
- Conventional Ultrasound
- Total Procedure Time