Mixed reality ultrasound guidance system: a case study in system development and a cautionary tale
Real-time ultrasound has become a crucial aspect of several image-guided interventions. One of the main constraints of such an approach is the difficulty in interpretability of the limited field of view of the image, a problem that has recently been addressed using mixed reality, such as augmented reality and augmented virtuality. The growing popularity and maturity of mixed reality has led to a series of informal guidelines to direct development of new systems and to facilitate regulatory approval. However, the goals of mixed reality image guidance systems and the guidelines for their development have not been thoroughly discussed. The purpose of this paper is to identify and critically examine development guidelines in the context of a mixed reality ultrasound guidance system through a case study.
A mixed reality ultrasound guidance system tailored to central line insertions was developed in close collaboration with an expert user. This system outperformed ultrasound-only guidance in a novice user study and has obtained clearance for clinical use in humans. A phantom study with 25 experienced physicians was carried out to compare the performance of the mixed reality ultrasound system against conventional ultrasound-only guidance. Despite the previous promising results, there was no statistically significant difference between the two systems.
Guidelines for developing mixed reality image guidance systems cannot be applied indiscriminately. Each design decision, no matter how well justified, should be the subject of scientific and technical investigation. Iterative and small-scale evaluation can readily unearth issues and previously unknown or implicit system requirements.
We recommend a wary eye in development of mixed reality ultrasound image guidance systems emphasizing small-scale iterative evaluation alongside system development. Ultimately, we recommend that the image-guided intervention community furthers and deepens this discussion into best practices in developing image-guided interventions.
KeywordsUltrasound guidance Mixed reality Augmented reality Augmented virtuality System development Guidelines Internal jugular vein catheterization Central line
The authors would like to acknowledge Jonathan McLeod, John Moore, and Adam Rankin for their invaluable discussion. This research was funded by The Canadian Foundation for Innovation, # 20994, The Natural Sciences and Engineering Council of Canada, # RPGIN 2014-04504, and The Canadian Institutes for Health Research, # FDN 143232. John S. H. Baxter is supported by the Natural Sciences and Engineering Research Council of Canada through the Canadian Graduate Scholarship program.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
- 3.Ameri G, Baxter JS, McLeod AJ, Peters TM, Chen ECS (2015) Augmented reality ultrasound guidance for central line procedures: preliminary results. In: Workshop on augmented environments for computer-assisted interventions, Springer, pp 11–20Google Scholar
- 5.Baumann M, Mozer P, Daanen V, Troccaz J (2009) Prostate biopsy assistance system with gland deformation estimation for enhanced precision. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 67–74Google Scholar
- 7.Bichlmeier C, Wimmer F, Heining SM, Navab N (2007) Contextual anatomic mimesis hybrid in-situ visualization method for improving multi-sensory depth perception in medical augmented reality. In: 6th IEEE and ACM international symposium on mixed and augmented reality, ISMAR 2007, IEEE, pp 129–138Google Scholar
- 13.Commission IE (2006) Medical device software–software life cycle processes. IEC Standard 62304Google Scholar
- 16.Ender J, Končar-Zeh J, Mukherjee C, Jacobs S, Borger MA, Viola C, Gessat M, Fassl J, Mohr FW, Falk V (2008) Value of augmented reality-enhanced transesophageal echocardiography (TEE) for determining optimal annuloplasty ring size during mitral valve repair. Ann Thorac Surg 86(5):1473–1478CrossRefPubMedGoogle Scholar
- 20.Jenssen C, Brkljacic B, Hocke M, Ignee A, Piscaglia F, Radzina M, Sidhu PS, Dietrich CF (2015) EFSUMB guidelines on interventional ultrasound (INVUS), part vi. Ultraschall Med Eur J Ultrasound 36:E1–E14Google Scholar
- 22.Linte CA, Davenport KP, Cleary K, Peters C, Vosburgh KG, Navab N, Jannin P, Peters TM, Holmes DR, Robb RA (2013) On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic. Comput Med Imaging Gr 37(2):83–97CrossRefGoogle Scholar
- 26.McHugh M, McCaffery F, Casey V (2012) Barriers to adopting agile practices when developing medical device software. In: International conference on software process improvement and capability determination, Springer, pp 141–147Google Scholar
- 27.McHugh M, McCaffery F, Fitzgerald B, Stol KJ, Casey V, Coady G (2013) Balancing agility and discipline in a medical device software organisation. In: International conference on software process improvement and capability determination, Springer, pp 199–210Google Scholar
- 31.Otto CM (2012) The practice of clinical echocardiography. Elsevier Health Sciences, AmsterdamGoogle Scholar
- 33.Paulk MC (2011) On empirical research into scrum adoption. Carnegie Mellon University, PittsburghGoogle Scholar
- 36.Rottier PA, Rodrigues V (2008) Agile development in a medical device company. In: Conference on AGILE’08, IEEE, pp 218–223Google Scholar
- 38.Rust P, Flood D, McCaffery F (2016) Software process improvement roadmaps–using design patterns to aid SMEs developing medical device software in the implementation of IEC 62304. In: International conference on software process improvement and capability determination, Springer, pp 43–56Google Scholar
- 39.Sauer F, Vogt S, Khamene A (2008) Augmented reality. In: Peters T, Cleary K (eds) Image-guided interventions. Springer, Boston, pp 81–119Google Scholar
- 43.Troianos CA, Kuwik RJ, Pasqual JR, Lim AJ, Odasso DP (1996) Internal jugular vein and carotid artery anatomic relation as determined by ultrasonography. Anesthesiology 85(1):43–48Google Scholar
- 45.Van Krevelen D, Poelman R (2010) A survey of augmented reality technologies, applications and limitations. Int J Virtual Real 9(2):1Google Scholar
- 48.Yaniv Z, Linte CA (2016) Applications of augmented reality in the operating room. In: Barfield W (ed) Fundamentals of wearable computers and augmented reality. CRC Press, USA, pp 485–518Google Scholar