Comparison of optical see-through head-mounted displays for surgical interventions with object-anchored 2D-display
- 1k Downloads
Optical see-through head-mounted displays (OST-HMD) feature an unhindered and instantaneous view of the surgery site and can enable a mixed reality experience for surgeons during procedures. In this paper, we present a systematic approach to identify the criteria for evaluation of OST-HMD technologies for specific clinical scenarios, which benefit from using an object-anchored 2D-display visualizing medical information.
Criteria for evaluating the performance of OST-HMDs for visualization of medical information and its usage are identified and proposed. These include text readability, contrast perception, task load, frame rate, and system lag. We choose to compare three commercially available OST-HMDs, which are representatives of currently available head-mounted display technologies. A multi-user study and an offline experiment are conducted to evaluate their performance.
Statistical analysis demonstrates that Microsoft HoloLens performs best among the three tested OST-HMDs, in terms of contrast perception, task load, and frame rate, while ODG R-7 offers similar text readability. The integration of indoor localization and fiducial tracking on the HoloLens provides significantly less system lag in a relatively motionless scenario.
With ever more OST-HMDs appearing on the market, the proposed criteria could be used in the evaluation of their suitability for mixed reality surgical intervention. Currently, Microsoft HoloLens may be more suitable than ODG R-7 and Epson Moverio BT-200 for clinical usability in terms of the evaluated criteria. To the best of our knowledge, this is the first paper that presents a methodology and conducts experiments to evaluate and compare OST-HMDs for their use as object-anchored 2D-display during interventions.
KeywordsMixed reality Intervention Optical see-through head-mounted display User study
Compliance with ethical standards
This study was funded by NIAMS of the National Institutes of Health under award number T32AR067708.
Conflict of interest
The authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article contains a study with human participants, which was approved by the JHU Homewood Institutional Review Board under the numbers HIRB00003228 and HIRB00004665.
Informed consent was obtained from all individual participants included in the study.
- 3.Azimi E, Doswell J, Kazanzides P (2012) Augmented reality goggles with an integrated tracking system for navigation in neurosurgery. In: 2012 IEEE Virtual Reality Workshops (VRW), pp 123–124. doi: 10.1109/VR.2012.6180913
- 5.Bajura M, Fuchs H, Ohbuchi R (1992) Merging virtual objects with the real world: Seeing ultrasound imagery within the patient. SIGGRAPH 26(2):203–210. doi: 10.1145/142920.134061
- 6.Bichlmeier C, Ockert B, Heining SM, Ahmadi A, Navab N (2008) Stepping into the operating theater: ARAV—augmented reality aided vertebroplasty. In: Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. ISMAR ’08, pp 165–166. IEEE Computer Society, Washington, DC. doi: 10.1109/ISMAR.2008.4637348
- 8.Cutolo F, Parchi PD, Ferrari V (2014) Video see through AR head-mounted display for medical procedures. In: 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), p 393–396. doi: 10.1109/ISMAR.2014.6948504
- 9.Hart SG (2006) NASA-task load index (NASA-TLX); 20 years later. In: Proceedings of the human factors and ergonomics society annual meeting, vol 50. Sage Publications, pp 904–908. doi: 10.1177/154193120605000909
- 13.Janin AL, Mizell DW, Caudell TP (1993) Calibration of head-mounted displays for augmented reality applications. In: Proceedings of VR annual international symposium, p 246–255. doi: 10.1109/VRAIS.1993.380772
- 18.Kress B, Starner T (2013) A review of head-mounted displays (HMD) technologies and applications for consumer electronics. Proc SPIE 8720:87200A–87200A-13. doi: 10.1117/12.2015654
- 21.Martin-Gonzalez A, Heining SM, Navab N (2009) Head-mounted virtual loupe with sight-based activation for surgical applications. In: 2009 8th IEEE international symposium on mixed and augmented reality, pp 207–208Google Scholar
- 22.Mentler T, Wolters C, Herczeg M (2015) Use cases and usability challenges for head-mounted displays in healthcare. Curr Dir Biomed Eng 1(1):534–537Google Scholar
- 23.Milgram P, Colquhoun H (1999) A taxonomy of real and virtual world display integration. Mixed Real Merg Real Virtual World 1:1–26Google Scholar
- 26.Qian L, Winkler A, Fuerst B, Kazanzides P, Navab N (2016) Reduction of interaction space in single point active alignment method for optical see-through head-mounted display calibration. In: 2016 IEEE international symposium on mixed and augmented reality (ISMAR-Adjunct), pp 156–157Google Scholar
- 27.Queisner M (2016) Medical screen operations: how head-mounted displays transform action and perception in surgical practice. Mediat 6(1):30–51Google Scholar
- 29.Sadda P, Azimi E, Jallo G, Doswell J, Kazanzides P (2012) Surgical navigation with a head-mounted tracking system and display. Stud Health Technol Inform 184:363–369Google Scholar
- 31.Suthau T, Vetter M, Hassenpflug P, Meinzer HP, Hellwich O (2002) A concept work for augmented reality visualisation based on a medical application in liver surgery. ISPRS Arch 34(5):274–280Google Scholar
- 32.Sutherland IE (1968) A head-mounted three dimensional display. In: Proceedings of the fall joint computer conference, part I, ACM, p 757–764, 9–11 December 1968Google Scholar
- 34.Thrun S, Leonard JJ (2008) Simultaneous localization and mapping. In: Siciliano B, Khatib O (eds) Springer handbook of robotics. Springer, Berlin, pp 871–889Google Scholar
- 35.Tuceryan M, Navab N (2000) Single point active alignment method (SPAAM) for optical see-through HMD calibration for AR. In: Proceedings IEEE and ACM international symposium on augmented reality (ISAR 2000), pp 149–158Google Scholar
- 36.Wagner D, Reitmayr G, Mulloni A, Drummond T, Schmalstieg D (2008) Pose Tracking from natural features on mobile phones. In: Proceedings of the 7th IEEE/ACM international symposium on mixed and augmented reality, Washington, DC, USA, pp 125–134Google Scholar
- 37.Wang H, Wang F, Leong APY, Xu L, Chen X, Wang Q (2015) Precision insertion of percutaneous sacroiliac screws using a novel augmented reality-based navigation system: a pilot study. Int Orthop 40:1–7Google Scholar
- 39.Wilcoxon F, Katti S, Wilcox RA (1970) Critical values and probability levels for the wilcoxon rank sum test and the wilcoxon signed rank test. Sel Table Math Stat 1:171–259Google Scholar
- 40.Yoon JW, Chen RE, Han PK, Si P, Freeman WD, Pirris SM (2016) Technical feasibility and safety of an intraoperative head-up display device during spine instrumentation. Int J Med Robot Comput Assist Surg. doi: 10.1002/rcs.1770
- 41.Yoshida S, Kihara K, Takeshita H, Nakanishi Y, Kijima T, Ishioka J, Matsuoka Y, Numao N, Saito K, Fujii Y (2014) Head-mounted display for a personal integrated image monitoring system: ureteral stent placement. Urol Int 94(1):117–120Google Scholar
- 42.Zhang X, Chen G, Liao H (2016) High quality see-through surgical guidance system using enhanced 3d autostereoscopic augmented reality. IEEE Trans Biomed Eng. doi: 10.1109/TBME.2016.2624632