A Hybrid 2D/3D User Interface for Radiological Diagnosis
- 502 Downloads
This paper presents a novel 2D/3D desktop virtual reality hybrid user interface for radiology that focuses on improving 3D manipulation required in some diagnostic tasks. An evaluation of our system revealed that our hybrid interface is more efficient for novice users and more accurate for both novice and experienced users when compared to traditional 2D only interfaces. This is a significant finding because it indicates, as the techniques mature, that hybrid interfaces can provide significant benefit to image evaluation. Our hybrid system combines a zSpace stereoscopic display with 2D displays, and mouse and keyboard input. It allows the use of 2D and 3D components interchangeably, or simultaneously. The system was evaluated against a 2D only interface with a user study that involved performing a scoliosis diagnosis task. There were two user groups: medical students and radiology residents. We found improvements in completion time for medical students, and in accuracy for both groups. In particular, the accuracy of medical students improved to match that of the residents.
Keywords3D input Hybrid user interface Diagnostic radiology Medical visualization User interface
We would like to thank all the radiology residents from the Christchurch Hospital and fourth year medical students from the University of Otago who participated in the evaluation of our system. This work was funded by the MARS Bioimaging .
- 2.Hinckley K, Tullio J, Pausch R, Proffitt D, Kassell N. Usability analysis of 3D rotation techniques. Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology. ACM; 1997. p. 1–10.Google Scholar
- 3.Bowman D, Kruijff E, LaViola Jr JJ, Poupyrev IP: 3D User interfaces: theory and practice, CourseSmart eTextbook. Addison-Wesley, 2004.Google Scholar
- 5.zSpace zSpace, inc. https://zspace.com/. Accessed: 2017-01-09.
- 6.Emerson T, Prothero JD, Weghorst SJ: Medicine and virtual reality: a guide to the literature (medVR). Human Interface Technology Laboratory, 1994.Google Scholar
- 10.Hand C. A survey of 3D interaction techniques. Computer graphics forum, vol 16 no 5. Wiley; 1997. p. 269–281.Google Scholar
- 11.Shoemake K. Arcball: a user interface for specifying three-dimensional orientation using a mouse. Graphics Interface; 1992. p. 151–156.Google Scholar
- 13.Bade R, Ritter F, Preim B. Usability comparison of mouse-based interaction techniques for predictable 3D rotation. International Symposium on Smart Graphics. Springer; 2005. p. 138–150.Google Scholar
- 14.Hinckley K, Pausch R, Goble JC, Kassell NF. Passive real-world interface props for neurosurgical visualization. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Celebrating Interdependence. ACM; 1994. p. 452–458.Google Scholar
- 16.Gallo L, De Pietro G, Marra I. 3D Interaction with volumetric medical data: experiencing the wiimote. Proceedings of the 1st International Conference on Ambient Media and Systems. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering); 2008. p. 14.Google Scholar
- 18.Gallo L, Placitelli AP, Ciampi M. Controller-free exploration of medical image data: Experiencing the Kinect. 2011 24th International Symposium on Computer-based medical systems (CBMS). IEEE; 2011. p. 1–6.Google Scholar
- 20.Balakrishnan R, Baudel T, Kurtenbach G, Fitzmaurice G. The Rockin’Mouse: integral 3D manipulation on a plane. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems. ACM; 1997. p. 311–318.Google Scholar
- 23.Bérard F, Ip J, Benovoy M, El-Shimy D, Blum JR, Cooperstock JR. Did Minority Report get it wrong? Superiority of the mouse over 3D input devices in a 3D placement task. IFIP Conference on Human-Computer Interaction. Springer; 2009. p. 400–414.Google Scholar
- 24.Wang G, McGuffin MJ, Bérard F, Cooperstock JR. Pop-up depth views for improving 3D target acquisition. Proceedings of Graphics Interface 2011. Canadian Human-Computer Communications Society; 2011. p. 41–48.Google Scholar
- 25.Feiner S, Shamash A. Hybrid user interfaces: breeding virtually bigger interfaces for physically smaller computers. Proceedings of the 4th annual ACM symposium on User interface software and technology. ACM; 1991. p. 9–17.Google Scholar
- 27.Angus IG, Sowizral HA. Embedding the 2D interaction metaphor in a real 3D virtual environment. IS&T/SPIE’s Symposium on Electronic Imaging: Science & Technology. International Society for Optics and Photonics; 1995. p. 282–293.Google Scholar
- 28.Hachet M, Guitton P, Reuter P. The CAT for efficient 2D and 3D interaction as an alternative to mouse adaptations. Proceedings of the ACM Symposium on Virtual Reality Software and Technology. ACM; 2003. p. 225–112.Google Scholar
- 29.Darken RP, Durost R. Mixed-dimension interaction in virtual environments. Proceedings of the ACM Symposium on Virtual Reality Software and Technology. ACM; 2005. p. 38–45.Google Scholar
- 30.Wang J, Lindeman RW. Object impersonation: towards effective interaction in tablet-and HMD-based hybrid virtual environments. 2015 IEEE Virtual Reality (VR). IEEE; 2015. p. 111–118.Google Scholar
- 31.Wloka M. Interacting with virtual reality. Virtual Prototyping. Springer; 1995. p. 199–212.Google Scholar
- 32.Coninx K, Van Reeth F, Flerackers E. A hybrid 2D/3D user interface for immersive object modeling. Proceedings of Computer Graphics International, 1997. IEEE; 1997. p. 47–55.Google Scholar
- 33.Rekimoto J. Pick-and-drop: a direct manipulation technique for multiple computer environments. Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology. ACM; 1997. p. 31–39.Google Scholar
- 34.Ullmer B, Ishii H. The metaDESK: models and prototypes for tangible user interfaces. Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology. ACM; 1997. p. 223–232.Google Scholar
- 36.Arvanitis TN. Virtual reality in medicine. Handbook of Research on Informatics in Healthcare and Biomedicine. IGI Global; 2006. p. 59–67.Google Scholar
- 37.Pensieri C, Pennacchini M. Virtual Reality in medicine. Handbook on 3D3C Platforms. Springer; 2016. p. 353–401.Google Scholar
- 38.Baumgärtner S, Ebert A, Deller M, Agne S. 2D meets 3D: a human-centered interface for visual data exploration. CHI’07 Extended Abstracts on Human Factors in Computing Systems. ACM; 2007. p. 2273–2278.Google Scholar
- 39.Bornik A, Beichel R, Kruijff E, Reitinger B, Schmalstieg D. A hybrid user interface for manipulation of volumetric medical data. IEEE Symposium on 3D User Interfaces, 3DUI 2006. IEEE; 2006. p. 29–36.Google Scholar
- 45.DCMTK OFFIS DICOM toolkit. http://dicom.offis.de/dcmtk.php.en. Accessed: 2017-01-12.
- 46.Qt API Th Qt Company. https://www.qt.io/ui/. Accessed: 2017-01-12.
- 47.OpenGL API Khronos Group. https://www.opengl.org/. Accessed: 2017-01-12.
- 48.CUDA API NVIDIA Corporation. https://developer.nvidia.com/cuda-toolkit. Accessed: 2017-01-12.
- 49.Inteleviewer Intelerad Medical Systems Incorporated. http://www.intelerad.com/en/products/inteleviewer/. Accessed: 2017-01-09.
- 50.Lorensen WE, Cline HE. Marching cubes: A high resolution 3D surface construction algorithm. ACM Siggraph Computer Graphics. ACM; 1987. p. 163–169.Google Scholar
- 51.Dai Y, Zheng J, Yang Y, Kuai D, Yang X: Volume-Rendering-Based Interactive 3D measurement for quantitative analysis of 3D medical images. Computational and mathematical methods in medicine, vol 2013, 2013.Google Scholar
- 52.Preim B, Tietjen C, Spindler W, Peitgen HO. Integration of measurement tools in medical 3D visualizations. Proceedings of the Conference on Visualization’02. IEEE Computer Society; 2002. p. 21–28.Google Scholar
- 53.Human Ethics Committee university of canterbury. http://www.canterbury.ac.nz/humanethics/. Accessed: 2016-06-01.
- 54.Brant WE, Helms CA: Fundamentals of diagnostic radiology. Lippincott Williams & Wilkins, 2012.Google Scholar
- 55.Drebin RA, Carpenter L, Hanrahan P. Volume rendering. ACM Siggraph Computer Graphics, vol 22 no 4. ACM; 1988. p. 65–74.Google Scholar
- 56.Cox GM, Cochran W: Experimental designs. JSTOR, 1953.Google Scholar
- 57.Brooke J, et al. SUS-a quick and dirty usability scale. Usability Evaluation in Industry 1996;189(194):4–7.Google Scholar
- 59.Maxwell SE, Delaney HD. Designing experiments and analyzing data: a model comparison perspective. Psychology Press, vol 1, 2004.Google Scholar
- 60.Hartung J, Knapp G, Sinha BK: Statistical meta-analysis with applications. Wiley, vol 738, 2011.Google Scholar
- 62.Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 2009;4(3):114–123.Google Scholar
- 63.Patterson RE: Human factors of stereoscopic 3D displays. 1270 Springer, 2015.Google Scholar
- 64.Frees S, Kessler GD. Precise and rapid interaction through scaled manipulation in immersive virtual environments. IEEE Proceedings of Virtual Reality, 2005. VR 2015. IEEE; 2005. p. 99–106.Google Scholar
- 65.MARS Bioimaging Ltd. http://www.marsbioimaging.com/mars/. Accessed: 2017-01-09.