Virtual reality in advanced medical immersive imaging: a workflow for introducing virtual reality as a supporting tool in medical imaging

Abstract

Radiologic evaluation of images from computed tomography (CT) or magnetic resonance imaging for diagnostic purposes is based on the analysis of single slices, occasionally supplementing this information with 3D reconstructions as well as surface or volume rendered images. However, due to the complexity of anatomical or pathological structures in biomedical imaging, innovative visualization techniques are required to display morphological characteristics three dimensionally. Virtual reality is a modern tool of representing visual data, The observer has the impression of being “inside” a virtual surrounding, which is referred to as immersive imaging. Such techniques are currently being used in technical applications, e.g. in the automobile industry. Our aim is to introduce a workflow realized within one simple program which processes common image stacks from CT, produces 3D volume and surface reconstruction and rendering, and finally includes the data into a virtual reality device equipped with a motion head tracking cave automatic virtual environment system. Such techniques have the potential to augment the possibilities in non-invasive medical imaging, e.g. for surgical planning or educational purposes to add another dimension for advanced understanding of complex anatomical and pathological structures. To this end, the reconstructions are based on advanced mathematical techniques and the corresponding grids which we can export are intended to form the basis for simulations of mathematical models of the pathogenesis of different diseases.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. 1.

    ProMesh is a free software which allows for visualization of unstructured grids in 2D and 3D and also for the improvement of surface and volume grids respectively tetrahedralization

  2. 2.

    A software standard defines how a format or a language has to be implemented such that each one who is interested in applying it may understand how to use it. The tiff standard defines the byte and bit order of a tiff image which allows the image to be called “tiff”.

  3. 3.

    A mesh is a set of of basic elements which describe a set. For example, a triangular mesh describes a surface. The surface is constructed by means of triangles. A connected surface hence is constructed by means of single triangles which all have neighbors which also belong to the surface. In particular, the nodes which describe the single triangles (three for each triangle) are shared with neighbor elements, in particular those who are not at an outer boundary (for open surfaces).

References

  1. 1.

    Community. Osirix software. http://www.osirix-viewer.com/

  2. 2.

    MITK. Software. http://mitk.org/wiki/MITK

  3. 3.

    Wössner, U., Aumüller, M., Lang, U.: Driving an active stereo cave with a cluster of pcs. VR-Cluster 03: Workshop on Commodity Clusters for Virtual Reality, IEEE VR, Los Angeles, (2003)

  4. 4.

    Hoffer, M., Poliwoda, C., Wittum, G.: Visual reflection library: a framework for declarative GUI programming on the Java platform. Comput. Vis. Sci. 16, 181 (2013). https://doi.org/10.1007/s00791-014-0230-y

    Article  Google Scholar 

  5. 5.

    Rantzau, D., Frank, K., Lang, U., Rainer, D., Wössner, U.: Covise in the cube: An environment for analyzing large and complex simulation data. In: Proceedings of the 2nd workshop on immersive projection technology (IPT 98), Ames, Iowa (1998)

  6. 6.

    Vogel, A., Reiter, S., Rupp, M., Nägel, A., Wittum, G.: Ug 4: a novel flexible software system for simulating pde based models on high performance computers. Comp. Vis. Sci. 16(4), 165–179 (2013)

    Article  MATH  Google Scholar 

  7. 7.

    Reiter, S., Vogel, A., Heppner, I., Rupp, M., Wittum, G.: A massively parallel geometric multigrid solver on hierarchically distributed grids. Comput. Vis. Sci. 16(4), 151–164 (2013)

    Article  MATH  Google Scholar 

  8. 8.

    http://dicom.nema.org/. Dicom standard. web page, (2011)

  9. 9.

    Lorensen, William E., Cline, Harvey E.: Marching cubes: a high resolution 3D surface construction algorithm. Comput. Gr. 21(4), 163–169 (1987)

    Article  Google Scholar 

  10. 10.

    Newman, T.S., Yi, H.: A survey of the marching cubes algorithm. Comput. Gr. 30(5), 854–879 (2006)

    Article  Google Scholar 

  11. 11.

    Reiter, S.: Effiziente algorithmen und datenstrukturen für die realisierung von adaptiven, hierarchischen gittern auf massiv parallelen systemen. Ph.D. thesis University of Frankfurt (2015)

  12. 12.

    Reiter, S., Wittum, G.: http://promesh3d.com/ (2017)

  13. 13.

    International Organization for Standardization. Tiff standard. https://www.iso.org/standard/2181.html (1998)

  14. 14.

    Wavefront Technologies. Wavefront obj standard. http://www.martinreddy.net/gfx/3d/OBJ.spec (1989)

  15. 15.

    Brooks, Rodney A.: A quantitative theory of the hounsfield unit and its application to dual energy scanning. J. Comput. Assist. Tomogr. 1(4), 487–493 (1977)

    Article  Google Scholar 

  16. 16.

    Broser, P.J., Schulte, R., Roth, A., Helmchen, F., Waters, J., Lang, S., Sakmann, B., Wittum, G.: Nonlinear anisotropic diffusion filtering of three-dimensional image data from 2-photon microscopy. J. Biom. Opt. 9(6), 1253–1264 (2004)

    Article  Google Scholar 

  17. 17.

    Jungblut, D., Queisser, G., Wittum, G.: Inertia based filtering of high resolution images using a GPU cluster. Comput. Vis. Sci. 14, 181–186 (2011)

    MathSciNet  Article  Google Scholar 

  18. 18.

    Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9, 62–66 (1979)

    Article  Google Scholar 

  19. 19.

    Oracle. Java native interface. http://docs.oracle.com/javase/7/docs/technotes/guides/jni/index.html 1993 (2014)

  20. 20.

    Vogel, A., Xu, J., Wittum, G.: A generalization of the vertex-centered finite volume scheme to arbitrary high order. Comput. Vis. Sci. 13(5), 221–228 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  21. 21.

    Bey, J.: Finite-Volumen- und Mehrgitter-Verfahren für Elliptische Randwertprobleme. B.G. Teubn, Berlin (2003)

    Google Scholar 

  22. 22.

    Hackbusch, W.: Iterative Solution of Large Sparse Systems of Equations. Springer, Berlin (1993)

    Google Scholar 

  23. 23.

    ter Romeny, B.M.H.: Computational Imaging and Vision Geometry-Driven Diffusion in Computer Vision. Kluwer Academic Publishers, Dordrecht (1994)

    Google Scholar 

  24. 24.

    Gonzales, R. C., Woods, R. E.: Digital image processing. Pearson Educ; 00004. ISBN-10: 0133356728 ISBN-13: 978-0133356724 (2017)

  25. 25.

    Suetens, P.: Fundamentals of medical imaging. Cambridge University Press, Cambridge. 2nd edn ASIN: B00DT608R2 (2009)

  26. 26.

    Dubois, E.: A projection method to generate anaglyph stereo images. In: Proceedings of IEEE international conference on acoustics speech signal processing, (Salt Lake City, UT), vol. 3, pp. 1661–1664 (2001)

  27. 27.

    Stefan Vilsmeier. Brainlab munich, germany. https://www.brainlab.com/en (1989)

Download references

Acknowledgements

We thank M. Aumüller, HLRS Stutttgart, for excellent and friendly support for the CAVE system installation and upgrade, and J. Pieper, GCSC, and A. Chizhov, Ioffe Physical-Technical Institute, RAS, St. Petersburg, for their support when performing the photos at the CAVE. MMK thanks Elisa Ficarra (Politecnico di Torino) for a stimulating discussion about the subject. The authors acknowledge the Goethe Universität Frankfurt for general support and computational resources and the University Erlangen-Nuremberg as well as the Politecnico di Torino for general support. This work has been supported in part by the “Fondazione Cassa di Risparmio di Torino” (Italy), through the “La Ricerca dei Talenti” (HR Excellence in Research) programme and in major part by the Institute of Radiology of the University Medical Center Erlangen. The Authors wish to express their sincere thanks to the anonymous Referees for their thorough and critical reviews of our work.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Markus M. Knodel.

Additional information

Communicated by Alfio Grillo.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Knodel, M.M., Lemke, B., Lampe, M. et al. Virtual reality in advanced medical immersive imaging: a workflow for introducing virtual reality as a supporting tool in medical imaging. Comput. Visual Sci. 18, 203–212 (2018). https://doi.org/10.1007/s00791-018-0292-3

Download citation

Keywords

  • Virtual reality
  • Imaging
  • Cave automatic virtual environment
  • Postprocessing
  • Volume rendering
  • Surface reconstruction
  • Surface rendering