Illustrative visualization of 3D planning models for augmented reality in liver surgery

  • Christian HansenEmail author
  • Jan Wieferich
  • Felix Ritter
  • Christian Rieder
  • Heinz-Otto Peitgen
Original Article



Augmented reality (AR) obtains increasing acceptance in the operating room. However, a meaningful augmentation of the surgical view with a 3D visualization of planning data which allows reliable comparisons of distances and spatial relations is still an open request.


We introduce methods for intraoperative visualization of 3D planning models which extend illustrative rendering and AR techniques. We aim to reduce visual complexity of 3D planning models and accentuate spatial relations between relevant objects. The main contribution of our work is an advanced silhouette algorithm for 3D planning models (distance-encoding silhouettes) combined with procedural textures (distance-encoding surfaces). In addition, we present a method for illustrative visualization of resection surfaces.


The developed algorithms have been embedded into a clinical prototype that has been evaluated in the operating room. To verify the expressiveness of our illustration methods, we performed a user study under controlled conditions. The study revealed a clear advantage in distance assessment with the proposed illustrative approach in comparison to classical rendering techniques.


The presented illustration methods are beneficial for distance assessment in surgical AR. To increase the safety of interventions with the proposed approach, the reduction of inaccuracies in tracking and registration is a subject of our current research.


Intraoperative visualization Augmented reality Image-guided surgery Illustrative rendering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Schenk A, Zidowitz S, Bourquain H, Hindennach M, Hansen C, Hahn H, Peitgen HO (2008) Clinical relevance of model based computer-assisted diagnosis and therapy. Proceedings of SPIE Medical Imaging, 6915(1):691502_1-19. doi: 10.1117/12.780270
  2. 2.
    Oldhafer KJ, Stavrou GA, Prause G, Peitgen HO, Lueth TC, Weber S (2009) How to operate a liver tumor you cannot see. Langenbecks Arch Surg 394(3): 489–494. doi: 10.1007/s00423-009-0469-9 CrossRefPubMedGoogle Scholar
  3. 3.
    Hildebrand P, Schlichting S, Martens V, Besiveric A, Roblick U, Roblick U, Mirow L, Buerk C, Schweikard A, Bruch H (2008) Prototype of an intraoperative navigation and documentation system for laparoscopic radiofrequency ablations: first experiences. Eur J Surg Oncol 34(4): 418–421. doi: 10.1016/j.ejso.2007.04.017 PubMedGoogle Scholar
  4. 4.
    Beller S, Eulenstein S, Lange T, Hünerbein M, Schlag PM (2009) Upgrade of an optical navigation system with a permanent electromagnetic position control: a first step towards “navigated control” for liver surgery. J Hepatobiliary Pancreat Surg 16(2): 165–170. doi: 10.1007/s00534-008-0040-z CrossRefPubMedGoogle Scholar
  5. 5.
    Cash DM, Miga MI, Glasgow SC, Dawant BM, Clements LW, Cao Z, Galloway RL, Chapman WC (2007) Concepts and preliminary data toward the realization of image-guided liver surgery. J Gastrointest Surg 11(7): 844–859. doi: 10.1007/s11605-007-0090-6 CrossRefPubMedGoogle Scholar
  6. 6.
    Ritter F, Hansen C, Dicken V, Konrad O, Preim B, Peitgen HO (2006) Real-time illustration of vascular structures. IEEE Trans Vis Comput Graph J 12(5): 877–884. doi: 10.1109/TVCG.2006.172 CrossRefGoogle Scholar
  7. 7.
    Sielhorst T, Feuerstein M, Navab N (2008) Advanced Medical Displays: A literature review of augmented reality. IEEE/OSA J Disp Technol; Special Issue on Medical Displays 4(4): 451–467. doi: 10.1109/JDT.2008.2001575 Google Scholar
  8. 8.
    Ayache N (2003) Epidaure: a research project in medical image analysis, simulation and robotics at INRIA. IEEE Trans Med Imaging 22(10): 1185–1201. doi: 10.1109/TMI.2003.812863 CrossRefPubMedGoogle Scholar
  9. 9.
    Samset E, Schmalstieg D, Vander SJ, Freudenthal A, Declerck J, Casciaro S, Rideng Ø, Gersak B (2008) Augmented reality in surgical procedures. Proc SPIE Med Imaging 6806(1):68060K_1-12. doi: 10.1117/12.784155 Google Scholar
  10. 10.
    Nicolau SA, Pennec X, Soler L, Buy X, Gangi A, Ayache N, Marescaux J (2009) An augmented reality system for liver thermal ablation: design and evaluation on clinical cases. Med Image Anal 13(3): 494–506. doi: 10.1016/ CrossRefPubMedGoogle Scholar
  11. 11.
    Feuerstein M, Mussack T, Heining SM, Navab N (2008) Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 27(1): 355–369. doi: 10.1109/TMI.2007.907327 CrossRefPubMedGoogle Scholar
  12. 12.
    Scheuering M, Schneider A, Schenk A, Preim B, Greiner G (2003) Intraoperative augmented reality for minimally invasive liver interventions. Proc SPIE Med Imaging 5029(1): 407–417. doi: 10.1117/12.480212 Google Scholar
  13. 13.
    Marescaux J, Rubino F, Arenas M, Mutter D, Soler L (2004) Augmented-reality-assisted laparoscopic adrenalectomy. J Am Med Assoc 292(18): 2214–2215. doi: 10.1001/jama.292.18.2214-c CrossRefGoogle Scholar
  14. 14.
    Krempien R, Hoppe H, Kahrs L, Daeuber S, Schorr O, Eggers G, Bischof M, Munter MW, Debus J, Harms W (2008) Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3D interstitial brachytherapy. Int J Radiat Oncol Biol Phys 70(3): 944–952. doi: 10.1016/j.ijrobp.2007.10.048 PubMedGoogle Scholar
  15. 15.
    Riechmann M, Kahrs LA, Hoppe H, Ulmer C, Raczkowsky J, Lamade W, Wörn H (2006) Visualisierungskonzept für die projektorbasierte Erweiterte Realität in der Leberchirurgie. Proc BMT 209(1): 1–2Google Scholar
  16. 16.
    Glossop ND, Wang Z (2003) Laser projection augmented reality system for computer-assisted surgery. Int Congr Ser 1256(1): 65–71. doi: 10.1016/S0531-5131(03)00515-6 CrossRefGoogle Scholar
  17. 17.
    Navab N, Feuerstein M, Bichlmeier C (2007) Laparoscopic virtual mirror—new interaction paradigm for monitor based augmented reality. Virtual Reality Conference IEEE, pp 43–50. doi: 10.1109/VR.2007.352462
  18. 18.
    Lerotic M, Chung AJ, Mylonas GP, Yang GZ (2007) Pq-space based non-photorealistic rendering for augmented reality. Proc MICCAI 4792: 102–109. doi: 10.1007/978-3-540-75759-7 Google Scholar
  19. 19.
    Bichlmeier C, Wimmer F, Heining SM, Navab N (2007) Contextual anatomic mimesis: hybrid in situ visualization method for improving multi-sensory depth perception in medical augmented reality. In: ISMAR ‘07: Proceedings of the 2007 6th IEEE and ACM international symposium on mixed and augmented reality, pp 1–10. doi: 10.1109/ISMAR.2007.4538837
  20. 20.
    Strothotte T, Schlechtweg S (2002) Non-photorealistic computer graphics. Morgan Kaufmann, San FranciscoGoogle Scholar
  21. 21.
    Bruckner S (2008) Interactive illustrative volume visualization. PhD thesis, University of Technology, Vienna, AustriaGoogle Scholar
  22. 22.
    Fischer J, Bartz D (2005) Stylized augmented reality for improved immersion. Proceedings of the IEEE Conference on Virtual Reality, pp 195–202. doi: 10.1109/VR.2005.1492774
  23. 23.
    Freudenberg B (2004) Real-time stroke-based halftoning. PhD thesis, Otto-von-Guericke University Magdeburg, GermanyGoogle Scholar
  24. 24.
    Isenberg T, Halper N, Strothotte T (2002) Stylizing silhouettes at interactive rates: from silhouette edges to silhouette strokes. Computer Graphics Forum. 21(3): 249–258. doi: 10.1111/1467-8659.00584 CrossRefGoogle Scholar
  25. 25.
    Ericsson K, Simon H (1993) Protocol Analysis: Verbal Reports as Data. MIT Press, BostonGoogle Scholar
  26. 26.
    Blum J, Padoy N, Feußner H, Navab N (2008) Workflow mining for visualization and analysis of surgeries. Int J Comput Assist Radiol Surg 3(5): 379–386. doi: 10.1007/s11548-008-0239-0 CrossRefGoogle Scholar

Copyright information

© CARS 2009

Authors and Affiliations

  • Christian Hansen
    • 1
    Email author
  • Jan Wieferich
    • 1
  • Felix Ritter
    • 1
  • Christian Rieder
    • 1
  • Heinz-Otto Peitgen
    • 1
  1. 1.Fraunhofer MEVIS, Insitute for Medical Image ComputingBremenGermany

Personalised recommendations