Skip to main content
SpringerLink
Log in

Handling topological changes during elastic registration

Application to augmented reality in laparoscopic surgery

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Locating the internal structures of an organ is a critical aspect of many surgical procedures. Minimally invasive surgery, associated with augmented reality techniques, offers the potential to visualize inner structures, allowing for improved analysis, depth perception or for supporting planning and decision systems.

Methods

Most of the current methods dealing with rigid or non-rigid augmented reality make the assumption that the topology of the organ is not modified. As surgery relies essentially on cutting and dissection of anatomical structures, such methods are limited to the early stages of the surgery.

We solve this shortcoming with the introduction of a method for physics-based elastic registration using a single view from a monocular camera. Singularities caused by topological changes are detected and propagated to the preoperative model. This significantly improves the coherence between the actual laparoscopic view and the model and provides added value in terms of navigation and decision-making, e.g., by overlaying the internal structures of an organ on the laparoscopic view.

Results

Our real-time augmentation method is assessed on several scenarios, using synthetic objects and real organs. In all cases, the impact of our approach is demonstrated, both qualitatively and quantitatively (http://www.open-cas.org/?q=PaulusIJCARS16).

Conclusion

The presented approach tackles the challenge of localizing internal structures throughout a complete surgical procedure, even after surgical cuts. This information is crucial for surgeons to improve the outcome for their surgical procedure and avoid complications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://www.open-cas.org/?q=PaulusIJCARS16.

References

  1. Agudo A, Calvo B, Montiel J (2012) Finite element based sequential bayesian non-rigid structure from motion. In: Computer vision and pattern recognition (CVPR), pp 1418–1425. IEEE

  2. Bartoli A, Zisserman A (2004) Direct estimation of non-rigid registrations. In: British machine vision conference, pp 899–908. BMVA

  3. Bielser D, Glardon P, Teschner M, Gross M (2004) A state machine for real-time cutting of tetrahedral meshes. Graph Models 66(6):398–417

    Article  Google Scholar 

  4. Diana M, Halvax P, Mertz D, Legner A, Brulé J-M, Robinet E, Mutter D, Pessaux P, Marescaux J (2015) Improving echo-guided procedures using an ultrasound-CT image fusion system. Surg Innov 22(3):217–222

    Article  PubMed  Google Scholar 

  5. Feldmar J, Ayache N, Betting F (1995) 3D–2D projective registration of free-form curves and surfaces. In: Computer vision, pp. 549–556. IEEE

  6. Ferrant M, Nabavi A, Macq B, Black PM, Jolesz FA, Kikinis R, Warfield SK (2002) Serial registration of intraoperative mr images of the brain. Med Image Anal 6(4):337–359

    Article  PubMed  Google Scholar 

  7. Haouchine N, Dequidt J, Berger M-O, Cotin S (2014) Single view augmentation of 3D elastic objects. In: Mixed and augmented reality (ISMAR), pp 229–236

  8. Haouchine N, Dequidt J, Kerrien E, Berger M-O, Cotin S (2012) Physics-based augmented reality for 3D deformable object. VRIPHYS Virtual Real Interact Phys Simul. Darmstadt, Germany, pp 31–38

    Google Scholar 

  9. Haouchine N, Dequidt J, Peterlik I, Kerrien E, Berger M-O, Cotin S (2013) Image-guided simulation of heterogeneous tissue deformation for augmented reality during hepatic surgery. In: IEEE international symposium on mixed and augmented reality (ISMAR), pp 199–208

  10. Kass M, Witkin A, Terzopoulos D (1988) Snakes: active contour models. Int J Comput Vis 1(4):321–331

    Article  Google Scholar 

  11. Koschier D, Lipponer S, Bender J (2014) Adaptive tetrahedral meshes for brittle fracture simulation. In: Proceedings of the ACM SIGGRAPH/eurographics symposium on computer animation, pp 57–66

  12. Leizea I, Álvarez H, Aguinaga I, Borro D (2014) Real-time deformation, registration and tracking of solids based on physical simulation. In: Mixed and augmented reality (ISMAR), pp 165–170

  13. Malti A, Hartley R, Bartoli A, Kim J-H (2013) Monocular template-based 3D reconstruction of extensible surfaces with local linear elasticity. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1522–1529

  14. Moreno-Noguer F, Salzmann M, Lepetit V, Fua P (2009) Capturing 3D stretchable surfaces from single images in closed form. In: Computer vision and pattern recognition CVPR, pp 1842–1849

  15. Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201

    Article  PubMed  Google Scholar 

  16. Nienhuys H-W, van der Stappen AF (2001) A surgery simulation supporting cuts and finite element deformation. In: International conference on medical image computing and computer-assisted intervention, pp 145–152

  17. Paulus CJ, Haouchine N, Cazier D, Cotin S (2015) Augmented reality during cutting and tearing of deformable objects. In: Mixed and augmented reality (ISMAR), pp 54–59

  18. Paulus CJ, Haouchine N, Cazier D, Cotin S (2015) Surgical augmented reality with topological changes. In: International conference on medical image computing and computer-assisted intervention

  19. Paulus CJ, Untereiner L, Courtecuisse H, Cotin S, Cazier D (2015) Virtual cutting of deformable objects based on efficient topological operations. Vis Comput 31(6–8):831–841

    Article  Google Scholar 

  20. Petit A, Lippiello V, Siciliano B (2015) Real-time tracking of 3D elastic objects with an RGB-D sensor. In: Intelligent robots and systems (IROS), pp 3914–3921

  21. Petit A, Lippiello V, Siciliano B (2015) Tracking fractures of deformable objects in real-time with an RGB-D sensor. In: International conference on 3D vision (3DV), pp 632–639

  22. Pilet J, Lepetit V, Fua P (2008) Fast non-rigid surface detection, registration and realistic augmentation. Int J Comput Vis 76(2):109–122

    Article  Google Scholar 

  23. Pratt P, Stoyanov D, Visentini-Scarzanella M, Yang G-Z (2010) Dynamic guidance for robotic surgery using image-constrained biomechanical models. In: Medical image computing and computer-assisted intervention (MICCAI), pp 77–85

  24. Salzmann M, Fua P (2011) Linear local models for monocular reconstruction of deformable surfaces. Pattern Anal Mach Intell 33(5):931–944

    Article  Google Scholar 

  25. Salzmann M, Pilet J, Ilic S, Fua P (2007) Surface deformation models for nonrigid 3D shape recovery. IEEE Trans Pattern Anal Mach Intell 29(8):1481–1487

    Article  PubMed  Google Scholar 

  26. Sifakis E, Barbic J (2012) Fem simulation of 3D deformable solids: a practitioner’s guide to theory, discretization and model reduction. In: ACM SIGGRAPH 2012 courses, p 20

  27. Suwelack S (2013) Real-time biomechanical modeling for intraoperative soft tissue registration. Karlsruhe Institute of Technology – Scientific Publishing, Karlsruhe

  28. Suwelack S, Röhl S, Bodenstedt S, Reichard D, Dillmann R, dos Santos T, Maier-Hein L, Wagner M, Wünscher J, Kenngott H, Müller BP, Speidel S (2014) Physics-based shape matching for intraoperative image guidance. Med Phys 41(11):111901

    Article  PubMed  Google Scholar 

  29. Wu J, Westermann R, Dick C (2014a). Physically-based simulation of cuts in deformable bodies: a survey. In: Eurographics state-of-the-art report

  30. Wu J, Westermann R, Dick C (2014) Real-time haptic cutting of high resolution soft tissues. Stud Health Technol Inform (Proc Med Meets Virtual Real) 196:469–475

    Google Scholar 

Download references

Acknowledgements

We thank Bruno Marques and Éric Leplat for assistance obtaining the experimental data, Etienne Schmitt for his ongoing support in the simulation of the topological changes and Igor Peterlik for the valuable comments improving this work.

Author information

Authors and Affiliations

  1. Inria, Nancy, France

    Christoph J. Paulus, Nazim Haouchine, David Cazier & Stephane Cotin

  2. IHU Strasbourg, Strasbourg, France

    Seong-Ho Kong & Renato Vianna Soares

  3. Seoul National University Hospital, Seoul, Korea

    Seong-Ho Kong

  4. University of Strasbourg - ICube lab CNRS, Strasbourg, France

    David Cazier

Authors
  1. Christoph J. Paulus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nazim Haouchine
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seong-Ho Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Renato Vianna Soares
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David Cazier
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Stephane Cotin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christoph J. Paulus.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animals participant

All applicable international, national, and/or institutional guidelines for the care and use of animals were followed. This article does not contain any studies with human participants and thus does not contain patient data.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (m4v 155412 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Paulus, C.J., Haouchine, N., Kong, SH. et al. Handling topological changes during elastic registration. Int J CARS 12, 461–470 (2017). https://doi.org/10.1007/s11548-016-1502-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-016-1502-4

Keywords

Search

Navigation