Soft-Body Registration of Pre-operative 3D Models to Intra-operative RGBD Partial Body Scans

  • Richard ModrzejewskiEmail author
  • Toby Collins
  • Adrien Bartoli
  • Alexandre Hostettler
  • Jacques Marescaux
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11073)


We present a novel solution to soft-body registration between a pre-operative 3D patient model and an intra-operative surface mesh of the patient lying on the operating table, acquired using an inexpensive and portable depth (RGBD) camera. The solution has several clinical applications, including skin dose mapping in interventional radiology and intra-operative image guidance. We propose to solve this with a robust non-rigid registration algorithm that handles partial surface data, significant posture modification and patient-table collisions. We investigate several unstudied and important aspects of this registration problem. These are the benefits of heterogeneous versus homogeneous biomechanical models and the benefits of modeling patient/table interaction as collision constraints. We also study how abdominal registration accuracy varies as a function of scan length in the caudal-cranial axis.


  1. 1.
    Johnson, P.B., Borrego, D., Balter, S., Johnson, K., Siragusa, D., Bolch, W.E.: Skin dose mapping for fluoroscopically guided interventions. Med. Phys. 38, 5490–5499 (2011)CrossRefGoogle Scholar
  2. 2.
    Rodas, N.L., Bert, J., Visvikis, D., De Mathelin, M., Padoy, N.: Pose optimization of a C-arm imaging device to reduce intraoperative radiation exposure of staff and patient during interventional procedures. In: ICRA (2017)Google Scholar
  3. 3.
    Mahmoud, N.: On-patient see-through augmented reality based on visual SLAM. Int. J. Comput. Assist. Radiol. Surg. 12, 1–11 (2017)CrossRefGoogle Scholar
  4. 4.
    Macedo, M.C.F., Apolinário Jr., A.L., Souza, A.C.S., Giraldi, G.A.: High-quality on-patient medical data visualization in a markerless augmented reality environment. J. Interact. Syst. 5, 41–52 (2014)Google Scholar
  5. 5.
    Lee, J.D., Huang, C.H., Huang, T.C., Hsieh, H.Y., Lee, S.T.: Medical augmented reality using a markerless registration framework. ESA 39, 5286–5294 (2012)Google Scholar
  6. 6.
    Chen, X.: Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display. Biomed. Inf. 55, 124–131 (2015)CrossRefGoogle Scholar
  7. 7.
    Tully, S.T.: BodySLAM: localization and mapping for surgical guidance (2012)Google Scholar
  8. 8.
    Amberg, B., Romdhani, S., Vetter, T.: Optimal step nonrigid ICP algorithms for surface registration. In: CVPR (2007)Google Scholar
  9. 9.
    Collins, T., et al.: A system for augmented reality guided laparoscopic tumour resection with quantitative ex-vivo user evaluation. In: Peters, T., et al. (eds.) CARE 2016. LNCS, vol. 10170, pp. 114–126. Springer, Cham (2017). Scholar
  10. 10.
    Petit, A., Lippiello, V., Siciliano, B.: Real-time tracking of 3D elastic objects with an RGB-D sensor. In: IROS (2015)Google Scholar
  11. 11.
    Huang, Q.X., Adams, B., Wicke, M., Guibas, L.J.: Non-rigid registration under isometric deformations. In: Eurographics Symposium on Geometry Processing (2008)Google Scholar
  12. 12.
    IRCAD: Visiblepatient.
  13. 13.
    Izadi, S., et al.: KinectFusion. In: ISMAR (2011)Google Scholar
  14. 14.
    Segal, A., Haehnel, D., Thrun, S.: Generalized-ICP. In: RSA (2009)Google Scholar
  15. 15.
    Collins, T., Bartoli, A., Bourdel, N., Canis, M.: Robust, real-time, dense and deformable 3D organ tracking in laparoscopic videos. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9900, pp. 404–412. Springer, Cham (2016). Scholar
  16. 16.
    Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: CVPR (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Richard Modrzejewski
    • 1
    • 2
    Email author
  • Toby Collins
    • 2
  • Adrien Bartoli
    • 1
  • Alexandre Hostettler
    • 2
  • Jacques Marescaux
    • 2
  1. 1.EnCoV, Institut Pascal, UMR 6602, CNRS/UBP/SIGMAClermont-FerrandFrance
  2. 2.IRCAD and IHU-StrasbourgStrasbourgFrance

Personalised recommendations