Advertisement

Multiple reconstruction and dynamic modeling of 3D digital objects using a morphing approach

Application to kidney animation and tumor tracking

Abstract

Organ segmentation and motion simulation of organs can be useful for many clinical purposes such as organ study, diagnostic aid, therapy planning or even tumor destruction. In this paper we present a full workflow starting from a CT-Scan resulting in kidney motion simulation and tumor tracking. Our method is divided into three major steps: kidney segmentation, surface reconstruction and animation. The segmentation is based on a semi-automatic region-growing approach that is refined to improve its results. The reconstruction is performed using the Poisson surface reconstruction and gives a manifold three-dimensional (3D) model of the kidney. Finally, the animation is accomplished using an automatic mesh morphing among the models previously obtained. Thus, the results are purely geometric because they are 3D animated models. Moreover, our method requires only a basic user interaction and is fast enough to be used in a medical environment, which satisfies our constraints. Finally, this method can be easily adapted to magnetic resonance imaging acquisition because only the segmentation part would require minor modifications.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

References

  1. 1.

    Pavlovich, C., Walther, M., Choyke, P., Pautler, S., Chang, R., Linehan, M., Wood, B.: Percutaneous radio frequency ablation of small renal tumors: initial results. J. Urol. 167, 10–15 (2002)

  2. 2.

    Kent, J., Carlson, W., Parent, R.: Shape transformation for polyhedral objects. Comput. Graph. 26(2) (1992)

  3. 3.

    Leonardi, V., Mari, J.L., Vidal, V., Daniel, M.: A morphing approach for kidney dynamic modeling—from 3d reconstruction to motion simulation. In: 20th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, WSCG 2012, pp. 179–187 (2012)

  4. 4.

    Ahmed, M., Yamany, S., Mohamed, N., Farag, A., Moriarty, T.: A modified fuzzy C-means algorithm for bias field estimation and segmentation of MRI data. IEEE Trans. Med. Imaging 21(3) (2002)

  5. 5.

    Pham, D., Prince, J.: Adaptative fuzzy segmentation of magnetic resonance images. IEEE Trans. Med. Imaging (1998)

  6. 6.

    Kaus, M., Warfield, S., Nabavi, A., Black, P., Jolesz, F., Kikinis, R.: Automated segmentation of MR images of brain tumors. Radiology 218, 586–591 (2001)

  7. 7.

    Lei, T., Sewchand, W.: A new stochastic model based image segmentation tech for CT images. IEEE Trans. Med. Imaging 11(1) (1992)

  8. 8.

    Liang, Z., MacFall, J., Harrington, D.: Parameter estimation and tissue segmentation from multispectral MR images. IEEE Trans. Med. Imaging 13(3) (1994)

  9. 9.

    Wells, W., Grimson, W., Kikinis, R., Jolesz, F.A.: Adaptive segmentation of MRI data. IEEE Trans. Med. Imaging 15, 429–442 (1996)

  10. 10.

    Rajapakse, J., Giedd, J., Rapoport, J.: Statistical approach to segmentation of single-channel cerebral MR images. IEEE Trans. Med. Imaging 16(2) (2001)

  11. 11.

    Kapur, T., Grimson, E., Kikinis, R., Wells, W.: Enhanced spatial priors for segmentation of magnetic resonance imagery. Med. Image Comput. Comput. Assist. Interv (1998)

  12. 12.

    Held, K., Kops, E.R., Krause, B., III, W.W., Kikinis, R., Müller-Gärtner, H.W.: Markov random field segmentation of brain MR images. IEEE Trans. Med. Imaging (1997)

  13. 13.

    Zhang, Y., Brady, M., Smith, S.: Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Trans. Med. Imaging 20(1) (2001)

  14. 14.

    Lee, C.C., Chung, P.C., Tsai, H.M.: Identifying multiple abdominal organs from CT image series using a multimodule contextual neural network and spatial fuzzy rules. IEEE Trans. Inf. Technol. Biomed. 7(3), 208–217 (2003)

  15. 15.

    Lin, J.S., Cheng, K.S., Mao, C.W.: Multispectral magnetic resonance images segmentation using fuzzy Hopfield neural network. Int. J. Biomed. Comput. 42(3), 205–214 (1996)

  16. 16.

    Vilariño, D., Cabello, D., Balsi, M., Brea, V.: Image segmentation based on active contours using discrete time cellular neural networks. J. VLSI Signal Process. Syst. 23, 403–414 (1999)

  17. 17.

    Atkins, S., Mackiewich, B.: Fully automatic segmentation of the brain in MRI. IEEE Trans. Med. Imaging 17(1), 98–107 (1998)

  18. 18.

    Bardinet, E., Cohen, L., Ayache, N.: A parametric deformable model to fit unstructured 3D data. Comput. Vis. Image Underst. 71, 39–54 (1995)

  19. 19.

    Boscolo, R., Brown, M., McNitt-Gray, M.: Medical image segmentation with knowledge-guided robust active contours. Radio Graph. 22, 437–448 (2002)

  20. 20.

    Gao, J., Kosaka, A., Kak, A.: A deformable model for human organ extraction. Int. Conf. Image Process. 3 (1998)

  21. 21.

    Boes, J., Weymouth, T., Meyer, C.: Multiple organ definition in CT using a Bayesian approach for 3D model fitting. Vis. Geom. 2573, 244–251 (1995)

  22. 22.

    Fritsch, D., Pizer, S., Yu, L., Johnson, V., Chaney, E.: Localization and segmentation of medical image objects using deformable shape loci. Inf. Process. Med. Imaging 1230, 127–140 (1997)

  23. 23.

    Ginneken, B., Frangi, A., Staal, J., ter, Haar Romeny, B., Viergever, M.: Active shape model segmentation with optimal features. IEEE Trans. Med. Imaging 21(8) (2002)

  24. 24.

    Lin, D.T., Lei, C.C., Hung, S.W.: Computer-aided kidney segmentation on abdominal CT images. IEEE Trans. Inf. Technol. Biomed. 10(1) (2006)

  25. 25.

    Wu, J., Poehlman, S., Noseworthy, M., Kamath, M.: Texture feature based automated seeded region growing in abdominal MRI segmentation. J. Biomed. Sci. Eng. 2, 1–8 (2009)

  26. 26.

    Rusko, L., Bekes, G., Fidrich, M.: Automatic segmentation of the liver from multi- and single-phase contrast-enhanced CT images. Med. Image Anal. 13, 871–882 (2009)

  27. 27.

    Fuchs, H., Kedem, Z.M., Uselton, S.P.: Implicit surfaces for semi-automatic medical organs reconstruction. Commun. ACM 20(10) (1977)

  28. 28.

    Ju, T., Warren, J., Carson, J., Eichele, G., Thaller, C., Chiu, W., Bello, M., Kakadiaris, I.: Building 3D surface networks from 2D curve networks with application to anatomical modeling. Proc. Pac. Graph. 21, 764–773 (2005)

  29. 29.

    Ekoule, A.B., Peyrin, F.C., Odet, C.L.: A triangulation algorithm from arbitrary shaped multiple planar contours. ACM Trans. Graph. 10(2), 182–199 (1991)

  30. 30.

    Klein, R., Schilling, A., Strasser, W.: Reconstruction and simplification of surfaces from contours. In: Proceedings of the Seventh Pacific Conference on Computer Graphics and Applications, pp. 198–207 (1999)

  31. 31.

    Wrazidlo, W., Brams, H., Lederer, W., Schneider, S., Geiger, B., Fischer, C.: An alternative method of three-dimensional reconstruction from two-dimensional CT and MR data sets. Eur. J. Radiol. 12, 11–16 (1991)

  32. 32.

    Barequet, G., Shapiro, D., Tal, A.: Multilevel sensitive reconstruction of polyhedral surfaces from parallel slices. Vis. Comput. 16(2), 116–133 (2000)

  33. 33.

    Boissonnat, J.D.: Shape reconstruction from planar cross-sections. Comput. Vis. Graph. Image Process. 44, 1–29 (1986)

  34. 34.

    Tsingos, N., Bittar, E., Gascuel, M.P.: Implicit surfaces for semi-automatic medical organs reconstruction. Comput. Graph. Int., pp. 3–15 (1995)

  35. 35.

    Pihuit, A., Palombi, O., Cani, M.P.: Reconstruction implicite de surfaces 3D à partir de régions 2D dans des plans parallèles. 22e Journées de l’Association Francophone d’Informatique Graphique (2009)

  36. 36.

    Amrani, M., Crespin, B., Shariat, B.: Skeletal implicit surface reconstruction and particle system for organs interactions simulation. In: Geometric Modeling: Techniques, Applications, Systems and Tools. Springer, Netherlands. pp. 171–192 (2004)

  37. 37.

    Galin, E., Akkouche, S.: Fast surface reconstruction from contours using implicit surfaces. In: Implicit Surfaces’98 Conference, vol. 3, pp. 139–144 (1998)

  38. 38.

    Xu, S.B., Lu, W.X.: Surface reconstruction of 3D objects in computerized tomography. Comput. Vis. Graph. Image Process. 44, 270–278 (1988)

  39. 39.

    Cotin, S., Delingette, H., Ayache, N.: Real-time elastic deformations of soft tissues for surgery simulation. IEEE Trans. Vis. Comput Graphics. 5(1), 62–73 (1999)

  40. 40.

    Delingette, H.: Simplex meshes: a general representation for 3D shape reconstruction. In: Rapport de Recherche INRIA, vol. 2214 (1994)

  41. 41.

    Mari, J.L.: Approaches dedicated to the modeling of complex shapes–application to medical data. J. Med. Inf. Technol. 7, 3–15 (2004)

  42. 42.

    Lorensen, W.E., Cline, H.E.: Marching cubes: a high resolution 3D surface reconstruction algorithm. SIGGRAPH (1987)

  43. 43.

    Akkouche, S., Galin, E.: Adaptive implicit surface polygonization using marching triangles. Comput. Graph. Forum 20, 67–80 (2001)

  44. 44.

    Hartmann, E.: A marching method for the triangulation of surfaces. Vis. Comput. 14, 95–108 (1998)

  45. 45.

    Villard, P.F.: Simulation du mouvement pulmonaire pour un traitement oncologique. Ph.D. Thesis at University Claude-Bernard Lyon 1, France, p.152 (2006)

  46. 46.

    Lim, S., Haron, H.: Surface reconstruction techniques: a review. Artif. Intell. Rev. 42(1), 59–78 (2014). doi:10.1007/s10462-012-9329-z

  47. 47.

    Nakamoto, M., Ukimura, O., Gill, I., Mahadevan, A., Miki, T., Hashizume, M., Sato, Y.: Realtime organ tracking for endoscopic augmented reality visualization using miniature wireless magnetic tracker. Med. Imaging Augment. Real., pp. 359–366 (2008)

  48. 48.

    Nicolau, S., Pennec, X., Soler, L., Ayache, N.: Clinical evaluation of a respiratory gated guidance system for liver punctures. Med. Image Comput. Comput. Assist. Interv. 4792, 77–85 (2007)

  49. 49.

    Olbricha, B., Trau, J., Wiesner, S., Wicherta, A., Feussner, H., Navab, N.: Respiratory motion analysis: towards gated augmentation of the liver. Comput. Assist. Radiol. Surg. 1281, 248–253 (2005)

  50. 50.

    Schweikard, A., Glosser, G., Bodduluri, M., Murphy, M., Adler, J.: Robotic motion compensation for respiratory movement during radiosurgery. J. Comput. Aided Surg (2000)

  51. 51.

    Shirato, H., Shimizu, S., Kitamura, K., Nishioka, T., Kagei, K., Hashimoto, S., Aoyama, H., Kunieda, T., Shinohara, N., Dosaka-Akita, H., Miyasaka, K.: Four dimensional treatment planning and fluoroscopic real-time tumor tracking radiotherapy for moving tumor. Int. J. Radiat. Oncol. Biol. Phys. 48, 435–442 (2000)

  52. 52.

    Murphy, M., Chang, S., Gibbs, I., Le, Q.T., Hai, J., Kim, D., Martin, D., Adler, J.: Patterns of patient movement during frameless image-guided radiosurgery. Int. J. Radiat. Oncol. Biol. Phys. 55(5), 1400–1408 (2003)

  53. 53.

    Sarrut, D., Boldea, V., Miguet, S., Ginestet, C.: Simulation of four-dimensional CT images from deformable registration between inhale and exhale breath-hold CT scans. Med. Phys. 33(3) (2006)

  54. 54.

    Rohlfing, T., Maurer, C., O’Dell, W., Zhong, J.: Modeling liver motion and deformation during the respiratory cycle using intensity-based free-form registration of gated MR images. In: Proceedings of SPIE, Medical Imaging 2001: Visualization, Image-Guided, vol. 4319, pp. 337–348 (2001)

  55. 55.

    Hostettler, A., Nicolau, S., Soler, L., Rémond, Y., Marescaux, J.: A real-time predictive simulation of abdominal organ positions induced by free breathing. In: International Symposium on Biomedical Simulation, pp. 89–97 (2008)

  56. 56.

    Reyes, M., Malandain, G., Koulibaly, P.M., Ballester, M.G., Darcourt, J.: Respiratory motion correction in emission tomography image reconstruction. In: Medical Image Computing and Computer-Assisted Intervention, pp. 396–376 (2005)

  57. 57.

    Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L.G., Leach, M.O., Hawkes, D.J.: Nonrigid registration using free-form deformations: application to breast MR images. IEEE Trans. Med. Imaging 18(8) (1999)

  58. 58.

    Bardinet, E., Cohen, L., Ayache, N.: Tracking and motion analysis of the left ventricle with deformable superquadrics. Med. Image Anal. 1(2), 129–149 (1996)

  59. 59.

    Noe, K.O., de Senneville, B.D., Elstrom, U.V., Tanderup, K., Sorensen, T.S.: Acceleration and validation of optical flow based deformable registration for image-guided radiotherapy. Acta Oncol. 47(7), 1286–1293 (2008)

  60. 60.

    Arnold, P., Preiswerk, F., Fasel, B., Salomir, R., Scheffler, K., Cattin, P.: 3d organ motion prediction for mr-guided high intensity focused ultrasound. MICCAI 6892, 623–630 (2011)

  61. 61.

    Preiswerk, F., Arnold, P., Fasel, B., Cattin, P.: A Bayesian framework for estimating respiratory liver motion from sparse measurements. Abdominal Imaging. Computational and Clinical Applications 7029, 207–214 (2012)

  62. 62.

    Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Eurographics Symposium on Geometry Processing (2006)

  63. 63.

    Alexa, M., Cohen-Or, D., Levin, D.: As-rigid-as-possible shape interpolation. In: Proceedings of Computer Graphics and Interactive Techniques (2000)

  64. 64.

    Gregory, A., State, A., Lin, M., Manocha, D., Livingston, M.: Feature-based surface decomposition for correspondence and morphing between polyhedra. In: Proceedings of Computer, Animation, pp. 64–71 (1998)

  65. 65.

    Kanai, T., Suzuki, H., Kimura, F.: Metamorphosis of arbitrary triangular meshes. Proc. Comput. Graph. Appl. 20(2) (2000)

  66. 66.

    Lee, A., Dobkin, D., Sweldens, W., Schroder, P.: Multiresolution mesh morphing. In: Proceedings of Computer Graphics and Interactive Techniques, pp. 343–350 (1999)

  67. 67.

    Lee, A., Sweldens, W., Schroder, P., Cowsar, L., Dobkin, D.: Maps: Multiresolution adaptive parameterization of surfaces. In: Proceedings of SIGGRAPH, pp. 95–104 (1998)

  68. 68.

    Yan, H.B., Hu, S.M., Martin, R.: 3d morphing using strain field interpolation. Comput. Sci. Technol. 1 (2007)

  69. 69.

    Polthier, K.: Conjugate harmonic maps and minimal surfaces. Technical report, Technische University of Berlin (2000)

  70. 70.

    Yao, C.L., Trinh, T., Wong, G.T.C., Irwin, M.G.: Anaesthesia for high intensity focused ultrasound (HIFU) therapy. J. Assoc. Anaesth. G. B. Irel. 63, 865–872 (2008)

  71. 71.

    Cresson, T., Chav, R., Branchaud, D., Humbert, L., Godbout, B., Aubert, B., Skalli, W., De Guise, J.: Coupling 2d/3d registration method and statistical model to perform 3d reconstruction from partial x-rays images data. In: Annual International Conference of the IEEE in Engineering in Medicine and Biology Society, pp. 1008–1011 (2009)

  72. 72.

    Cresson, T., Branchaud, D., Chav, R., Godbout, B., de Guise, J.A.: 3d shape reconstruction of bone from two x-ray images using 2d/3d non-rigid registration based on moving least-squares deformation. In: Proceedings of the SPIE, Medical Imaging 2010: Image Processing, vol. 7623 (2010). doi:10.1117/12.844098

Download references

Acknowledgments

This work was granted by the group Novartis and by the Foundation “Santé, Sport et Développement Durable”, presided over Pr. Yvon Berland. The authors would like to thank the persons involved in the Kidney Tumor Tracking (KiTT) project: Christian Coulange for his precious help, Marc André, Frédéric Cohen Philippe Souteyrand and Julien Frandon for their wise advice and for providing CT scan data, and Pierre-Henri Rolland for his support.

Author information

Correspondence to Valentin Leonardi.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mpg 3270 KB)

Supplementary material 2 (mpg 1722 KB)

Supplementary material 1 (mpg 3270 KB)

Supplementary material 2 (mpg 1722 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Leonardi, V., Vidal, V., Daniel, M. et al. Multiple reconstruction and dynamic modeling of 3D digital objects using a morphing approach. Vis Comput 31, 557–574 (2015). https://doi.org/10.1007/s00371-014-0978-6

Download citation

Keywords

  • Geometric modeling
  • Surface reconstruction
  • Dynamic modeling
  • Mesh morphing