Advertisement

Image Reconstruction in a Manifold of Image Patches: Application to Whole-Fetus Ultrasound Imaging

  • Alberto GomezEmail author
  • Veronika Zimmer
  • Nicolas Toussaint
  • Robert Wright
  • James R. Clough
  • Bishesh Khanal
  • Milou P. M. van Poppel
  • Emily Skelton
  • Jackie Matthews
  • Julia A. Schnabel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11905)

Abstract

We propose an image reconstruction framework to combine a large number of overlapping image patches into a fused reconstruction of the object of interest, that is robust to inconsistencies between patches (e.g. motion artefacts) without explicitly modelling them. This is achieved through two mechanisms: first, manifold embedding, where patches are distributed on a manifold with similar patches (where similarity is defined only in the region where they overlap) closer to each other. As a result, inconsistent patches are set far apart in the manifold. Second, fusion, where a sample in the manifold is mapped back to image space, combining features from all patches in the region of the sample.

For the manifold embedding mechanism, a new method based on a Convolutional Variational Autoencoder (\(\beta \)-VAE) is proposed, and compared to classical manifold embedding techniques: linear (Multi Dimensional Scaling) and non-linear (Laplacian Eigenmaps). Experiments using synthetic data and on real fetal ultrasound images yield fused images of the whole fetus where, in average, \(\beta \)-VAE outperforms all the other methods in terms of preservation of patch information and overall image quality.

References

  1. 1.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)CrossRefGoogle Scholar
  2. 2.
    Cerrolaza, J.J., et al.: 3D fetal skull reconstruction from 2DUS via deep conditional generative networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 383–391. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00928-1_44CrossRefGoogle Scholar
  3. 3.
    Che, C., Mathai, T.S., Galeotti, J.: Ultrasound registration: a review. Methods 115, 128–143 (2017)CrossRefGoogle Scholar
  4. 4.
    Gomez, A., Bhatia, K., Tharin, S., Housden, J., Toussaint, N., Schnabel, J.A.: Fast registration of 3D fetal ultrasound images using learned corresponding salient points. In: Cardoso, M.J., et al. (eds.) FIFI/OMIA -2017. LNCS, vol. 10554, pp. 33–41. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-67561-9_4CrossRefGoogle Scholar
  5. 5.
    Heinrich, M.P., Jenkinson, M., Papież, B.W., Brady, S.M., Schnabel, J.A.: Towards realtime multimodal fusion for image-guided interventions using self-similarities. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8149, pp. 187–194. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-40811-3_24CrossRefGoogle Scholar
  6. 6.
    Higgins, I., et al.: \(\beta \)-VAE: learning basic visual concepts with a constrained variational framework. In: ICLR, pp. 1–22 (2017)Google Scholar
  7. 7.
    James, A.P., Dasarathy, B.V.: Medical image fusion: a survey of the state of the art. Inf. Fusion 19(1), 4–19 (2014)CrossRefGoogle Scholar
  8. 8.
    Kavitha, C.T., Chellamuthu, C.: Multimodal medical image fusion based on integer wavelet transform and neuro-fuzzy. In: IEEE ICSIP, pp. 296–300 (2010)Google Scholar
  9. 9.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)Google Scholar
  10. 10.
    Kingma, D.P., Welling, M.: Auto-Encoding Variational Bayes. In: ICLR, December 2014Google Scholar
  11. 11.
    Mead, A.: Review of the development of multidimensional scaling methods. Statistician 41(1), 27 (1992)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Rivaz, H., Chen, S.J.-S., Collins, D.L.: Automatic deformable MR-ultrasound registration for image-guided neurosurgery. IEEE TMI 34(2), 366–380 (2015)Google Scholar
  13. 13.
    Shepard, D.: A two-dimensional interpolation function for irregularly-spaced data. In: Proceedings of ACM, pp. 517–524 (1968)Google Scholar
  14. 14.
    Wachinger, C., Wein, W., Navab, N.: Three-dimensional ultrasound mosaicing. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007. LNCS, vol. 4792, pp. 327–335. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-75759-7_40CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Alberto Gomez
    • 1
    Email author
  • Veronika Zimmer
    • 1
  • Nicolas Toussaint
    • 1
  • Robert Wright
    • 1
  • James R. Clough
    • 1
  • Bishesh Khanal
    • 1
    • 2
  • Milou P. M. van Poppel
    • 1
  • Emily Skelton
    • 1
  • Jackie Matthews
    • 1
  • Julia A. Schnabel
    • 1
  1. 1.Department of Biomedical EngineeringKing’s College LondonLondonUK
  2. 2.NAAMIIKathmanduNepal

Personalised recommendations