Isotropic Reconstruction of 3D Fluorescence Microscopy Images Using Convolutional Neural Networks

  • Martin Weigert
  • Loic Royer
  • Florian Jug
  • Gene Myers
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10434)

Abstract

Fluorescence microscopy images usually show severe anisotropy in axial versus lateral resolution. This hampers downstream processing, i.e. the automatic extraction of quantitative biological data. While deconvolution methods and other techniques to address this problem exist, they are either time consuming to apply or limited in their ability to remove anisotropy. We propose a method to recover isotropic resolution from readily acquired anisotropic data. We achieve this using a convolutional neural network that is trained end-to-end from the same anisotropic body of data we later apply the network to. The network effectively learns to restore the full isotropic resolution by restoring the image under a trained, sample specific image prior. We apply our method to 3 synthetic and 3 real datasets and show that our results improve on results from deconvolution and state-of-the-art super-resolution techniques. Finally, we demonstrate that a standard 3D segmentation pipeline performs on the output of our network with comparable accuracy as on the full isotropic data.

Notes

Acknowledgments

We thank V. Stamataki, C. Schmied (Tomancak lab), S. Merret and S. Janosch (Sarov Group), H.A. Morales-Navarrete (Zerial lab) for providing the datasets, and U. Schmidt (all MPI-CBG) for helpful feedback. Datasets were recorded by the Light Microscopy Facility (LMF) of MPI-CBG.

References

  1. 1.
    Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Cham (2014). doi: 10.1007/978-3-319-10593-2_13 Google Scholar
  2. 2.
    Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)CrossRefGoogle Scholar
  3. 3.
    Economo, M.N., Clack, N.G., Lavis, L.D., Gerfen, C.R., Svoboda, K., Myers, E.W., Chandrashekar, J.: A platform for brain-wide imaging and reconstruction of individual neurons. Elife 5, e10566 (2016)CrossRefGoogle Scholar
  4. 4.
    Huisken, J., Swoger, J., Del Bene, F., Wittbrodt, J., Stelzer, E.H.K.: Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science (New York, N.Y.) 305, 1007–1009 (2004)CrossRefGoogle Scholar
  5. 5.
    Jog, A., Carass, A., Prince, J.L.: Self super-resolution for magnetic resonance images. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 553–560. Springer, Cham (2016). doi: 10.1007/978-3-319-46726-9_64 CrossRefGoogle Scholar
  6. 6.
    Kim, J., Kwon Lee, J., Mu Lee, K.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1646–1654 (2016)Google Scholar
  7. 7.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  8. 8.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
  9. 9.
    Lucy, L.B.: An iterative technique for the rectification of observed distributions. The Astron. J. 79, 745 (1974)CrossRefGoogle Scholar
  10. 10.
    Morales-Navarrete, H., Segovia-Miranda, F., Klukowski, P., Meyer, K., Nonaka, H., Marsico, G., Chernykh, M., Kalaidzidis, A., Zerial, M., Kalaidzidis, Y.: A versatile pipeline for the multi-scale digital reconstruction and quantitative analysis of 3D tissue architecture. Elife 4, e11214 (2015)CrossRefGoogle Scholar
  11. 11.
    Preibisch, S., Amat, F., Stamataki, E., Sarov, M., Singer, R.H., Myers, E., Tomancak, P.: Efficient bayesian-based multiview deconvolution. Nat. Methods 11, 645–648 (2014)CrossRefGoogle Scholar
  12. 12.
    Richardson, W.H.: Bayesian-based iterative method of image restoration. JOSA 62(1), 55–59 (1972)CrossRefGoogle Scholar
  13. 13.
    Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi: 10.1007/978-3-319-24574-4_28 CrossRefGoogle Scholar
  14. 14.
    Sezgin, M., Sankur, B.I.: Survey over image thresholding techniques and quantitative performance evaluation. J. Electron. Imaging 13(1), 146–168 (2004)CrossRefGoogle Scholar
  15. 15.
    Tsien, R.Y.: The green fluorescent protein. Annu. Rev. Biochem. 67, 509–544 (1998)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Martin Weigert
    • 1
    • 2
  • Loic Royer
    • 1
    • 2
  • Florian Jug
    • 1
    • 2
  • Gene Myers
    • 1
    • 2
  1. 1.Max Planck Institute of Molecular Cell Biology and GeneticsDresdenGermany
  2. 2.Center for Systems Biology DresdenDresdenGermany

Personalised recommendations