Microscopy Image Restoration with Deep Wiener-Kolmogorov Filters

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12365)


Microscopy is a powerful visualization tool in biology, enabling the study of cells, tissues, and the fundamental biological processes; yet, the observed images typically suffer from blur and background noise. In this work, we propose a unifying framework of algorithms for Gaussian image deblurring and denoising. These algorithms are based on deep learning techniques for the design of learnable regularizers integrated into the Wiener-Kolmogorov filter. Our extensive experimentation line showcases that the proposed approach achieves a superior quality of image reconstruction and surpasses the solutions that rely either on deep learning or on optimization schemes alone. Augmented with the variance stabilizing transformation, the proposed reconstruction pipeline can also be successfully applied to the problem of Poisson image deblurring, surpassing the state-of-the-art methods. Moreover, several variants of the proposed framework demonstrate competitive performance at low computational complexity, which is of high importance for real-time imaging applications .


Deblurring Denoising Learnable regularizers Microscopy deblurring Wiener filter 

Supplementary material

504476_1_En_12_MOESM1_ESM.pdf (27.1 mb)
Supplementary material 1 (pdf 27745 KB)


  1. 1.
    PSF Generator. Accessed 25 Feb 2020
  2. 2.
    Diffraction PSF 3D. Accessed 30 May 2019
  3. 3.
    Al-Kofahi, Y., Zaltsman, A.B., Graves, R.M., Marshall, W., Rusu, M.: A deep learning-based algorithm for 2-D cell segmentation in microscopy images. BMC Bioinform. 19 (2018)Google Scholar
  4. 4.
    AL-Qinani, I.H.: Deblurring image and removing noise from medical images for cancerous diseases using a wiener filter. IRJET 8(4), 2354–2365 (2017)Google Scholar
  5. 5.
    Anscombe, F.J.: The transformation of Poisson, binomial and negative-binomial data. Biometrika 35(3–4), 246–254 (1948)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Arjomand Bigdeli, S., Zwicker, M., Favaro, P., Jin, M.: Deep mean-shift priors for image restoration. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 763–772. Curran Associates, Inc. (2017)Google Scholar
  7. 7.
    van Beek, P., Yang, J., Yamamoto, S., Ueda, Y.: Image deblurring and denoising with non-local regularization constraint. In: Information Processing and Communications, vol. 7543, January 2010Google Scholar
  8. 8.
    Bertsekas, D.P.: Nonlinear Programming, 2nd edn. (1999)Google Scholar
  9. 9.
    Boyat, A.K., Joshi, B.K.: Image denoising using wavelet transform and Wiener filter based on log energy distribution over Poisson-Gaussian noise model, pp. 1–6 (2014)Google Scholar
  10. 10.
    Buades, A., Coll, B., Morel, J.: A non-local algorithm for image denoising. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 60–65 (2005)Google Scholar
  11. 11.
    Chen, D.Q.: Regularized generalized inverse accelerating linearized alternating minimization algorithm for frame-based Poissonian image deblurring. SIAM J. Imaging Sci. 7, 716–739 (2014)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Chen, K.: Introduction to variational image-processing models and applications. Int. J. Comput. Math. 90, 1–8 (2013)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Chowdhury, A., et al.: Blood vessel characterization using virtual 3D models and convolutional neural networks in fluorescence microscopy. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI), pp. 629–632. IEEE (2017)Google Scholar
  14. 14.
    Conchello, J.A., Lichtman, J.W.: Fluorescence microscopy. Nat. Methods 2(12), 910–919 (2005)CrossRefGoogle Scholar
  15. 15.
    Dey, N., et al.: Richardson-Lucy algorithm with total variation regularization for 3D confocal microscope deconvolution. Microsc. Res. Tech. 69, 4 (2006)CrossRefGoogle Scholar
  16. 16.
    Eigen, D., Krishnan, D., Fergus, R.: Restoring an image taken through a window covered with dirt or rain. In: 2013 IEEE International Conference on Computer Vision, pp. 633–640 (2013)Google Scholar
  17. 17.
    Evangelista, V., Barsanti, L., Passarelli, V., Gualtieri, P.: From cells to proteins: imaging nature across dimensions. In: Proceedings of the NATO Advanced Study Institute, Pisa, Italy (2005)Google Scholar
  18. 18.
    Foi, A., Trimeche, M., Katkovnik, V., Egiazarian, K.: Practical Poissonian-Gaussian noise modeling and fitting for single-image raw-data. IEEE Trans. Image Process. 17(10), 1737–1754 (2008)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014)Google Scholar
  20. 20.
    Kokkinos, F., Lefkimmiatis, S.: Deep image demosaicking using a cascade of convolutional residual denoising networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 317–333. Springer, Cham (2018). Scholar
  21. 21.
    Kokkinos, F., Lefkimmiatis, S.: Iterative joint image demosaicking and denoising using a residual denoising network. IEEE Trans. Image Process. PP, 1 (2019)Google Scholar
  22. 22.
    Krishnan, D., Fergus, R.: Fast image deconvolution using hyper-laplacian priors. In: Bengio, Y., Schuurmans, D., Lafferty, J.D., Williams, C.K.I., Culotta, A. (eds.) Advances in Neural Information Processing Systems 22, pp. 1033–1041. Curran Associates, Inc. (2009)Google Scholar
  23. 23.
    Kruse, J., Rother, C., Schmidt, U.: Learning to push the limits of efficient FFT-based image deconvolution. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4596–4604 (2017)Google Scholar
  24. 24.
    Lefkimmiatis, S.: Universal denoising networks: a novel CNN architecture for image denoising. In: Proceedings of the CVPR, June 2018Google Scholar
  25. 25.
    Lefkimmiatis, S.: Non-local color image denoising with convolutional neural networks, pp. 5882–5891 (2017)Google Scholar
  26. 26.
    Lefkimmiatis, S., Unser, M.: Poisson image reconstruction with Hessian Schatten-norm regularization. IEEE Trans. Image Process. 22, 4314–4327 (2013)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Li, J., Luisier, F., Blu, T.: PURE-LET image deconvolution. IEEE Trans. Image Process. 27(1), 92–105 (2018)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Lu, H., Cheng, J.H., Han, G., Li, L., Liang, Z.: 3D distance-weighted Wiener filter for Poisson noise reduction in sinogram space for SPECT imaging. In: Antonuk, L.E., Yaffe, M.J. (eds.) Medical Imaging 2001: Physics of Medical Imaging, vol. 4320, pp. 905–913. International Society for Optics and Photonics, SPIE (2001)Google Scholar
  29. 29.
    Lucy, L.B.: An iterative technique for the rectification of observed distributions. Astron. J. 79, 745–754 (1974)CrossRefGoogle Scholar
  30. 30.
    Makitalo, M., Foi, A.: A closed-form approximation of the exact unbiased inverse of the Anscombe variance-stabilizing transformation. IEEE Trans. Image Process. 20(9), 2697–2698 (2011)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Makitalo, M., Foi, A.: Optimal inversion of the generalized Anscombe transformation for Poisson-Gaussian noise. IEEE Trans. Image Process. 22(1), 91–103 (2013)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Mildenhall, B., Barron, J.T., Chen, J., Sharlet, D., Ng, R., Carroll, R.: Burst denoising with kernel prediction networks, pp. 2502–2510 (2018)Google Scholar
  33. 33.
    de Monvel, J.B., Calvez, S.L., Ulfendahl, M.: Image restoration for confocal microscopy: improving the limits of deconvolution, with application to the visualization of the mammalian hearing organ. Biophys. J. 80(5), 2455–70 (2001)CrossRefGoogle Scholar
  34. 34.
    Reeves, S.J.: Fast image restoration without boundary artifacts. IEEE Trans. Image Process. 14(10), 1448–1453 (2005)CrossRefGoogle Scholar
  35. 35.
    Richardson, W.H.: Bayesian-based iterative method of image restoration\(\ast \). J. Opt. Soc. Am. 62(1), 55–59 (1972)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear Total Variation Based Noise Removal Algorithms. Elsevier North-Holland Inc., USA (1992)CrossRefGoogle Scholar
  37. 37.
    Schmidt, U., Roth, S.: Shrinkage fields for effective image restoration. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2774–2781 (2014)Google Scholar
  38. 38.
    Shaw, P.J., Rawlins, D.J.: The point-spread function of a confocal microscope: its measurement and use in deconvolution of 3-D data. J. Microsc. 163(2), 151–165 (1991)CrossRefGoogle Scholar
  39. 39.
    Sheppard, C., Wilson, T.: Image formation in confocal scanning microscopes. Optik - Int. J. Light Electron Opt. 55, 331–342 (1980)Google Scholar
  40. 40.
    Shewchuk, J.R.: An introduction to the conjugate gradient method without the agonizing pain. Technical report, USA (1994)Google Scholar
  41. 41.
    Tao, X., Gao, H., Liao, R., Wang, J., Jia, J.: Detail-revealing deep video super-resolution. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4482–4490 (2017)Google Scholar
  42. 42.
    Tikhonov, A.N.: Solution of incorrectly formulated problems and the regularization method. Soviet Math. Dokl. 4, 1035–1038 (1963)zbMATHGoogle Scholar
  43. 43.
    Tintner, G., Kailath, T.: Linear least-squares estimation (1980)Google Scholar
  44. 44.
    Ulyanov, D., Vedaldi, A., Lempitsky, S.V.: Instance normalization: the missing ingredient for fast stylization. arXiv: 1607.08022. Computer Vision and Pattern Recognition (2016)
  45. 45.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)CrossRefGoogle Scholar
  46. 46.
    Wiener, N.: The Extrapolation, Interpolation and Smoothing of Stationary Time Series, with Engineering Applications. Wiley, New York (1949)CrossRefGoogle Scholar
  47. 47.
    Wu, Q., Merchant, F., Castleman, K.: Microscope Image Processing. Elsevier (2010)Google Scholar
  48. 48.
    Xu, L., Ren, J.S., Liu, C., Jia, J.: Deep convolutional neural network for image deconvolution. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 1790–1798. Curran Associates, Inc. (2014)Google Scholar
  49. 49.
    Zhang, J., Pan, J., Lai, W., Lau, R.W.H., Yang, M.: Learning fully convolutional networks for iterative non-blind deconvolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6969–6977 (2017)Google Scholar
  50. 50.
    Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2808–2817 (2017)Google Scholar
  51. 51.
    Zhang, X.: An effective SURE-based Wiener filter for image denoising. In: Liang, Q., Mu, J., Wang, W., Zhang, B. (eds.) CSPS 2016. LNEE, vol. 423, pp. 889–895. Springer, Singapore (2018). Scholar
  52. 52.
    Zhang, Y., et al.: A Poisson-Gaussian denoising dataset with real fluorescence microscopy images. In: 2019 IEEE Conference on Computer Vision and Pattern Recognition (2019)Google Scholar
  53. 53.
    Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for image restoration with neural networks. IEEE Trans. Comput. Imaging 3, 47–57 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Skolkovo Institute of Science and TechnologyMoscowRussia
  2. 2.University College LondonLondonUK
  3. Inc.Redwood CityUSA

Personalised recommendations