Skip to main content
SpringerLink
Log in

Fluctuation-Based Deconvolution in Fluorescence Microscopy Using Plug-and-Play Denoisers

  • Conference paper
  • First Online:
Scale Space and Variational Methods in Computer Vision (SSVM 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14009))

Abstract

The spatial resolution of images of living samples obtained by fluorescence microscopes is physically limited due to the diffraction of visible light, which makes the study of entities of size less than the diffraction barrier (around 200 nm in the x-y plane) very challenging. To overcome this limitation, several deconvolution and super-resolution techniques have been proposed. Within the framework of inverse problems, modern approaches in fluorescence microscopy reconstruct a super-resolved image from a temporal stack of frames by carefully designing suitable hand-crafted sparsity-promoting regularisers. Numerically, such approaches are solved by proximal gradient-based iterative schemes. Aiming at obtaining a reconstruction more adapted to sample geometries (e.g. thin filaments), we adopt a plug-and-play denoising approach with convergence guarantees and replace the proximity operator associated with the explicit image regulariser with an image denoiser (i.e. a pre-trained network) which, upon appropriate training, mimics the action of an implicit prior. To account for the independence of the fluctuations between molecules, the model relies on second-order statistics. The denoiser is then trained on covariance images coming from data representing sequences of fluctuating fluorescent molecules with filament structure. The method is evaluated on both simulated and real fluorescence microscopy images, showing its ability to correctly reconstruct filament structures with high values of peak signal-to-noise ratio (PSNR).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://srm.epfl.ch/Challenge/ChallengeSimulatedData.

  2. 2.

    https://github.com/samuro95/Prox-PnP.

  3. 3.

    https://srm.epfl.ch/Challenge/Challenge2013.

References

  1. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1), 459–494 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  2. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  3. Cohen, R., Blau, Y., Freedman, D., Rivlin, E.: It has potential: gradient-driven denoisers for convergent solutions to inverse problems. In: Advances in Neural Information Processing Systems, vol. 34, pp. 18152–18164. Curran Associates, Inc. (2021)

    Google Scholar 

  4. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)

    Article  MathSciNet  Google Scholar 

  5. Dertinger, T., Colyer, R., Iyer, G., Weiss, S., Enderlein, J.: Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI). PNAS 106, 22287–22292 (2009)

    Article  Google Scholar 

  6. Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two and three space variables. Trans. Am. Math. Soc. 82(2), 421–439 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  7. Duval, V., Peyre, G.: Exact support recovery for sparse spikes. Found. Comput. Math. 15, 1315–1355 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  8. Geman, D., Yang, C.: Nonlinear image recovery with half-quadratic regularization. IEEE Trans. Image Process. 4(7), 932–946 (1995)

    Article  Google Scholar 

  9. Girsault, A., et al.: SOFI simulation tool: a software package for simulating and testing super-resolution optical fluctuation imaging. PLoS ONE 11(9), 1–13 (2016)

    Article  Google Scholar 

  10. Gribonval, R., Nikolova, M.: A characterization of proximity operators. J. Math. Imaging Vis. 62, 773–789 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  11. Gustafsson, N., Culley, S., Ashdown, G., Owen, D.M., Pereira, P.M., Henriques, R.: Fast live-cell conventional fluorophore nanoscopy with ImageJ through super-resolution radial fluctuations. Nature Commun. 7(1), 12471 (2016)

    Article  Google Scholar 

  12. Hell, S.W., Wichmann, J.: Breaking the diffraction resolution limit by stimulated emission: stimulated-emission-depletion fluorescence microscopy. Opt. Lett. 19, 780–782 (1994)

    Article  Google Scholar 

  13. Hurault, S., Leclaire, A., Papadakis, N.: Gradient step denoiser for convergent plug-and-play. In: International Conference on Learning Representations (2022)

    Google Scholar 

  14. Hurault, S., Leclaire, A., Papadakis, N.: Proximal denoiser for convergent plug-and-play optimization with nonconvex regularization. In: Proceedings of Machine Learning Research, vol. 162, pp. 9483–9505. PMLR (2022)

    Google Scholar 

  15. Kamilov, U.S., Bouman, C.A., Buzzard, G.T., Wohlberg, B.: Plug-and-play methods for integrating physical and learned models in computational imaging: theory, algorithms, and applications. IEEE Signal Process. Mag. 40(1), 85–97 (2023)

    Article  Google Scholar 

  16. Kamilov, U.S., Mansour, H., Wohlberg, B.: A plug-and-play priors approach for solving nonlinear imaging inverse problems. IEEE Signal Process. Lett. 24(12), 1872–1876 (2017)

    Article  Google Scholar 

  17. Laumont, R., Bortoli, V.D., Almansa, A., Delon, J., Durmus, A., Pereyra, M.: Bayesian imaging using plug & play priors: when Langevin meets tweedie. SIAM J. Imaging Sci. 15(2), 701–737 (2022)

    Article  MathSciNet  MATH  Google Scholar 

  18. Laville, B., Blanc-Féraud, L., Aubert, G.: Off-the-grid curve reconstruction through divergence regularisation: an extreme point result (2022). HAL preprint

  19. Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1(3), 123–231 (2014)

    Google Scholar 

  20. Reehorst, E.T., Schniter, P.: Regularization by denoising: clarifications and new interpretations. IEEE Trans. Comput. Imaging 5(1), 52–67 (2019)

    Article  Google Scholar 

  21. Romano, Y., Elad, M., Milanfar, P.: The little engine that could: regularization by denoising (red). SIAM J. Imaging Sci. 10(4), 1804–1844 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  22. Ryu, E., Liu, J., Wang, S., Chen, X., Wang, Z., Yin, W.: Plug-and-play methods provably converge with properly trained denoisers. In: Proceedings of the 36th International Conference on Machine Learning, Long Beach, California, USA, 09–15 June 2019, vol. 97, pp. 5546–5557. PMLR (2019)

    Google Scholar 

  23. Sage, D., et al.: Quantitative evaluation of software packages for single-molecule localization microscopy. Nat. Methods 12, 717–724 (2015). https://doi.org/10.1038/nmeth.3442

    Article  Google Scholar 

  24. Sage, D., et al.: Super-resolution fight club: assessment of 2D & 3D single-molecule localization microscopy software. Nat. Methods 16, 387–395 (2019)

    Article  Google Scholar 

  25. Solomon, O., Eldar, Y.C., Mutzafi, M., Segev, M.: SPARCOM: sparsity-based super-resolution correlation microscopy. SIAM J. Imaging Sci. 12(1), 392–419 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  26. Soubies, E., Blanc-Féraud, L., Aubert, G.: A continuous exact \(\ell ^0\) penalty (CEL0) for least squares regularized problem. SIAM J. Imaging Sci. 8(3), 1607–1639 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  27. Stergiopoulou, V., Calatroni, L., de Morais Goulart, H., Schaub, S., Blanc-Féraud, L.: COL0RME: super-resolution microscopy based on sparse blinking/fluctuating fluorophore localization and intensity estimation. Biol. Imaging 2 (2022)

    Google Scholar 

  28. Stergiopoulou, V., Calatroni, L., Schaub, S., Blanc-Féraud, L.: 3D image super-resolution by fluorophore fluctuations and MA-TIRF microscopy reconstruction (3D-COL0RME). In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–4 (2022)

    Google Scholar 

  29. Stergiopoulou, V., de M. Goulart, J.H., Schaub, S., Calatroni, L., Blanc-Féraud, L.: COL0RME: covariance-based \(\ell _0\) super-resolution microscopy with intensity estimation. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 349–352 (2021)

    Google Scholar 

  30. Terris, M., Repetti, A., Pesquet, J.C., Wiaux, Y.: Building firmly nonexpansive convolutional neural networks. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8658–8662 (2020)

    Google Scholar 

  31. Venkatakrishnan, S.V., Bouman, C.A., Wohlberg, B.: Plug-and-play priors for model based reconstruction. In: 2013 IEEE Global Conference on Signal and Information Processing, pp. 945–948 (2013)

    Google Scholar 

  32. Xu, X., Liu, J., Sun, Y., Wohlberg, B., Kamilov, U.S.: Boosting the performance of plug-and-play priors via denoiser scaling. In: 2020 54th Asilomar Conference on Signals, Systems, and Computers, pp. 1305–1312. IEEE (2020)

    Google Scholar 

  33. Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., Timofte, R.: Plug-and-play image restoration with deep denoiser prior. IEEE Trans. Pattern Anal. Mach. Intell. 44(10), 6360–6376 (2021)

    Article  Google Scholar 

  34. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

LC acknowledges the support received by the ANR project TASKABILE (ANR-22-CE48-0010) and the GdR ISIS project SPLIN. VS, LC and LBF acknowledge the support received by the ANR project MICROBLIND (ANR-21-CE48-0008). All authors acknowledge the support received by the H2020 RISE projects NoMADS (GA. 777826).

Author information

Authors and Affiliations

  1. UCA, CNRS, INRIA, Laboratoire I3S, 06903, Sophia-Antipolis, France

    Vasiliki Stergiopoulou, Luca Calatroni & Laure Blanc-Féraud

  2. Department of Computer Science, University of Bath, Bath, UK

    Subhadip Mukherjee

Authors
  1. Vasiliki Stergiopoulou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Subhadip Mukherjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luca Calatroni
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Laure Blanc-Féraud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luca Calatroni .

Editor information

Editors and Affiliations

  1. CNRS, Université Côte d'Azur, Sophia-Antipolis, France

    Luca Calatroni

  2. University of Insubria, Como, Italy

    Marco Donatelli

  3. University of Bologna, Bologna, Italy

    Serena Morigi

  4. University of Modena and Reggio Emilia, Modena, Italy

    Marco Prato

  5. University of Genova, Genova, Italy

    Matteo Santacesaria

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Stergiopoulou, V., Mukherjee, S., Calatroni, L., Blanc-Féraud, L. (2023). Fluctuation-Based Deconvolution in Fluorescence Microscopy Using Plug-and-Play Denoisers. In: Calatroni, L., Donatelli, M., Morigi, S., Prato, M., Santacesaria, M. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2023. Lecture Notes in Computer Science, vol 14009. Springer, Cham. https://doi.org/10.1007/978-3-031-31975-4_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-31975-4_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-31974-7

  • Online ISBN: 978-3-031-31975-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation