Skip to main content
SpringerLink
Log in

Acceleration of 3D feature-enhancing noise filtering in hybrid CPU/GPU systems

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

FlowDenoising is a new approach to noise reduction in biological volumes obtained with three-dimensional electron microscopy (3DEM). Its abilities to enhance the structural features stem from the fact that an anisotropic Gaussian filtering is steered according to the local structures. To this end, the Optical Flow (OF) among consecutive slices is estimated, which is the most computationally expensive step in this approach. In this article, a hybrid CPU/GPU implementation of FlowDenoising is introduced and evaluated. It exploits parallel computing by distributing the workload among multiple cores and takes advantage of the massive processing in GPUs to accelerate the OF estimation. The hybrid implementation provides remarkable speed-up factors and an important reduction of the processing time, which is particularly relevant for the denoising of huge volumes typically found in 3DEM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The data of this study will be available from the corresponding authors on reasonable request. Code will be available through the github of the authors (https://github.com/microscopy-processing/FlowDenoising).

Notes

  1. Available at https://github.com/microscopy-processing/FlowDenoising.

  2. Notice that we have used the NumPy indexing notation.

  3. https://docs.opencv.org/3.4/d9/d30/classcv_1_1cuda_1_1FarnebackOpticalFlow.html

References

  1. Flynn M (1972) Some computer organizations and their effectiveness. IEEE Trans Comput C–21:948–960

    Article  Google Scholar 

  2. Hennessy JL, Patterson DA (2019) Computer architecture. A quantitative approach. 6th Ed. Morgan Kauffman, USA

  3. Brooks D (2011) CPUs, GPUs, and hybrid computing. IEEE Micro 31:4–6

    Article  Google Scholar 

  4. Agulleiro JI, Vazquez F, Garzón EM, Fernandez JJ (2012) Hybrid computing: CPU+GPU co-processing and its application to tomographic reconstruction. Ultramicroscopy 115:109–114. https://doi.org/10.1016/j.ultramic.2012.02.003

    Article  CAS  PubMed  Google Scholar 

  5. Eisenstein M (2023) Seven technologies to watch in 2023. Nature 613:794–797

    Article  ADS  CAS  PubMed  Google Scholar 

  6. Peddie CJ et al (2022) Volume electron microscopy. Nat Rev Methods Prim 2:51. https://doi.org/10.1038/s43586-022-00131-9

    Article  CAS  Google Scholar 

  7. Fernandez JJ, Martinez-Sanchez A (2022) Computational methods for three-dimensional electron microscopy (3DEM). Comput Methods Programs Biomed 225:107039. https://doi.org/10.1016/j.cmpb.2022.107039

    Article  PubMed  Google Scholar 

  8. Fernandez JJ (2012) Computational methods for electron tomography. Micron 43:1010–1030. https://doi.org/10.1016/j.micron.2012.05.003

    Article  CAS  PubMed  Google Scholar 

  9. Fernandez JJ, Li S (2003) An improved algorithm for anisotropic nonlinear diffusion for denoising cryo-tomograms. J Struct Biol 144:152–161. https://doi.org/10.1016/j.jsb.2003.09.010

    Article  PubMed  Google Scholar 

  10. Fernandez JJ, Li S (2005) Anisotropic nonlinear filtering of cellular structures in cryo-electron tomography. Comput Sci Eng 7(5):54–61. https://doi.org/10.1109/MCSE.2005.89

    Article  Google Scholar 

  11. Moreno JJ, Martinez-Sanchez A, Martinez JA, Garzon EM, Fernandez JJ (2018) TomoEED: fast edge-enhancing denoising of tomographic volumes. Bioinformatics 34:3776–3778. https://doi.org/10.1093/bioinformatics/bty435

    Article  CAS  PubMed  Google Scholar 

  12. Lehtinen J, Munkberg J, Hasselgren J, Laine S, Karras T, Aittala M, Aila T (2018) Noise2Noise: learning image restoration without clean data. In: Dy J, Krause A (eds) Proceedings of the 35th International Conference on Machine Learning, vol. 80, pp 2965–2974

  13. Krull A, Buchholz T-O, Jug F (2019) Noise2void-learning denoising from single noisy images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2129–2137. https://doi.org/10.1109/CVPR.2019.00223

  14. Buchholz T-O, Krull A, Shahidi R, Pigino G, Jekely G, Jug F (2019) Content-aware image restoration for electron microscopy. In: Muller-Reichert T, Pigino G (eds) Three-dimensional electron microscopy. Methods in cell biology, vol 152. Academic Press, Cambridge, pp 277–289

    Chapter  Google Scholar 

  15. Tabik S, Garzón EM, García I, Fernandez JJ (2007) High performance noise reduction for biomedical multidimensional data. Digit Signal Process 17:724–736. https://doi.org/10.1016/j.dsp.2006.11.004

    Article  Google Scholar 

  16. Fernandez JJ (2008) High performance computing in structural determination by electron cryomicroscopy. J Struct Biol 164:1–6. https://doi.org/10.1016/j.jsb.2008.07.005

    Article  CAS  PubMed  Google Scholar 

  17. Cuomo S, Michele PD, Piccialli F (2014) 3D data denoising via Nonlocal means filter by using parallel GPU strategies. Comput Math Methods Med 164:523862

    Google Scholar 

  18. Tabik S, Murarasu A, Romero LF (2014) Anisotropic nonlinear diffusion for filtering 3d images on gpus. In: 2014 IEEE International Conference on Cluster Computing (CLUSTER). https://doi.org/10.1109/CLUSTER.2014.6968786

  19. Kwon K, Kim MS, Shin BS (2016) A fast 3D adaptive bilateral filter for ultrasound volume visualization. Comput Methods Programs Biomed 133:25–34. https://doi.org/10.1016/j.cmpb.2016.05.008

    Article  PubMed  Google Scholar 

  20. Yano K, Sugimoto K, Kamata S-i (2018) GPU-friendly Approximate Bilateral Filter for 3D Volume Data. In: 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), pp 2054–2058. https://doi.org/10.23919/APSIPA.2018.8659773

  21. Steyer AM, Ruhwedel T, Nardis C, Werner HB, Nave KA, Möbius W (2020) Pathology of myelinated axons in the PLP-deficient mouse model of spastic paraplegia type 2 revealed by volume imaging using focused ion beam-scanning electron microscopy. J Struct Biol 210:107492. https://doi.org/10.1016/j.jsb.2020.107492

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Hennies J, Lleti JMS, Schieber NL, Templin RM, Steyer AM, Schwab Y (2020) AMST: alignment to median smoothed template for focused ion beam scanning electron microscopy image stacks. Sci Rep 10:2004. https://doi.org/10.1038/s41598-020-58736-7

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  23. Uwizeye C, Decelle J, Jouneau P-H, Flori S, Gallet B, Keck J-B, Bo DD, Moriscot C, Seydoux C, Chevalier F, Schieber NL, Templin R, Allorent G, Courtois F, Curien G, Schwab Y, Schoehn G, Zeeman SC, Falconet D, Finazzi G (2021) Morphological bases of phytoplankton energy management and physiological responses unveiled by 3D subcellular imaging. Nat Commun 12:1049. https://doi.org/10.1038/s41467-021-21314-0

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  24. González-Ruiz V, Fernández-Fernández MR, Fernandez JJ (2023) Structure-preserving gaussian denoising of FIB-SEM volumes. Ultramicroscopy 246:113674. https://doi.org/10.1016/j.ultramic.2022.113674

    Article  CAS  PubMed  Google Scholar 

  25. González-Ruiz V, Fernandez J-J (2023) Flowdenoising: structure-preserving denoising in 3d electron microscopy (3dem). SoftwareX 23:101413. https://doi.org/10.1016/j.softx.2023.101413

    Article  Google Scholar 

  26. Farnebäck G (2003) Two-frame motion estimation based on polynomial expansion. In: Scandinavian Conference on Image Analysis, pp 363–370. https://doi.org/10.1007/3-540-45103-X_50

  27. Bykov YS et al (2017) The structure of the COPI coat determined within the cell. eLife 6:32493. https://doi.org/10.7554/eLife.32493

    Article  Google Scholar 

  28. Knott G, Rosset S, Cantoni M (2011) Focussed ion beam milling and scanning electron microscopy of brain tissue. JoVE (Journal of Visualized Experiments) 53:2588. https://doi.org/10.3791/2588

    Article  Google Scholar 

  29. Lucchi A, Li Y, Fua P (2013) Learning for structured prediction using approximate subgradient descent with working sets. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp 1987–1994. https://doi.org/10.1109/CVPR.2013.259

  30. Wietrzynski W et al (2020) Charting the native architecture of chlamydomonas thylakoid membranes with single-molecule precision. eLife 9:53740. https://doi.org/10.7554/eLife.53740

    Article  Google Scholar 

Download references

Acknowledgements

Work supported by MCIN/AEI/10.13039/501100011033, “ERDF A way of making Europe” and by the “European Union NextGenerationEU/PRTR” through grants PID2021-123278OB-I00, TED2021-132020B-I00, PID2022-139071NB-I00 and PDC2022-133370-I00.

Funding

This work was supported by Spanish Ministry of Science and Innovation MCIN/AEI/10.13039/501100011033, “ERDF A way of making Europe” and by the “European Union NextGenerationEU/PRTR” through grants PID2021-123278OB-I00, TED2021-132020B-I00, PID2022-139071NB-I00 and PDC2022-133370-I00.

Author information

Authors and Affiliations

  1. Department of Informatics, University of Almería, Agrifood Campus of International Excellence (ceiA3), Carretera de Sacramento s/n, 04120, Almería, Spain

    V. González-Ruiz & J. J. Moreno

  2. Spanish National Research Council (CINN-CSIC), Health Research Institute of Asturias (ISPA), Av Hospital Universitario s/n, 33011, Oviedo, Spain

    J. J. Fernández

Authors
  1. V. González-Ruiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. J. Moreno
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. J. Fernández
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

VGR and JJF contributed to Conceptualization, Investigation, Software, Validation, Writing, and Funding Acquisition. JJM contributed to Investigation and Software.

Corresponding authors

Correspondence to V. González-Ruiz or J. J. Fernández.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

González-Ruiz, V., Moreno, J.J. & Fernández, J.J. Acceleration of 3D feature-enhancing noise filtering in hybrid CPU/GPU systems. J Supercomput (2024). https://doi.org/10.1007/s11227-024-05928-x

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11227-024-05928-x

Keywords

Search

Navigation