Advertisement

UltraGAN: Ultrasound Enhancement Through Adversarial Generation

  • Maria EscobarEmail author
  • Angela Castillo
  • Andrés Romero
  • Pablo Arbeláez
Conference paper
  • 510 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12417)

Abstract

Ultrasound images are used for a wide variety of medical purposes because of their capacity to study moving structures in real time. However, the quality of ultrasound images is significantly affected by external factors limiting interpretability. We present UltraGAN, a novel method for ultrasound enhancement that transfers quality details while preserving structural information. UltraGAN incorporates frequency loss functions and an anatomical coherence constraint to perform quality enhancement. We show improvement in image quality without sacrificing anatomical consistency. We validate UltraGAN on a publicly available dataset for echocardiography segmentation and demonstrate that our quality-enhanced images are able to improve downstream tasks. To ensure reproducibility we provide our source code and training models.

Keywords

Generative Adversarial Networks Echocardiography Ultrasound images Image quality enhancement 

Notes

Acknowledgements

The present study is funded by MinCiencias, contract number 853-2019 project ID# 120484267276.

References

  1. 1.
    Abdi, A.H., Jafari, M.H., Fels, S., Tsang, T., Abolmaesumi, P.: A study into echocardiography view conversion. In: Workshop of Medical Imaging Meets NeurIPS (2019)Google Scholar
  2. 2.
    Abdi, A.H., Tsang, T., Abolmaesumi, P.: GAN-enhanced conditional echocardiogram generation. In: Workshop of Medical Imaging Meets NeurIPS (2019)Google Scholar
  3. 3.
    Abhishek, K., Hamarneh, G.: Mask2Lesion: mask-constrained adversarial skin lesion image synthesis. In: Burgos, N., Gooya, A., Svoboda, D. (eds.) Simulation and Synthesis in Medical Imaging, pp. 71–80. Springer International Publishing, Cham (2019).  https://doi.org/10.1007/978-3-030-32778-1_8CrossRefGoogle Scholar
  4. 4.
    Armstrong, A.C., et al.: Quality control and reproducibility in M-mode, two-dimensional, and speckle tracking echocardiography acquisition and analysis: the cardia study, year 25 examination experience. Echocardiography 32(8), 1233–1240 (2015)CrossRefGoogle Scholar
  5. 5.
    Cahalan, M.K., et al.: American society of echocardiography and society of cardiovascular anesthesiologists task force guidelines for training in perioperative echocardiography. J. Am. Soc. Echocardiogr. 15(6), 647–652 (2002)CrossRefGoogle Scholar
  6. 6.
    Chen, L., Bentley, P., Mori, K., Misawa, K., Fujiwara, M., Rueckert, D.: Intelligent image synthesis to attack a segmentation CNN using adversarial learning. In: Burgos, N., Gooya, A., Svoboda, D. (eds.) Simulation and Synthesis in Medical Imaging, pp. 90–99. Springer International Publishing, Cham (2019).  https://doi.org/10.1007/978-3-030-32778-1_10CrossRefGoogle Scholar
  7. 7.
    Chen, Y.S., Wang, Y.C., Kao, M.H., Chuang, Y.Y.: Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6306–6314 (2018)Google Scholar
  8. 8.
    Deshpande, A., Lu, J., Yeh, M.C., Jin Chong, M., Forsyth, D.: Learning diverse image colorization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6837–6845 (2017)Google Scholar
  9. 9.
    Duarte-Salazar, C.A., Castro-Ospina, A.E., Becerra, M.A., Delgado-Trejos, E.: Speckle noise reduction in ultrasound images for improving the metrological evaluation of biomedical applications: an overview. IEEE Access 8, 15983–15999 (2020)CrossRefGoogle Scholar
  10. 10.
    Fritsche, M., Gu, S., Timofte, R.: Frequency separation for real-world super-resolution. In: ICCV Workshop (2019)Google Scholar
  11. 11.
    Gardin, J.M., et al.: Recommendations for a standardized report for adult transthoracic echocardiography: a report from the American society of echocardiography’s nomenclature and standards committee and task force for a standardized echocardiography report. J. Am. Soc. Echocardiogr. 15(3), 275–290 (2002)CrossRefGoogle Scholar
  12. 12.
    Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  13. 13.
    Gottdiener, J.S., et al.: American society of echocardiography recommendations for use of echocardiography in clinical trials: a report from the American society of echocardiography’s guidelines and standards committee and the task force on echocardiography in clinical trials. J. Am. Soc. Echocardiogr. 17(10), 1086–1119 (2004)Google Scholar
  14. 14.
    Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  15. 15.
    Jafari, M.H., et al.: Semi-supervised learning for cardiac left ventricle segmentation using conditional deep generative models as prior. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 649–652. IEEE (2019)Google Scholar
  16. 16.
    Jafari, M.H., et al.: Cardiac point-of-care to cart-based ultrasound translation using constrained cyclegan. Int, J. Comput. Assist. Radiol. Surg. 15(5), 877–886 (2020).  https://doi.org/10.1007/s11548-020-02141-y
  17. 17.
    Jafari, M.H., et al.: Echocardiography segmentation by quality translation using anatomically constrained CycleGAN. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 655–663. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-32254-0_73CrossRefGoogle Scholar
  18. 18.
    Lartaud, P.-J., Rouchaud, A., Rouet, J.-M., Nempont, O., Boussel, L.: Spectral CT based training dataset generation and augmentation for conventional CT vascular segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 768–775. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-32245-8_85CrossRefGoogle Scholar
  19. 19.
    Leclerc, S., et al.: Deep learning for segmentation using an open large-scale dataset in 2D echocardiography. IEEE Trans. Med. Imaging 38(9), 2198–2210 (2019)CrossRefGoogle Scholar
  20. 20.
    Liao, Z., et al.: Echocardiography view classification using quality transfer star generative adversarial networks. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 687–695. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-32245-8_76CrossRefGoogle Scholar
  21. 21.
    Mazaheri, S., et al.: Echocardiography image segmentation: a survey. In: 2013 International Conference on Advanced Computer Science Applications and Technologies, pp. 327–332. IEEE (2013)Google Scholar
  22. 22.
    Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)
  23. 23.
    Ng, A., Swanevelder, J.: Resolution in ultrasound imaging. Continuing Educ. Anaesth. Crit. Care Pain 11(5), 186–192 (2011)CrossRefGoogle Scholar
  24. 24.
    Oliva, A., Torralba, A., Schyns, P.G.: Hybrid images. ACM Trans. Graph. (TOG) 25(3), 527–532 (2006)CrossRefGoogle Scholar
  25. 25.
    Ortiz, S.H.C., Chiu, T., Fox, M.D.: Ultrasound image enhancement: a review. Biomed. Signal Process. Control 7(5), 419–428 (2012)CrossRefGoogle Scholar
  26. 26.
    Romero, A., Arbeláez, P., Van Gool, L., Timofte, R.: SMIT: stochastic multi-label image-to-image translation. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (2019)Google Scholar
  27. 27.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  28. 28.
    Wang, X., Chan, K.C., Yu, K., Dong, C., Loy, C.C.: EDVR: video restoration with enhanced deformable convolutional networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2019Google Scholar
  29. 29.
    Yang, H., et al.: Unpaired brain MR-to-CT synthesis using a structure-constrained CycleGAN. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 174–182. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-00889-5_20CrossRefGoogle Scholar
  30. 30.
    Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV) (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Center for Research and Formation in Artificial Intelligence, Universidad de los AndesBogotáColombia
  2. 2.Computer Vision Lab, ETHZZürichSwitzerland

Personalised recommendations