Advertisement

Comparison of Privacy-Preserving Distributed Deep Learning Methods in Healthcare

Conference paper
  • 148 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12722)

Abstract

Data privacy regulations pose an obstacle to healthcare centres and hospitals to share medical data with other organizations, which in turn impedes the process of building deep learning models in the healthcare domain. Distributed deep learning methods enable deep learning models to be trained without the need for sharing data from these centres while still preserving the privacy of the data at these centres. In this paper, we compare three privacy-preserving distributed learning techniques: federated learning, split learning, and SplitFed. We use these techniques to develop binary classification models for detecting tuberculosis from chest X-rays and compare them in terms of classification performance, communication and computational costs, and training time. We propose a novel distributed learning architecture called SplitFedv3, which performs better than split learning and SplitFedv2 in our experiments. We also propose alternate mini-batch training, a new training technique for split learning, that performs better than alternate client training, where clients take turns to train a model.

Keywords

Privacy-preserving Distributed deep learning Federated learning Split learning SplitFed Medical imaging 

References

  1. 1.
    Bustos, A., Pertusa, A., Salinas, J.M., de la Iglesia-Vayá, M.: Padchest: alarge chest x-ray image dataset with multi-label annotated reports. Med.Image Anal. 66, 101797 (2020)Google Scholar
  2. 2.
    Centers for Medicare & Medicaid Services: The Health Insurance Portability and Accountability Act of 1996 (HIPAA) (1996). http://www.cms.hhs.gov/hipaa/
  3. 3.
    Gupta, O., Raskar, R.: Distributed learning of deep neural network over multiple agents. J. Netw. Comput. Appl. 116, 1–8 (2018)CrossRefGoogle Scholar
  4. 4.
    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)Google Scholar
  5. 5.
    Johnson, A.E., et al.: Mimic-cxr-jpg, a large publicly available database of labeled chest radiographs. arXiv preprint arXiv:1901.07042 (2019)
  6. 6.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  7. 7.
    Konečnỳ, J., McMahan, H.B., Ramage, D., Richtárik, P.: Federated optimization: Distributed machine learning for on-device intelligence. arXiv preprint arXiv:1610.02527 (2016)
  8. 8.
    Li, W., et al.: Privacy-preserving federated brain tumour segmentation. In: Suk, H.-I., Liu, M., Yan, P., Lian, C. (eds.) MLMI 2019. LNCS, vol. 11861, pp. 133–141. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-32692-0_16CrossRefGoogle Scholar
  9. 9.
    Liu, B., Yan, B., Zhou, Y., Yang, Y., Zhang, Y.: Experiments of federated learning for covid-19 chest x-ray images. arXiv preprint arXiv:2007.05592 (2020)
  10. 10.
    McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: Artificial Intelligence and Statistics, pp. 1273–1282. PMLR (2017)Google Scholar
  11. 11.
    McMahan, B., Rampage, D.: Federated learning: collaborative machine learning without centralized training data. https://ai.googleblog.com/2017/04/federated-learning-collaborative.html
  12. 12.
    Poirot, M.G., Vepakomma, P., Chang, K., Kalpathy-Cramer, J., Gupta, R., Raskar, R.: Split learning for collaborative deep learning in healthcare. arXiv preprint arXiv:1912.12115 (2019)
  13. 13.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28CrossRefGoogle Scholar
  14. 14.
    Roth, H.R., et al.: Federated learning for breast density classification: a real-world implementation. In: Albarqouni, S., et al. (eds.) DART/DCL -2020. LNCS, vol. 12444, pp. 181–191. Springer, Cham (2020).  https://doi.org/10.1007/978-3-030-60548-3_18CrossRefGoogle Scholar
  15. 15.
    Ryffel, T., et al.: A generic framework for privacy preserving deep learning. arXiv preprint arXiv:1811.04017 (2018)
  16. 16.
    Sheller, M.J., et al.: Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci. Rep. 10(1), 1–12 (2020)CrossRefGoogle Scholar
  17. 17.
    Sheller, M.J., Reina, G.A., Edwards, B., Martin, J., Bakas, S.: Multi-institutional deep learning modeling without sharing patient data: a feasibility study on brain tumor segmentation. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 92–104. Springer, Cham (2019).  https://doi.org/10.1007/978-3-030-11723-8_9CrossRefGoogle Scholar
  18. 18.
    Thapa, C., Chamikara, M.A.P., Camtepe, S.: Splitfed: when federated learning meets split learning. arXiv preprint arXiv:2004.12088 (2020)
  19. 19.
    Voigt, P., Von dem Bussche, A.: The eu general data protection regulation (gdpr). A Practical Guide, 1st edn. Springer International Publishing, Cham 10, 3152676 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2021

Authors and Affiliations

  1. 1.DeepTek IncPuneIndia
  2. 2.Singapore Bioimaging Consortium - A*StarSingaporeSingapore

Personalised recommendations