Skip to main content

k-SALSA: k-Anonymous Synthetic Averaging of Retinal Images via Local Style Alignment

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Abstract

The application of modern machine learning to retinal image analyses offers valuable insights into a broad range of human health conditions beyond ophthalmic diseases. Additionally, data sharing is key to fully realizing the potential of machine learning models by providing a rich and diverse collection of training data. However, the personally-identifying nature of retinal images, encompassing the unique vascular structure of each individual, often prevents this data from being shared openly. While prior works have explored image de-identification strategies based on synthetic averaging of images in other domains (e.g. facial images), existing techniques face difficulty in preserving both privacy and clinical utility in retinal images, as we demonstrate in our work. We therefore introduce k-SALSA, a generative adversarial network (GAN)-based framework for synthesizing retinal fundus images that summarize a given private dataset while satisfying the privacy notion of k-anonymity. k-SALSA brings together state-of-the-art techniques for training and inverting GANs to achieve practical performance on retinal images. Furthermore, k-SALSA leverages a new technique, called local style alignment, to generate a synthetic average that maximizes the retention of fine-grain visual patterns in the source images, thus improving the clinical utility of the generated images. On two benchmark datasets of diabetic retinopathy (EyePACS and APTOS), we demonstrate our improvement upon existing methods with respect to image fidelity, classification performance, and mitigation of membership inference attacks. Our work represents a step toward broader sharing of retinal images for scientific collaboration. Code is available at https://github.com/hcholab/k-salsa.

H. Park—This work was performed while the author was at Korea University.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.kaggle.com/c/aptos2019-blindness-detection.

  2. 2.

    https://www.kaggle.com/c/diabetic-retinopathy-detection.

References

  1. Abramian, D., Eklund, A.: Refacing: reconstructing anonymized facial features using GANS. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) (2019)

    Google Scholar 

  2. Alaluf, Y., Patashnik, O., Cohen-Or, D.: Restyle: A residual-based stylegan encoder via iterative refinement. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2021)

    Google Scholar 

  3. Bischoff-Grethe, A., et al.: A technique for the deidentification of structural brain MR images. Hum. Brain Mapp. 28(9), 892–903 (2007)

    Article  Google Scholar 

  4. Burlina, P., Paul, W., Liu, T.Y.A., Bressler, N.M.: Detecting anomalies in retinal diseases using generative, discriminative, and self-supervised deep learning. JAMA Ophthalmol. 140(2), 185–189 (2022)

    Article  Google Scholar 

  5. Burlina, P.M., Joshi, N., Pacheco, K.D., Liu, T.Y.A., Bressler, N.M.: Assessment of deep generative models for high-resolution synthetic retinal image generation of age-related macular degeneration. JAMA Ophthalmol. 137(3), 258–264 (2019)

    Article  Google Scholar 

  6. Chen, Y., Long, J., Guo, J.: RF-GANs: a method to synthesize retinal fundus images based on generative adversarial network. Comput. Intell. Neurosci. (2021)

    Google Scholar 

  7. Coyner, A.S., et al.: Diagnosability of synthetic retinal fundus images for plus disease detection in retinopathy of prematurity. In: AMIA Symposium (2020)

    Google Scholar 

  8. Dwork, C., Roth, A., et al.: The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9(3–4), 211–407 (2014)

    MathSciNet  MATH  Google Scholar 

  9. Fei-Fei, L., Deng, J., Li, K.: ImageNet: constructing a large-scale image database. J. Vis. 9(8), 1037–1037 (2009)

    Article  Google Scholar 

  10. Garfinkel, S., et al.: De-identification of Personal Information: US Department of Commerce. National Institute of Standards and Technology (2015)

    Google Scholar 

  11. Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  12. Gkoulalas-Divanis, A., Loukides, G., Sun, J.: Publishing data from electronic health records while preserving privacy: a survey of algorithms. J. Biomed. Inf. 50, 4–19 (2014)

    Article  Google Scholar 

  13. Goodfellow, I.: Nips 2016 tutorial: generative adversarial networks. arXiv preprint. arXiv:1701.00160 (2016)

  14. der Goten, V., Alexander, L., Hepp, T., Akata, Z., Smith, K.: Conditional de-identification of 3d magnetic resonance images. arXiv preprint. arXiv:2110.09927 (2021)

  15. Gui, J., Sun, Z., Wen, Y., Tao, D., Ye, J.: A review on generative adversarial networks: Algorithms, theory, and applications. IEEE Trans. Knowl. Data Eng. (2021)

    Google Scholar 

  16. He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  17. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  18. Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in Neural Information Processing Systems (2017)

    Google Scholar 

  19. Jakob, C.E., Kohlmayer, F., Meurers, T., Vehreschild, J.J., Prasser, F.: Design and evaluation of a data anonymization pipeline to promote open science on COVID-19. Sci. Data 7(1), 1–10 (2020)

    Article  Google Scholar 

  20. Jeong, Y., et al.: FICGAN: facial identity controllable GAN for de-identification. arXiv preprint. arXiv:2110.00740 (2021)

  21. Jourabloo, A., Yin, X., Liu, X.: Attribute preserved face de-identification. In: 2015 International Conference on Biometrics (ICB) (2015)

    Google Scholar 

  22. Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., Aila, T.: Training generative adversarial networks with limited data. In: Advances in Neural Information Processing Systems (2020)

    Google Scholar 

  23. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)

    Google Scholar 

  24. Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  25. Kifer, D., Machanavajjhala, A.: Pufferfish: a framework for mathematical privacy definitions. ACM Trans. Database Syst. (TODS) 39(1), 1–36 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  26. Korot, E., et al.: Predicting sex from retinal fundus photographs using automated deep learning. Sci. Rep. 11(1), 1–8 (2021)

    Article  Google Scholar 

  27. Long, Y., et al.: G-PATE: scalable differentially private data generator via private aggregation of teacher discriminators. In: Advances in Neural Information Processing Systems (2021)

    Google Scholar 

  28. Mariño, C., Penedo, M.G., Penas, M., Carreira, M.J., Gonzalez, F.: Personal authentication using digital retinal images. Pattern Anal. Appl. 9, 21 (2006)

    Article  MathSciNet  Google Scholar 

  29. Maximov, M., Elezi, I., Leal-Taixé, L.: Ciagan: conditional identity anonymization generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)

    Google Scholar 

  30. Meden, B., Emeršič, Ž, Štruc, V., Peer, P.: k-same-Net: k-anonymity with generative deep neural networks for face deidentification. Entropy 20(1), 60 (2018)

    Article  Google Scholar 

  31. Milchenko, M., Marcus, D.: Obscuring surface anatomy in volumetric imaging data. Neuroinformatics 11, 65–75 (2013). https://doi.org/10.1007/s12021-012-9160-3

    Article  Google Scholar 

  32. Mohamed, Q., Gillies, M.C., Wong, T.Y.: Management of diabetic retinopathy: a systematic review. Jama 298(8), 902–916 (2007)

    Article  Google Scholar 

  33. Newton, E.M., Sweeney, L., Malin, B.: Preserving privacy by de-identifying face images. IEEE Trans. Knowl. Data Eng. 17(2), 232–243 (2005)

    Article  Google Scholar 

  34. Niu, Y., Gu, et al.: Pathological evidence exploration in deep retinal image diagnosis. In: Proceedings of the AAAI Conference on Artificial Intelligence (2019)

    Google Scholar 

  35. Paul, W., Cao, Y., Zhang, M., Burlina, P.: Defending medical image diagnostics against privacy attacks using generative methods. arXiv preprint. arXiv:2103.03078 (2021)

  36. Poplin, R., et al.: Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat. Biomed. Eng. 2(3), 158–164 (2018)

    Article  Google Scholar 

  37. Ravindra, V., Grama, A.: De-anonymization attacks on neuroimaging datasets. In: Proceedings of the 2021 International Conference on Management of Data (2021)

    Google Scholar 

  38. Ribaric, S., Pavesic, N.: An overview of face de-identification in still images and videos. In: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) (2015)

    Google Scholar 

  39. Schimke, N., Kuehler, M., Hale, J.: Preserving privacy in structural neuroimages. In: Li, Y. (ed.) DBSec 2011. LNCS, vol. 6818, pp. 301–308. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22348-8_26

    Chapter  Google Scholar 

  40. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)

    Google Scholar 

  41. Sweeney, L.: k-anonymity: a model for protecting privacy. Int. J. Uncertainty Fuzziness Knowl.-Based Syst. 10(05), 557–570 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  42. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)

    Google Scholar 

  43. Taylor, R.: AI and the retina: finding Patterns of Systemic Disease. EyeNet Magazine (2021)

    Google Scholar 

  44. Tom, E., et al.: Protecting data privacy in the age of AI-enabled ophthalmology. Transl. Vis. Sci. Technol. 9(2), 36–36 (2020)

    Article  Google Scholar 

  45. Dept, U.S.: of Health and Human Services: Standards for privacy of individually identifiable health information. Final Rule, Federal Registrar (2002)

    Google Scholar 

  46. Wagner, S.K., et al.: Insights into systemic disease through retinal imaging-based oculomics. Trans. Vis. Sci. Technol. 9(2), 6 (2020)

    Article  Google Scholar 

  47. Wang, X., Zhang, R., Shen, C., Kong, T., Li, L.: Dense contrastive learning for self-supervised visual pre-training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021)

    Google Scholar 

  48. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  49. Wisely, C.E., et al.: Convolutional neural network to identify symptomatic alzheimer’s disease using multimodal retinal imaging. Br. J. Ophthalmol. 106(3), 388–395 (2022)

    Article  Google Scholar 

  50. Wu, Y., Yang, F., Xu, Y., Ling, H.: Privacy-protective-GAN for privacy preserving face De-identification. J. Comput. Sci. Technol. 34(1), 47–60 (2019). https://doi.org/10.1007/s11390-019-1898-8

    Article  Google Scholar 

  51. Xia, W., Zhang, Y., Yang, Y., Xue, J.H., Zhou, B., Yang, M.H.: Gan inversion: a survey. arXiv preprint. arXiv:2101.05278 (2021)

  52. Xu, C., Ren, J., Zhang, D., Zhang, Y., Qin, Z., Ren, K.: GANobfuscator: mitigating information leakage under GAN via differential privacy. IEEE Trans. Inf. Forensics Secur. 14(9), 2358–2371 (2019)

    Article  Google Scholar 

  53. Yu, Z., Xiang, Q., Meng, J., Kou, C., Ren, Q., Lu, Y.: Retinal image synthesis from multiple-landmarks input with generative adversarial networks. Biomed. Eng. Online 18(1), 1–15 (2019)

    Article  Google Scholar 

  54. Zhou, Y., Wang, B., He, X., Cui, S., Shao, L.: DR-GAN: conditional generative adversarial network for fine-grained lesion synthesis on diabetic retinopathy images. IEEE J. Biomed. Health Inform. (2020)

    Google Scholar 

Download references

Acknowledgement

M.J. is supported by the Ministry of Trade, Industry, and Energy in Korea, under Human Resource Development Program for Industrial Innovation (Global) (P0017311) supervised by the Korea Institute for Advancement of Technology. H.C. is supported by NIH DP5 OD029574-01 and by the Schmidt Fellows Program at Broad Institute.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Hyunwoo J. Kim , Michael Morley or Hyunghoon Cho .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 3022 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jeon, M., Park, H., Kim, H.J., Morley, M., Cho, H. (2022). k-SALSA: k-Anonymous Synthetic Averaging of Retinal Images via Local Style Alignment. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13681. Springer, Cham. https://doi.org/10.1007/978-3-031-19803-8_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19803-8_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19802-1

  • Online ISBN: 978-3-031-19803-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics