Skip to main content
SpringerLink
Log in

Heterogeneous graph neural network with semantic-aware differential privacy guarantees

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Most social networks can be modeled as heterogeneous graphs. Recently, advanced graph learning methods exploit the rich node properties and topological relationships for downstream tasks. That means that more private information is embedded in the representation. However, the existing privacy-preserving methods only focus on protecting the single type of node attributes or relationships, which neglect the significance of high-order semantic information. To address this issue, we propose a novel Heterogeneous graph neural network with Semantic-aware Differential privacy Guarantees named HeteSDG, which provides a double privacy guarantee and performance trade-off in terms of both graph features and topology. In particular, we first reveal the privacy leakage in heterogeneous graphs and define a membership inference attack with a semantic enhancement (MIS) that will improve the means of member inference attacks by obtaining side background knowledge through semantics. Then we design a two-stage mechanism, which includes the feature attention personalized mechanism and the topology gradient perturbation mechanism, where the privacy-preserving technologies are based on differential privacy. These mechanisms will defend against MIS and provide stronger interpretation, but simultaneously bring in noise for representation learning. To better balance the noise perturbation and learning performance, we utilize a bi-level optimization pattern to allocate a suitable privacy budget for the above two modules. Our experiments on four public benchmarks conduct performance experiments, ablation studies, inference attack verification, etc. The results show the privacy protection capability and generalization of HeteSDG.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The source code is released at https://github.com/AixWinnie/HeteDP.

References

  1. He Y, Song Y, Li J, Ji C, Peng J, Peng H (2019) Hetespaceywalk: a heterogeneous spacey random walk for heterogeneous information network embedding. In: Proceedings of the 28th ACM international conference on information and knowledge management, CIKM 2019, Beijing, China, ACM 3–7 Nov 2019, pp 639–648

  2. Dong Y, Hu Z, Wang K, Sun Y, Tang J (2020) Heterogeneous network representation learning. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI 2020, pp 4861–4867

  3. Gong J, Wang S, Wang J, Feng W, Peng H, Tang J, Yu PS (2020) Attentional graph convolutional networks for knowledge concept recommendation in moocs in a heterogeneous view. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, ACM, 25–30 July 2020, pp 79–88

  4. Chen C, Ma W, Zhang M, Wang Z, He X, Wang C, Liu Y, Ma S (2021) Graph heterogeneous multi-relational recommendation. In: Thirty-fifth AAAI conference on artificial intelligence, AAAI 2021, thirty-third conference on innovative applications of artificial intelligence, IAAI 2021. The eleventh symposium on educational advances in artificial intelligence, EAAI 2021, Virtual Event, 2-9 Feb 2021. AAAI Press, pp 3958–3966

  5. Sun Q, Li J, Peng H, Wu J, Ning Y, Yu PS, He L (2021) SUGAR: subgraph neural network with reinforcement pooling and self-supervised mutual information mechanism. In: WWW’21: the web conference 2021, virtual event/Ljubljana, Slovenia, 19–23 April 2021, ACM/IW3C2. pp 2081–2091

  6. Li J, Fu X, Peng H, Wang S, Zhu S, Sun Q, Yu PS, He L (2021) A robust and generalized framework for adversarial graph embedding. arXiv preprint arXiv:2105.10651

  7. Sun Q, Li J, Peng H, Wu J, Fu X, Ji C, Yu PS (2022) Graph structure learning with variational information bottleneck. In: Thirty-sixth AAAI conference on artificial intelligence, AAAI 2022, thirty-fourth conference on innovative applications of artificial intelligence, IAAI 2022, the 12th symposium on educational advances in artificial intelligence, EAAI 2022 virtual event, pp 4165–4174, AAAI Press. Feb 22–Mar 1 2022

  8. Li C, Peng H, Li J, Sun L, Lyu L, Wang Lihong Yu, Philip S, He L (2022) Joint stance and rumor detection in hierarchical heterogeneous graph. IEEE Trans Neural Netw Learn Syst 33(6):2530–2542

    Article  Google Scholar 

  9. Sun Q, Li J, Yuan H, Fu X, Peng H, Ji C, Li Q, Yu PS (2022) Position-aware structure learning for graph topology-imbalance by relieving under-reaching and over-squashing. In: Proceedings of the 31st ACM international conference on information and knowledge management, Atlanta, GA, USA, 17–21 Oct 2022. ACM, pp 1848–1857

  10. Hong H, Guo H, Lin Y, Yang X, Li Z, Ye J (2020) An attention-based graph neural network for heterogeneous structural learning. In: The thirty-fourth AAAI conference on artificial intelligence, AAAI 2020, The thirty-second innovative applications of artificial intelligence conference, IAAI 2020, the tenth AAAI symposium on educational advances in artificial intelligence, EAAI 2020, New York, NY, USA, 7–12 Feb 2020. AAAI Press, pp 4132–4139

  11. Fu X, Zhang J, Meng Z, King I (2020) MAGNN: metapath aggregated graph neural network for heterogeneous graph embedding. In: WWW’20: the web conference 2020, Taipei, Taiwan, 20–24 April 2020. ACM/IW3C2, pp 2331–2341

  12. Hu Z, Dong Y, Wang K, Sun Y (2020) Heterogeneous graph transformer. In: WWW’20: the web conference 2020, Taipei, Taiwan, 20–24 April 2020. ACM/IW3C2, pp 2704–2710

  13. Li J, Peng H, Cao Y, Dou Y, Zhang H, Yu PS, He L (2023) Higher-order attribute-enhancing heterogeneous graph neural networks. IEEE Trans Knowl Data Eng 35(1):560–574

  14. Luo L,Fang Y, Cao X, Zhang X, Zhang W (2021) Detecting communities from heterogeneous graphs: a context path-based graph neural network model. In: CIKM’21: the 30th ACM international conference on information and knowledge management, virtual event, Queensland, Australia. ACM, 1–5 Nov 2021, pp 1170–1180

  15. Zhao J, Wang X, Shi C, Hu B, Song G, Ye Y (2021) Heterogeneous graph structure learning for graph neural networks. In: Thirty-fifth AAAI conference on artificial intelligence, AAAI 2021, thirty-third conference on innovative applications of artificial intelligence, IAAI 2021, the eleventh symposium on educational advances in artificial intelligence, EAAI 2021, Virtual Event, pp 4697–4705. AAAI Press, 2–9 Feb 2021

  16. Yu J, Yin H, Li J, Wang Q, Hung NQV, Zhang X (2021) Self-supervised multi-channel hypergraph convolutional network for social recommendation. In: WWW’21: the web conference 2021, virtual event/Ljubljana, Slovenia, 19–23 April 2021. ACM/IW3C2, pp 413–424

  17. Liu Y, Liang C, He X, Peng J, Zheng Z, Tang J (2022) Modelling high-order social relations for item recommendation. IEEE Trans Knowl Data Eng 34(9):4385–4397

    Article  Google Scholar 

  18. Fu T-Y, Lee W-C, Lei Z (2017) Hin2vec: explore meta-paths in heterogeneous information networks for representation learning. In: Proceedings of the 2017 ACM on conference on information and knowledge management, CIKM 2017, Singapore, 06–10 Nov 2017. ACM, pp 1797–1806

  19. Dong Y, Chawla NV, Swami A (2017) metapath2vec: scalable representation learning for heterogeneous networks. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, Halifax, NS, Canada, 13–17 Aug 2017. ACM, pp 135–144

  20. Li H, Chen Q, Zhu H, Ma D, Wen H, Xuemin SS (2020) Privacy leakage via de-anonymization and aggregation in heterogeneous social networks. IEEE Trans Dependable Secur Comput 17(2):350–362

    Article  Google Scholar 

  21. Li J, Chen G (2021) A personalized trajectory privacy protection method. Comput Secur 108:102323

    Article  Google Scholar 

  22. Bostanipour B, Theodorakopoulos G (2021) Joint obfuscation of location and its semantic information for privacy protection. Comput Secur 107:102310

    Article  Google Scholar 

  23. Li Y, Cao X, Yuan Y, Wang G (2019) Privsem: protecting location privacy using semantic and differential privacy. World Wide Web 22(6):2407–2436

    Article  Google Scholar 

  24. Cunha M, Mendes R, Vilela João P (2021) A survey of privacy-preserving mechanisms for heterogeneous data types. Comput Sci Rev 41:100403

    Article  MathSciNet  MATH  Google Scholar 

  25. Zhang S, Yin H, Chen T, Huang Z, Cui L, Zhang X (2021) Graph embedding for recommendation against attribute inference attacks. In: WWW’21: the web conference 2021, virtual event/Ljubljana, Slovenia, 19–23 April 2021. ACM/IW3C2, pp 3002–3014

  26. Yang C, Wang H, Zhang K, Chen L, Sun L (2021) Secure deep graph generation with link differential privacy. In: Proceedings of the thirtieth international joint conference on artificial intelligence, IJCAI 2021, Virtual Event/Montreal, Canada, 19–27 Aug 2021, pp 3271–3278

  27. Dwork C, Roth A (2014) The algorithmic foundations of differential privacy. Found Trends Theor Comput Sci 9(3–4):211–407

    MathSciNet  MATH  Google Scholar 

  28. Kipf TN, Welling M (2016) Variational graph auto-encoders. arXiv preprint arXiv:1611.07308

  29. Wei Y, Fu X, Sun Q, Peng H, Wu J, Wang J, Li X(2022) Heterogeneous graph neural network for privacy-preserving recommendation. arXiv preprint arXiv:2210.00538

  30. Zhang C, Song D, Huang C, Swami A, Chawla NV (2019) Heterogeneous graph neural network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery and data mining, KDD 2019, Anchorage, AK, USA, 4–8 Aug 2019. ACM, pp 793–803

  31. Schlichtkrull MS, Kipf TN, Bloem P, van den Berg R, Titov I, Max W (2018) Modeling relational data with graph convolutional networks. In: The semantic web–15th international conference, ESWC 2018, Heraklion, Crete, Greece, 3–7 June 2018, Proceedings, volume 10843 of lecture notes in computer science. Springer, Berlin, pp 593–607

  32. van den Berg R, Kipf TN, Welling M (2017) Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263

  33. Yu L, Sun L, Du B, Liu C, Lv W, Xiong H (2020) Hybrid micro/macro level convolution for heterogeneous graph learning. arXiv preprint arXiv:2012.14722

  34. Le W, Li J, Sun P, Hong R, Ge Y, Wang M (2022) Diffnet++: a neural influence and interest diffusion network for social recommendation. IEEE Trans Knowl Data Eng 34(10):4753–4766

    Article  Google Scholar 

  35. Xu F, Lian J, Han Z, Li Y, Xu Y, Xie X (2019) Relation-aware graph convolutional networks for agent-initiated social e-commerce recommendation. In: Proceedings of the 28th ACM international conference on information and knowledge management, CIKM 2019, Beijing, China, 3–7 Nov 2019. ACM, pp 529–538

  36. Yuan M, Chen L, Yu PS (2010) Personalized privacy protection in social networks. Proc VLDB Endow 4(2):141–150

    Article  Google Scholar 

  37. Zheleva E, Getoor L (2007) Preserving the privacy of sensitive relationships in graph data. In: Privacy, security, and trust in KDD, First ACM SIGKDD international workshop, PinKDD 2007, San Jose, CA, USA, August 12, 2007, revised selected papers, volume 4890 of lecture notes in computer science. Springer, Berlin, pp 153–171

  38. Liu K, Terzi E (2008) Towards identity anonymization on graphs. In: Proceedings of the ACM SIGMOD international conference on management of data, SIGMOD 2008, Vancouver, BC, Canada, 10–12 June 2008. ACM, pp 93–106

  39. Abadi M, Chu A, Goodfellow IJ, McMahan HB, Mironov I, Talwar K, Zhang L (2016) Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, Vienna, Austria, 24–28 Oct 2016, pp 308–318. ACM

  40. Sina S, Daniel G-P (2021) Locally private graph neural networks. In: Kim Y, Kim J, Vigna G, Shi E (eds) CCS’21: 2021 ACM SIGSAC conference on computer and communications security, virtual event, Republic of Korea, 15–19 Nov 2021. ACM, pp 2130–2145

  41. Olatunji IE, Funke T, Khosla M (2021) Releasing graph neural networks with differential privacy guarantees. arXiv preprint arXiv:2109.08907

  42. Torkamani S, Ebrahimi JB, Sadeghi P, D’Oliveira RGL, Médard M (2022) Heterogeneous differential privacy via graphs. In: IEEE international symposium on information theory, ISIT 2022, Espoo, Finland, June 26–July 1 2022. IEEE, pp 1623–1628

  43. Dwork C (2006) Differential privacy. In: Automata, languages and programming, 33rd international colloquium, ICALP 2006, Venice, Italy, 10–14 July 2006, Proceedings, Part II, volume 4052 of lecture notes in computer science. Springer, Berlin, pp 1–12

  44. Dwork C, McSherry F, Nissim K, Smith AD (2006) Calibrating noise to sensitivity in private data analysis. In: Theory of cryptography, third theory of cryptography conference, TCC 2006, New York, NY, USA, 4–7 Mar 2006, Proceedings, volume 3876 of lecture notes in computer science. Springer, Berlin. pp 265–284

  45. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, 4–9 Dec 2017, Long Beach, CA, USA, pp 5998–6008

  46. Hamilton WL, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, 4–9 Dec 2017, Long Beach, CA, USA, pp 1024–1034

  47. Franceschi L, Donini M, Frasconi P, Pontil M (2017) Forward and reverse gradient-based hyperparameter optimization. In: Proceedings of the 34th international conference on machine learning, ICML 2017, Sydney, NSW, Australia, 6–11 Aug 2017, volume 70 of proceedings of machine learning research. PMLR, pp 1165–1173

  48. Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9(11):2579–2605

  49. Shokri R, Stronati M, Song C, Shmatikov V(2017) Membership inference attacks against machine learning models. In: 2017 IEEE symposium on security and privacy, SP 2017, San Jose, CA, USA, 22–26 May 2017. IEEE Computer Society, pp 3–18

Download references

Acknowledgements

This paper was supported by the National Natural Science Foundation of China (No. 62162005), Guangxi Science and Technology Major Project (No. AA22067070), National Natural Science Foundation of China (No. U21A20474), Research Fund of Guangxi Key Lab of Multi-source Information Mining and Security (No. 19-A-02-01), Guangxi 1000-Plan of Training Middle-aged/Young Teachers in Higher Education Institutions, Guangxi “Bagui Scholar” Teams for Innovation and Research Project, and Guangxi Collaborative Innovation Center of Multisource Information Integration and Intelligent Processing.

Author information

Authors and Affiliations

  1. Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University, Guilin, 541004, China

    Yuecen Wei, Dongqi Yan, Jinyan Wang & Xianxian Li

  2. Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, Guilin, 541004, China

    Yuecen Wei, Dongqi Yan, Jinyan Wang & Xianxian Li

  3. School of Computer Science and Engineering, Guangxi Normal University, Guilin, 541004, Guangxi, China

    Yuecen Wei, Dongqi Yan, Jinyan Wang & Xianxian Li

  4. Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University, Beijing, 100191, China

    Xingcheng Fu, Qingyun Sun & Hao Peng

  5. School of Computer Science and Engineering, Beihang University, Beijing, 100191, China

    Xingcheng Fu & Qingyun Sun

  6. School of Computing, Macquarie University, Sydney, Australia

    Jia Wu

Authors
  1. Yuecen Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xingcheng Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dongqi Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qingyun Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hao Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jia Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jinyan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xianxian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jinyan Wang or Xianxian Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, Y., Fu, X., Yan, D. et al. Heterogeneous graph neural network with semantic-aware differential privacy guarantees. Knowl Inf Syst 65, 4085–4110 (2023). https://doi.org/10.1007/s10115-023-01895-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-023-01895-6

Keywords

Search

Navigation