Skip to main content
SpringerLink
Log in

Identity-Preserving Adversarial Training for Robust Network Embedding

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Network embedding, as an approach to learning low-dimensional representations of nodes, has been proved extremely useful in many applications, e.g., node classification and link prediction. Unfortunately, existing network embedding models are vulnerable to random or adversarial perturbations, which may degrade the performance of network embedding when being applied to downstream tasks. To achieve robust network embedding, researchers introduce adversarial training to regularize the embedding learning process by training on a mixture of adversarial examples and original examples. However, existing methods generate adversarial examples heuristically, failing to guarantee the imperceptibility of generated adversarial examples, and thus limit the power of adversarial training. In this paper, we propose a novel method Identity-Preserving Adversarial Training (IPAT) for network embedding, which generates imperceptible adversarial examples with explicit identity-preserving regularization. We formalize such identity-preserving regularization as a multi-class classification problem where each node represents a class, and we encourage each adversarial example to be discriminated as the class of its original node. Extensive experimental results on real-world datasets demonstrate that our proposed IPAT method significantly improves the robustness of network embedding models and the generalization of the learned node representations on various downstream tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Cui P, Wang X, Pei J, Zhu W W. A survey on network embedding. IEEE Trans. Knowledge and Data Engineering, 2019, 31(5): 833–852. DOI: https://doi.org/10.1109/TKDE.2018.2849727.

    Article  Google Scholar 

  2. Qu L, Zhu H S, Zheng R Q, Shi Y H, Yin H Z. ImGAGN: Imbalanced network embedding via generative adversarial graph networks. In Proc. the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Aug. 2021, pp.1390–1398. DOI: 10.1145/3447548.3467334.

  3. Ruan C Y, Wang Y, Ma J G, Zhang Y C, Chen X T. Adversarial heterogeneous network embedding with metapath attention mechanism. Journal of Computer Science and Technology, 2019, 34(6): 1217–1229. DOI: https://doi.org/10.1007/s11390-019-1971-3.

    Article  Google Scholar 

  4. Hu Z N, Dong Y X, Wang K S, Chang K W, Sun Y Z. GPT-GNN: Generative pre-training of graph neural networks. In Proc. the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Aug. 2020, pp.1857–1867. DOI: 10.1145/3394486.3403237.

  5. Perozzi B, Al-Rfou R, Skiena S. DeepWalk: Online learning of social representations. In Proc. the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Aug. 2014, pp.701–710. DOI: 10.1145/2623330.2623732.

  6. Dai Q Y, Shen X, Zhang L, Li Q, Wang D. Adversarial training methods for network embedding. In Proc. the World Wide Web Conference, May 2019, pp.329–339. DOI: https://doi.org/10.1145/3308558.3313445.

  7. Dai Q Y, Li Q, Tang J, Wang D. Adversarial network embedding. In Proc. the 32nd AAAI Conference on Artificial Intelligence, Feb. 2018, pp.2167–2174. DOI: 10.1609/aaai.v32i1.11865.

  8. Qin Y, Carlini N, Cottrell G W, Goodfellow I J, Raffel C. Imperceptible, robust, and targeted adversarial examples for automatic speech recognition. In Proc. the 36th International Conference on Machine Learning, Jun. 2019, pp.5231–5240.

  9. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I J, Fergus R. Intriguing properties of neural networks. In Proc. the 2nd International Conference on Learning Representations, Apr. 2014.

  10. Kipf T N, Welling M. Variational graph autoencoders. In Proc. the NIPS Workshop on Bayesian Deep Learning, Dec. 2016.

  11. Veličković P, Fedus W, Hamilton W L, Liò P, Bengio Y, Hjelm R D. Deep graph infomax. In Proc. the 7th International Conference on Learning Representations, May 2019.

  12. Grover A, Leskovec J. node2vec: Scalable feature learning for networks. In Proc. the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Aug. 2016, pp.855–864. DOI: 10.1145/2939672.2939754.

  13. Wang D X, Cui P, Zhu W W. Structural deep network embedding. In Proc. the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Aug. 2016, pp.1225–1234. DOI: 10.1145/2939672.2939753.

  14. Liu J, He Z C, Wei L, Huang Y L. Content to node: Selftranslation network embedding. In Proc. the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Aug. 2018, pp.1794–1802. DOI: 10.1145/3219819.3219988.

  15. Gao H C, Huang H. Deep attributed network embedding. In Proc. the 27th International Joint Conference on Artificial Intelligence, Jul. 2018, pp.3364–3370. DOI: 10.24963/ijcai.2018/467.

  16. Goodfellow I J, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. In Proc. the 3rd International Conference on Learning Representations, May 2015.

  17. Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. In Proc. the 5th International Conference on Learning Representations, Apr. 2017.

  18. Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. In Proc. the 6th International Conference on Learning Representations, Apr. 2018.

  19. Sen P, Namata G, Bilgic M, Getoor L, Gallagher B, Eliassi-Rad T. Collective classification in network data. AI Magazine, 2008, 29(3): 93–106. DOI: https://doi.org/10.1609/aimag.v29i3.2157.

    Article  Google Scholar 

  20. Golub G H, Reinsch C. Singular value decomposition and least squares solutions. Numerische Mathematik, 1970, 14(5): 403–420. DOI: https://doi.org/10.1007/BF02163027.

    Article  MathSciNet  Google Scholar 

  21. You Y N, Chen T L, Sui Y D, Chen T, Wang Z Y, Shen Y. Graph contrastive learning with augmentations. In Proc. the 34th International Conference on Neural Information Processing Systems, Dec. 2020.

  22. Pan S R, Hu R Q, Long G D, Jiang J, Yao L N, Zhang C Q. Adversarially regularized graph autoencoder for graph embedding. In Proc. the 27th International Joint Conference on Artificial Intelligence, Jul. 2018, pp.2609–2615. DOI: 10.24963/ijcai.2018/362.

  23. Yang C, Liu Z Y, Zhao D L, Sun M S, Chang E Y. Network representation learning with rich text information. In Proc. the 24th International Conference on Artificial Intelligence, Jun. 2015, pp.2111–2117. DOI: 10.5555/2832415.2832542.

  24. Abadi M, Barham P, Chen J M, Chen Z F, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg J, Monga R, Moore S, Murray D G, Steiner B, Tucker P, Vasudevan V, Warden P, Wicke M, Yu Y, Zheng X Q. Tensorflow: A system for large-scale machine learning. In Proc. the 12th USENIX Conference on Operating Systems Design and Implementation, Jun. 2016, pp.265–283. DOI: 10.5555/3026877.3026899.

  25. Noble W S. What is a support vector machine? Nature Biotechnology, 2006, 24(12): 1565–1567. DOI: https://doi.org/10.1038/nbt1206-1565.

    Article  Google Scholar 

  26. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, VanderPlas J T, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay É. Scikit-learn: Machine learning in Python. The Journal of Machine Learning Research, 2011, 12: 2825–2830. DOI: https://doi.org/10.5555/1953048.2078195.

    Article  MathSciNet  Google Scholar 

  27. Yang Z L, Cohen W W, Salakhutdinov R. Revisiting semi- supervised learning with graph embeddings. In Proc. the 33rd International Conference on Machine Learning, Jun. 2016, pp.40–48.

  28. Hu W H, Fey M, Zitnik M, Dong Y X, Ren H Y, Liu B W, Catasta M, Leskovec J. Open graph benchmark: Datasets for machine learning on graphs. In Proc. the 34th International Conference on Neural Information Processing Systems, Dec. 2020. DOI: 10.5555/2999792.2999959.

  29. Van Der Maaten L, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9(86): 2579–2605.

    Google Scholar 

  30. Xu B B, Shen H W, Cao Q, Qiu Y Q, Cheng X Q. Graph wavelet neural network. In Proc. the 7th International Conference on Learning Representations, Apr. 2019.

  31. Chen T, Kornblith S, Norouzi M, Hinton G E. A simple framework for contrastive learning of visual representations. In Proc. the 37th International Conference on Machine Learning, Jul. 2020, Article No. 149.

  32. Qiu J Z, Chen Q B, Dong Y X, Zhang J, Yang H X, Ding M, Wang K S, Tang J. GCC: Graph contrastive coding for graph neural network pre-training. In Proc. the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Aug. 2020, pp.1150–1160. DOI: 10.1145/3394486.3403168.

  33. Hassani K, Khasahmadi A H. Contrastive multi-view representation learning on graphs. In Proc. the 37th International Conference on Machine Learning, Jul. 2020, pp.4116–4126. DOI: 10.5555/3524938.3525323.

  34. Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y. Generative adversarial nets. In Proc. the 27th International Conference on Neural Information Processing Systems, Dec. 2014, pp.2672–2680.

  35. Wang H W, Wang J, Wang J L, Zhao M, Zhang W N, Zhang F Z, Xie X, Guo M Y. GraphGAN: Graph representation learning with generative adversarial nets. In Proc. the 32nd AAAI Conference on Artificial Intelligence, Feb. 2018, pp.2508–2515. DOI: 10.1609/aaai.v32i1.11872.

  36. Yu W C, Zheng C, Cheng W, Aggarwal C C, Song D L, Zong B, Chen H F, Wang W. Learning deep network representations with adversarially regularized autoencoders. In Proc. the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Aug. 2018, pp.2663–2671. DOI: 10.1145/3219819.3220000.

  37. Qin C L, Martens J, Gowal S, Krishnan D, Dvijotham K, Fawzi A, De S, Stanforth R, Kohli P. Adversarial robustness through local linearization. In Proc. the 33rd International Conference on Neural Information Processing Systems, Dec. 2019, pp.13824–13833.

  38. Miyato T, Dai A M, Goodfellow I. Adversarial training methods for semi-supervised text classification. In Proc. the 5th International Conference on Learning Representations, Apr. 2017.

  39. Zhu D Y, Zhang Z W, Cui P, Zhu W W. Robust graph convolutional networks against adversarial attacks. In Proc. the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Aug. 2019, pp.1399–1407. DOI: 10.1145/3292500.3330851.

  40. Hu W B, Chen C, Chang Y M, Zheng Z B, Du Y F. Robust graph convolutional networks with directional graph adversarial training. Applied Intelligence, 2021, 51(11): 7812–7826. DOI: https://doi.org/10.1007/s10489-021-02272-y.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Data Intelligence System Research Center, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, 100190, China

    Ke-Ting Cen, Hua-Wei Shen, Qi Cao & Bing-Bing Xu

  2. University of Chinese Academy of Sciences, Beijing, 101480, China

    Ke-Ting Cen, Hua-Wei Shen & Xue-Qi Cheng

  3. Beijing Academy of Artificial Intelligence, Beijing, 100000, China

    Hua-Wei Shen

  4. Chinese Academy of Sciences Key Laboratory of Network Data Science and Technology, Institute of Computing Technology Chinese Academy of Sciences, Beijing, 100190, China

    Xue-Qi Cheng

Authors
  1. Ke-Ting Cen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hua-Wei Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qi Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bing-Bing Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xue-Qi Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hua-Wei Shen.

Supplementary Information

ESM 1

(PDF 300 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cen, KT., Shen, HW., Cao, Q. et al. Identity-Preserving Adversarial Training for Robust Network Embedding. J. Comput. Sci. Technol. 39, 177–191 (2024). https://doi.org/10.1007/s11390-023-2256-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-023-2256-4

Keywords

Search

Navigation