Recurrent collective classification

  • Shuangfei FanEmail author
  • Bert Huang
Regular Paper


We propose a new method for training iterative collective classifiers for labeling nodes in network data. The iterative classification algorithm (ICA) is a canonical method for incorporating relational information into classification. Yet, existing methods for training ICA models rely on the assumption that relational features reflect the true labels of the nodes. This unrealistic assumption introduces a bias that is inconsistent with the actual prediction algorithm. In this paper, we introduce recurrent collective classification (RCC), a variant of ICA analogous to recurrent neural network prediction. RCC accommodates any differentiable local classifier and relational feature functions. We provide gradient-based strategies for optimizing over model parameters to more directly minimize the loss function. In our experiments, this direct loss minimization translates to improved accuracy and robustness on real network data. We demonstrate the robustness of RCC in settings where local classification is very noisy, settings that are particularly challenging for ICA.


Collective classification Recurrent neural network Iterative classification algorithm 



We thank anonymous reviewers for their very useful comments and suggestions which help us in the revision of this paper. We also want to thank the editor for his guidance.


  1. 1.
    Angeline PJ, Saunders GM, Pollack JB (1994) An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans Neural Netw 5(1):54–65CrossRefGoogle Scholar
  2. 2.
    Bilgic M, Namata GM, Getoor L (2007) Combining collective classification and link prediction. In: Data mining workshops, 2007. ICDM workshops 2007. Seventh IEEE international conference on. IEEE, pp 381–386Google Scholar
  3. 3.
    Borenstein E, Ullman S (2004) Learning to segment. In: European conference on computer vision. Springer, pp 315–328Google Scholar
  4. 4.
    Connor JT, Martin RD, Atlas LE (1994) Recurrent neural networks and robust time series prediction. IEEE Trans Neural Netw 5(2):240–254CrossRefGoogle Scholar
  5. 5.
    Domke J (2013) Learning graphical model parameters with approximate marginal inference. IEEE Trans Pattern Anal Mach Intell 35(10):2454–2467CrossRefGoogle Scholar
  6. 6.
    Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 12:2122–2159MathSciNetzbMATHGoogle Scholar
  7. 7.
    Fan S, Huang B (2016) Training iterative collective classifiers with back-propagation. In: 12th International workshop on mining and learning with graphs, San Francisco, USAGoogle Scholar
  8. 8.
    Getoor L, Diehl CP (2005) Link mining: a survey. ACM SIGKDD Explor Newsletter 7(2):3–12CrossRefGoogle Scholar
  9. 9.
    Graves A (2012) Supervised sequence labelling. In: Supervised sequence labelling with recurrent neural networks. Springer, Berlin, Heidelberg, pp 5–13Google Scholar
  10. 10.
    Graves A, Jaitly N (2014) Towards end-to-end speech recognition with recurrent neural networks. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 1764–1772Google Scholar
  11. 11.
    Hochreiter S, Bengio Y, Frasconi P, Schmidhuber J (2001) Gradient flow in recurrent nets: the difficulty of learning longterm dependencies. In: Kremer SC, Kolen JF (eds) A Field Guide to Dynamical Recurrent Neural Networks. WileyGoogle Scholar
  12. 12.
    Ishikawa M (1996) Structural learning with forgetting. Neural Netw 9(3):509–521CrossRefGoogle Scholar
  13. 13.
    Jensen D, Neville J, Gallagher B (2004) Why collective inference improves relational classification. In: SIGKDD international conference on knowledge discovery and data mining, pp 593–598Google Scholar
  14. 14.
    Kong X, Shi X, Yu Philip S (2011) Multi-label collective classification. In: SDM, vol 11. SIAM, pp 618–629Google Scholar
  15. 15.
    Lindamood L, Heatherly R, Kantarcioglu M, Thuraisingham B (2009) Inferring private information using social network data. In: Proceedings of the 18th international conference on World wide web. ACM, pp 1145–1146Google Scholar
  16. 16.
    London B, Getoor L (2014) Collective classification of network data. In: Aggarwal CC (ed) Data classification: algorithms and applications. CRC Press, Boca RatonGoogle Scholar
  17. 17.
    Lu Q, Getoor L (2003) Link-based classification. In: Proceedings of the international conference on machine learning (ICML)Google Scholar
  18. 18.
    McAuley JJ, Leskovec J (2012) Learning to discover social circles in ego networks. In: Advances in neural information processing systems, pp 539–547Google Scholar
  19. 19.
    McDowell LK, Gupta KM, Aha DW (2007) Cautious inference in collective classification. In: AAAI, vol 7, pp 596–601Google Scholar
  20. 20.
    McPherson M, Smith-Lovin L, Cook JM (2001) Birds of a feather: homophily in social networks. Annu Rev Sociol 27:415–444CrossRefGoogle Scholar
  21. 21.
    Mikolov T, Karafiát M, Burget L, Cernockỳ J, Khudanpur S (2010) Recurrent neural network based language model. In: INTERSPEECH, vol 2, p 3Google Scholar
  22. 22.
    Moore J, Neville J (2017) Deep collective inference. In: Proceedings of the 31st AAAI conference on artificial intelligenceGoogle Scholar
  23. 23.
    Neville J, Jensen D (2000) Iterative classification in relational data. In: Proc. AAAI-2000 workshop on learning statistical models from relational data, pp 13–20Google Scholar
  24. 24.
    Neville J, Jensen D (2003) Collective classification with relational dependency networks. In: Proceedings of the second international workshop on multi-relational data mining. Citeseer, pp 77–91Google Scholar
  25. 25.
    Pham T, Tran T, Phung D, Venkatesh S (2016) Column networks for collective classification. arXiv preprint arXiv:1609.04508
  26. 26.
    Sen P, Namata G, Bilgic M, Getoor L, Galligher Brian, Eliassi-Rad Tina (2008) Collective classification in network data. AI Mag 29(3):93CrossRefGoogle Scholar
  27. 27.
    Socher R, Lin CC, Manning C, Ng AY (2011) Parsing natural scenes and natural language with recursive neural networks. In: International conference on machine learning (ICML), pp 129–136Google Scholar
  28. 28.
    Taskar B, Abbeel P, Koller D (2002) Discriminative probabilistic models for relational data. In: Proceedings of the eighteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc, pp 485–492Google Scholar
  29. 29.
    Werbos PJ (1990) Backpropagation through time: what it does and how to do it. Proc IEEE 78(10):1550–1560CrossRefGoogle Scholar
  30. 30.
    Xiang R, Neville J (2011) Relational learning with one network: an asymptotic analysis. In: International conference on artificial intelligence & statistics (AISTATS)Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceVirginia TechBlacksburgUSA

Personalised recommendations