Skip to main content
SpringerLink
Log in

Ml-rbf: RBF Neural Networks for Multi-Label Learning

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Multi-label learning deals with the problem where each instance is associated with multiple labels simultaneously. The task of this learning paradigm is to predict the label set for each unseen instance, through analyzing training instances with known label sets. In this paper, a neural network based multi-label learning algorithm named Ml-rbf is proposed, which is derived from the traditional radial basis function (RBF) methods. Briefly, the first layer of an Ml-rbf neural network is formed by conducting clustering analysis on instances of each possible class, where the centroid of each clustered groups is regarded as the prototype vector of a basis function. After that, second layer weights of the Ml-rbf neural network are learned by minimizing a sum-of-squares error function. Specifically, information encoded in the prototype vectors corresponding to all classes are fully exploited to optimize the weights corresponding to each specific class. Experiments on three real-world multi-label data sets show that Ml-rbf achieves highly competitive performance to other well-established multi-label learning algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, New York

    Google Scholar 

  2. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recognition 37(9): 1757–1771

    Article  Google Scholar 

  3. Brinker K, Hüllermeier E (2007) Case-based multilabel ranking. In: Proceedings of the 20th international joint conference on artificial intelligence. Hyderabad, India, pp 702–707

  4. Brinker K, Fürnkranz J, Hüllermeier E (2006) A unified model for multilabel classification and ranking. In: Proceedings of the 17th European conference on artificial intelligence. Riva del Garda, Italy, pp 489–493

  5. Cai L, Hofmann T (2004) Hierarchical document categorization with support vector machines. In: Proceedings of the 13th ACM international conference on information and knowledge management. Washington DC, pp 78–87

  6. Chen G, Song Y, Wang F, Zhang C (2008) Semi-supervised multi-label learning by solving a sylvester equation. In: Proceedings of the 2008 SIAM international conference on data mining. Atlanta, GA, pp 410–419

  7. Clare A, King RD (2001) Knowledge discovery in multi-label phenotype data. In: De Raedt L, Siebes A (eds) Lecture notes in computer science, vol 2168. Springer, Berlin, pp 42–53

    Google Scholar 

  8. Comité FD, Gilleron R, Tommasi M (2003) Learning multi-label altenating decision tree from texts and data. In: Perner P, Rosenfeld A (eds) Lecture notes in computer science, vol 2734. Springer, Berlin,, pp 35–49

    Google Scholar 

  9. Crammer K, Singer Y (2002) A new family of online algorithms for category ranking. In: Proceedings of the 25th annual international ACM SIGIR conference on research and development in information retrieval. Tampere, Finland, pp 151–158

  10. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Statist Soc B 39(1): 1–38

    MATH  MathSciNet  Google Scholar 

  11. Elisseeff A, Weston J (2002) A kernel method for multi-labelled classification. In: Dietterich TG, Becker S, Ghahramani Z (eds) Advances in neural information processing systems, vol 14. MIT Press, Cambridge, MA, pp 681–687

    Google Scholar 

  12. Freund Y, Mason L (1999) The alternating decision tree learning algorithm. In: Proceedings of the 16th international conference on machine learning. Bled, Slovenia, pp 124–133

  13. Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1): 119–139

    Article  MATH  MathSciNet  Google Scholar 

  14. Gao S, Wu W, Lee C-H, Chua T-S (2003) A maximal figure-of-merit learning approach to text categorization. In: Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval. Toronto, Canada, pp 174–181

  15. Gao S, Wu W, Lee C-H, Chua T-S (2004) A MFoM learning approach to robust multiclass multi-label text categorization. In: Proceedings of the 21st international conference on machine learning. Banff, Canada, pp 329–336

  16. Ghamrawi N, McCallum A (2005) Collective multi-label classification. In: Proceedings of the 14th ACM international conference on information and knowledge management. Bremen, Germany, pp 195–200

  17. Godbole S, Sarawagi S (2004) Discriminative methods for multi-labeled classification. In: Dai H, Srikant R, Zhang C (eds) Lecture notes in artificial intelligence, vol 3056. Springer, Berlin, pp 22–30

    Google Scholar 

  18. Joachims T (1998) Text categorization with support vector machines: learning with many relevant features. In: Proceedings of the 10th European conference on machine learning. Chemnitz, Germany, pp 137–142

  19. Kang F, Jin R, Sukthankar R (2006) Correlated label propagation with application to multi-label learning. In: Proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition. New York, NY, pp 1719–1726

  20. Kazawa H, Izumitani T, Taira H, Maeda E (2005) Maximal margin labeling for multi-topic text categorization. In: Saul LK, Weiss Y, Bottou L (eds) Advances in neural information processing systems, vol 17. MIT Press, Cambridge, MA, pp 649–656

    Google Scholar 

  21. Kohonen T (1997) Self-orgnanizing maps, 2nd edn. Springer, Berlin

    Google Scholar 

  22. Liu Y, Jin R, Yang L (2006) Semi-supervised multi-label learning by constrained non-negative matrix factorization. In: Proceedings of the 21st national conference on artificial intelligence. Boston, MA, pp 421–426

  23. Martinetz TM, Schulten KJ (1991) A “neural-gas" network learns topologies. In: Kohonen T, Mäkisara K, Simula O, Kangas J (eds) Artificial neural networks. North-Holland, Amsterdam, pp 397–402

  24. McCallum A (1999) Multi-label text classification with a mixture model trained by EM. In: Working notes of the AAAI’99 workshop on text learning, Orlando, FL

  25. Nigam K, Lafferty J, McCallum A (1999) Using maximum entropy for text classification. In: IJCAI-99 Workshop on machine learning for information filtering. Stockholm, Sweden, pp 61–67

  26. Press WH, Teukolsky SA, Vetterling WT, Flannery BP (1992) Numerical recipes in C: the art of scientific computing. Cambridge University Press, New York

    Google Scholar 

  27. Qi G-J, Hua X-S, Rui Y, Tang J, Mei T, Zhang H-J (2007) Correlative multi-label video annotation. In: Proceedings of the 15th ACM international conference on multimedia. Augsburg, Germany, pp 17–26

  28. Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65: 386–407

    Article  MathSciNet  Google Scholar 

  29. Rousu J, Saunders C, Szedmak S, Shawe-Taylor J (2005) Learning hierarchical multi-category text classifcation models. In: Proceedings of the 22nd international conference on machine learning. Bonn, Germany, pp 774–751

  30. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL (eds) Parallel distributed processing: explorations in the microstructure of cognition, vol 1. MIT Press, Cambridge, MA, pp 318–362

    Google Scholar 

  31. Schapire RE, Singer Y (1998) Improved boosting algorithms using confidence-rated predictions. In: Proceedings of the 11th annual conference on computational learning theory. Madison, WI, pp 80–91

  32. Schapire RE, Singer Y (2000) Boostexter: a boosting-based system for text categorization. Machine Learning 39(2/3): 135–168

    Article  MATH  Google Scholar 

  33. Thabtah FA, Cowling PI, Peng Y (2004) MMAC: a new multi-class, multi-label associative classification approach. In: Proceedings of the 4th IEEE international conference on data mining. Brighton, UK, pp 217–224

  34. Ueda N, Saito K (2003) Parametric mixture models for multi-label text. In: Becker S, Thrun S, Obermayer K (eds) Advances in neural information processing systems, vol 15. MIT Press, Cambridge, MA,, pp 721–728

    Google Scholar 

  35. Veloso A, Wagner M Jr, Gonçalves M, Zaki M (2007) Multi-label lazy associative classification. In: Kok JN, Koronacki J, Mantaras RL, Matwin S, Mladenič D, Skowron A (eds) Lecture notes in artificial intelligence, vol 4702. Springer, Berlin, pp 605–612

    Google Scholar 

  36. Zhang ML, Zhou Z-H (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10): 1338–1351

    Article  Google Scholar 

  37. Zhang M-L, Zhou Z-H (2007) Ml-knn: a lazy learning approach to multi-label learning. Pattern Recog 40(7): 2038–2048

    Article  MATH  Google Scholar 

  38. Zhu S, Ji X, Xu W, Gong Y (2005) Multi-labelled classification using maximum entropy method. In: Proceedings of the 28th annual international ACM SIGIR conference on research and development in information retrieval. Salvador, Brazil, pp 274–281

Download references

Author information

Authors and Affiliations

  1. College of Computer and Information Engineering, Hohai University, Nanjing, 210098, China

    Min-Ling Zhang

  2. National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210093, China

    Min-Ling Zhang

Authors
  1. Min-Ling Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min-Ling Zhang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, ML. Ml-rbf: RBF Neural Networks for Multi-Label Learning. Neural Process Lett 29, 61–74 (2009). https://doi.org/10.1007/s11063-009-9095-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-009-9095-3

Keywords

Search

Navigation