Label Distribution Learning Based on Ensemble Neural Networks

  • Yansheng Zhai
  • Jianhua Dai
  • Hong Shi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11303)


Label distribution learning (LDL), as an extension of multi-label learning, is a new arising machine learning technique to deal with label ambiguity problems. The maximum entropy model is commonly used in label distribution learning. However, it does not consider the correlation between the labels and is not suitable for nonlinear relationships, and the prediction performance is also limited. In this paper, we propose a label distribution learning algorithm based on ensemble neural networks. The algorithm trains neural networks with preferences using training sets with different label sets to construct base learners, and combines the base learners with the weights, which is learned by the combined learner to obtain the final learning results. Experimental results show that the proposed algorithm is effective for label distribution data.


Label distribution learning Neural networks Ensemble learning Maximum entropy model 



This work was partially supported by the National Natural Science Foundation of China (Nos. 61473259, 61502335, 61070074, 60703038) and the Hunan Provincial Science & Technology Program Project (No. 2018TP1018).


  1. 1.
    Berger, A.L., Pietra, S.A.D., Pietra, V.J.D.: A maximum entropy approach to natural language processing. Comput. Linguist. 22, 39–71 (1996)Google Scholar
  2. 2.
    Gao, B., Xing, C., Xie, C., Wu, J., Geng, X.: Deep label distribution learning with label ambiguity. IEEE Trans. Image Process. 26(6), 2825–2838 (2017)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Geng, X.: Label distribution learning. IEEE Trans. Knowl. Data Eng. 28(7), 1734–1748 (2014)CrossRefGoogle Scholar
  4. 4.
    Geng, X., Xia, Y.: Head pose estimation based on multivariate label distribution. In: Computer Vision and Pattern Recognition, pp. 1837–1842 (2014)Google Scholar
  5. 5.
    Geng, X., Xu, N., Sao, R.: Label enhancement for label distribution learning. J. Comput. Res. Dev. 54(6), 1171–1184 (2017)Google Scholar
  6. 6.
    Geng, X., Yin, C., Zhou, Z.: Facial age estimation by learning from label distributions. IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2401–2412 (2013)CrossRefGoogle Scholar
  7. 7.
    Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: International Conference on Artificial Intelligence and Statistics, pp. 315–323 (2012)Google Scholar
  8. 8.
    He, Z., et al.: Data-dependent label distribution learning for age estimation. IEEE Trans. Image Process. 26(8), 3846–3858 (2017)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014).
  10. 10.
    Sao, D., Yang, W.Y., Zhao, H.: Application of k-means algorithm to realize label distribution learning. Trans. Intell. Syst. 12(3), 325–332 (2017)Google Scholar
  11. 11.
    Sung-Hyuk, C.: Comprehensive survey on distance/similarity measures between probability density functions. Int. J. Math. Model. Methods Appl. Sci. 1(4), 300–307 (2007)Google Scholar
  12. 12.
    Wang, Q., Geng, X.: Face estimation based on adaptive label distribution. Master’s thesis, Southeast University, NanJing (2015)Google Scholar
  13. 13.
    Yuan, Y.X.: A modified bfgs algorithm for unconstrained optimization. IMA J. Numer. Anal. 11(3), 325–332 (1991)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Zhang, M., Zhou, Z.: ML-KNN: a lazy learning approach to multi-label learning. Pattern Recogn. 40(7), 2038–2048 (2007)CrossRefGoogle Scholar
  15. 15.
    Zhang, Z., Wang, M., Geng, X.: Crowd counting in public video surveillance by label distribution learning. Neurocomputing 166(1), 151–163 (2015)CrossRefGoogle Scholar
  16. 16.
    Zhao, Q., Geng, X.: Study on several problems of label distribution learning. Master’s thesis, Southeast University, NanJing (2016)Google Scholar
  17. 17.
    Zhao, Q., Geng, X.: Selection of objective function in label distribution learning. J. Front. Comput. Sci. Technol. 11(5), 708–719 (2017)Google Scholar
  18. 18.
    Zhou, Y., Geng, X.: Application of label distribution in face image feature recognition. Master’s thesis, Southeast University, NanJing (2016)Google Scholar
  19. 19.
    Zhou, Y., Xue, H., Geng, X.: Emotion distribution recognition from facial expressions. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 1247–1250. ACM (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer Science and TechnologyTianjin UniversityTianjinChina
  2. 2.Hunan Provincial Key Laboratory of Intelligent Computing and Language Information Processing, College of Information Science and EngineeringHunan Normal UniversityChangshaChina

Personalised recommendations