Multimedia Tools and Applications

, Volume 77, Issue 3, pp 3083–3104 | Cite as

Semi-supervised adaptive feature analysis and its application for multimedia understanding

  • Xiao-dong Wang
  • Rung-Ching ChenEmail author
  • Fei Yan
  • Zhi-qiang Zeng
  • Chao-qun Hong


Multimedia understanding for high dimensional data is still a challenging work, due to redundant features, noises and insufficient label information it contains. Graph-based semi-supervised feature learning is an effective approach to address this problem. Nevertheless, Existing graph-based semi-supervised methods usually depend on the pre-constructed Laplacian matrix but rarely modify it in the subsequent classification tasks. In this paper, an adaptive local manifold learning based semi-supervised feature selection is proposed. Compared to the state-of-the-art, the proposed algorithm has two advantages: 1) Adaptive local manifold learning and feature selection are integrated jointly into a single framework, where both the labeled and unlabeled data are utilized. Besides, the correlations between different components are also considered. 2) A group sparsity constraint, i.e. l 2 , 1-norm, is imposed to select the most relevant features. We also apply the proposed algorithm to serval kinds of multimedia understanding applications. Experimental results demonstrate the effectiveness of the proposed algorithm.


feature selection semi-supervised learning adaptive learning image annotation 3D human action recognition 



This paper is supported by National Natural Science Foundation of China (Grant No. 61502405), National Natural Science Foundation of Fujian Province, China (Grant Nos. 2016 J01324, 2016J01327, 2017 J01511), the International Science and Technology Cooperation Program of Xiamen university of technology (No.E201400400), Xiamen Science and Technology Planning Project (Nos.3502Z20143030, 3502Z20103037, 3502Z20133043), Scientific Research Fund of Fujian Provincial Education Department (Nos. JA15385, JAT160357), and Ministry of Science and Technology, Taiwan, (Grant Nos. MOST-104-2221-E-324-019-MY2, MOST-103-2632-E-324-001-MY3).


  1. 1.
    Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recogn 37:1757–1771CrossRefGoogle Scholar
  2. 2.
    Cai D, Zhang C, He X (2010) Unsupervised Feature Selection for Multi-Cluster Data. conference on Knowledge discovery and data 333–342Google Scholar
  3. 3.
    Chang X, Yang Y (2016) Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Transactions on Neural Networks and Learning Systems PP:1–12. doi: 10.1109/TNNLS.2016.2582746
  4. 4.
    Chang X, Nie F, Yang Y, Huang H (2014) A Convex Formulation for Semi-supervised Multi-label Feature Selection. In: Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence. pp 1171–1177Google Scholar
  5. 5.
    Chang X, Shen H, Wang S, et al (2014) Semi-supervised feature analysis for multimedia annotation by mining label correlation. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8444 LNAI:74–85Google Scholar
  6. 6.
    Chang X, Yang Y, Hauptmann AG, et al (2015) Semantic Concept Discovery for Large-Scale Zero-Shot Event Detection. In: Proceedings of the 24th International Conference on Artificial Intelligence. AAAI Press, pp 2234–2240Google Scholar
  7. 7.
    Chang X, Nie F, Yang Y et al (2016) Convex Sparse PCA for Unsupervised Feature Learning. ACM Trans Knowl Discov Data 11:1–16. doi: 10.1145/2910585 Google Scholar
  8. 8.
    Chang X, Ma Z, Yang Y et al (2017) Bi-Level Semantic Representation Analysis for Multimedia Event Detection. IEEE Trans Cybern 47:1180–1197. doi: 10.1109/TCYB.2016.2539546 CrossRefGoogle Scholar
  9. 9.
    Chen C, Jafari R, Kehtarnavaz N (2015) UTD-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor. In: Proceedings - International Conference on Image Processing, ICIP. pp 168–172Google Scholar
  10. 10.
    Chen B, Yang J, Jeon B, Zhang X (2017) Kernel quaternion principal component analysis and its application in RGB-D object recognition. Neurocomputing. doi: 10.1016/j.neucom.2017.05.047
  11. 11.
    Du L, Shen Y-D (2015) Unsupervised Feature Selection with Adaptive Structure Learning. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 209–218Google Scholar
  12. 12.
    Duda RO, Hart PE, Stork DG (2000) Pattern Classification, 2Nd edn. Wiley-InterscienceGoogle Scholar
  13. 13.
    He X, Cai D, Niyogi P (2005) Laplacian Score for Feature Selection. Adv Neural Inf Proces Syst 18:507–514Google Scholar
  14. 14.
    Hou C, Nie F, Li X et al (2014) Joint embedding learning and sparse regression: A framework for unsupervised feature selection. IEEE Trans Cybern 44:793–804CrossRefGoogle Scholar
  15. 15.
    Hou C, Nie F, Yi D, Tao D (2015) Discriminative Embedded Clustering: A Framework for Grouping High-Dimensional Data. IEEE Trans Neural Netw Learn Syst 26:1287–1299MathSciNetCrossRefGoogle Scholar
  16. 16.
    Li H, Wang M, Hua XS (2009) MSRA-MM 2.0: A large-scale web multimedia dataset. In: ICDM Workshops 2009 - IEEE International Conference on Data Mining. pp 164–169Google Scholar
  17. 17.
    Li Z, Yang Y, Liu J, et al (2012) Unsupervised Feature Selection Using Nonnegative Spectral Analysis. Twenty-Sixth AAAI Conference on Artificial Intelligence Unsupervised 1026–1032Google Scholar
  18. 18.
    Liu Y, Nie F, Wu J, Chen L (2013) Efficient semi-supervised feature selection with noise insensitive trace ratio criterion. Neurocomputing 105:12–18CrossRefGoogle Scholar
  19. 19.
    Luo M, Chang X, Nie L, et al (2017) An Adaptive Semisupervised Feature Analysis for Video Semantic Recognition. IEEE Transactions on Cybernetics 1–13. doi: 10.1109/TCYB.2017.2647904
  20. 20.
    Ma Z, Nie F, Yang Y et al (2012) Discriminating Joint Feature Analysis for Multimedia Data Understanding. IEEE Trans Multimed 14:1662–1672CrossRefGoogle Scholar
  21. 21.
    Ma Z, Nie F, Yang Y et al (2012) Web Image Annotation via Subspace-Sparsity Collaborated Feature Selection. IEEE Tran Multimed 14:1021–1030CrossRefGoogle Scholar
  22. 22.
    Ma Z, Yang Y, Sebe N, Hauptmann AG (2014) Multiple Features But Few Labels? In: Proceedings of the ACM International Conference on Multimedia - MM ‘14. ACM Press, New York, New York, USA, pp 77–86Google Scholar
  23. 23.
    Nene SA, Nayar SK, Murase H (1996) Columbia University Image Library (COIL-20)Google Scholar
  24. 24.
    Nie F, Xu D, Tsang IWH, Zhang C (2010) Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19:1921–1932MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Nie F, Huang H, Cai X, Ding C (2010) Efficient and Robust Feature Selection via Joint l2,1-Norms Minimization. Adv Neural Inf Proces Syst 23:1813–1821Google Scholar
  26. 26.
    Nie F, Wang H, Huang H, Ding C (2011) Unsupervised and semi-supervised learning via l1-norm graph. 2011 International Conference on Computer Vision 2268–2273Google Scholar
  27. 27.
    Nie F, Wang X, Huang H (2014) Clustering and projected clustering with adaptive neighbors. Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD ‘14 977–986Google Scholar
  28. 28.
    Nie F, Zhu W, Li X (2016) Unsupervised Feature Selection with Structured Graph Optimization. Proceedings of the 30th Conference on Artificial Intelligence (AAAI 2016) 13:1302–1308Google Scholar
  29. 29.
    Siddiqi MH, Ali R, Idris M et al (2016) Human facial expression recognition using curvelet feature extraction and normalized mutual information feature selection. Multimed Tools Appl 75:935–959CrossRefGoogle Scholar
  30. 30.
    Sigal L, Balan AO, Black MJ (2010) HumanEva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int J Comput Vis 87:4–27CrossRefGoogle Scholar
  31. 31.
    Song J, Yang Y, Huang Z et al (2013) Effective multiple feature hashing for large-scale near-duplicate video retrieval. IEEE Trans Multimed 15:1997–2008CrossRefGoogle Scholar
  32. 32.
    Wang X, Zhang X, Zeng Z et al (2016) Unsupervised spectral feature selection with l 1 -norm graph. Neurocomputing 200:47–54CrossRefGoogle Scholar
  33. 33.
    Wang X, Chen R-C, Hong C et al (2017) Semi-supervised multi-label feature selection via label correlation analysis with l 1 -norm graph embedding. Image Vis Comput 63:10–23. doi: 10.1016/j.imavis.2017.05.004 CrossRefGoogle Scholar
  34. 34.
    Yang Y, Shen HT, Ma Z, et al (2011) l2,1-Norm regularized discriminative feature selection for unsupervised learning. IJCAI International Joint Conference on Artificial Intelligence 1589–1594Google Scholar
  35. 35.
    Yang Y, Nie F, Xu D et al (2012) A Multimedia Retrieval Framework Based on Semi-Supervised Ranking and Relevance Feedback. IEEE Trans Pattern Anal Mach Intell 34:723–742CrossRefGoogle Scholar
  36. 36.
    Yang Y, Ma Z, Hauptmann AG et al (2013) Feature Selection for Multimedia Analysis by Shareing Information Among Multiple Tasks. IEEE Trans Multimed 15:661–669CrossRefGoogle Scholar
  37. 37.
    Yang Y, Ma Z, Nie F et al (2014) Multi-Class Active Learning by Uncertainty Sampling with Diversity Maximization. Int J Comput Vis 113:113–127MathSciNetCrossRefGoogle Scholar
  38. 38.
    Yang XK, He L, Qu D, Zhang W (2016) Semi-supervised minimum redundancy maximum relevance feature selection for audio classification. Multimedia Tools and Applications 1–27Google Scholar
  39. 39.
    Yuan C, Sun X, Lv R (2016) Fingerprint liveness detection based on multi-scale LPQ and PCA. China Commun 13:60–65. doi: 10.1109/CC.2016.7559076 CrossRefGoogle Scholar
  40. 40.
    Zeng Z, Wang X, Zhang J, Wu Q (2016) Semi-supervised feature selection based on local discriminative information. Neurocomputing 173:102–109CrossRefGoogle Scholar
  41. 41.
    Zhang Z, Bai L (2015) Unsupervised Feature Selection by Graph Optimization. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol 9279, pp 130–140Google Scholar
  42. 42.
    Zhao Z, Liu H (2007) Semi-supervised feature selection via spectral analysis. Proceedings of the 2007 siam International Conference on Data Mining 641–646Google Scholar
  43. 43.
    Zhou Z-H, Zhang M-L (2007) Multi-Instance Multi-Label Learning with Application to Scene Classification. In: Schölkopf B, Platt J, Hoffman T (eds) Advances in Neural Information Processing Systems. MIT Press, Cambridge, pp 1609–1616Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Xiao-dong Wang
    • 1
    • 2
  • Rung-Ching Chen
    • 2
    Email author
  • Fei Yan
    • 1
  • Zhi-qiang Zeng
    • 1
  • Chao-qun Hong
    • 1
  1. 1.College of Computer and Information EngineeringXiamen University of TechnologyXiamenChina
  2. 2.Department of Information ManagementChaoyang University of TechnologyTaichungTaiwan

Personalised recommendations