Abstract
In automatic image annotation (AIA) different features describe images from different aspects or views. Part of information embedded in some views is common for all views, while other parts are individual and specific. In this paper, we present the Mvg-NMF approach, a multi-view-group non-negative matrix factorization (NMF) method for an AIA system which considers both common and individual factors. The NMF framework discovers a latent space by decomposing data into a set of non-negative basis vectors and coefficients. The views divided into homogeneous groups and latent spaces are extracted for each group. After mapping the test images into these spaces, a unified distance matrix is computed from the distance between images in all spaces. Then a search-based method is used to propagate tags from the nearest neighbors to test images. The evaluation on three datasets commonly used for image annotation showed that the Mvg-NMF is highly competitive with the recent state-of-the-art works.
Similar content being viewed by others
Notes
The features we used are available in http://lear.inrialpes.fr/people/guillaumin/data.php
References
Ballan L, Uricchio T, Seidenari L, Del Bimbo A (2014) A cross-media model for automatic image annotation. In: Proceedings of International Conference on Multimedia Retrieval. ACM, p 73
BenAbdallah J, Caicedo JC, Gonzalez FA, Nasraoui O (2010) Multimodal image annotation using non-negative matrix factorization. In: Web Intelligence and Intelligent Agent Technology (WI-IAT), 2010 IEEE/WIC/ACM International Conference on. IEEE, pp 128–135
Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560
Caicedo JC, González FA (2012) Multimodal fusion for image retrieval using matrix factorization. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. ACM, pp 1–8
Caicedo JC, BenAbdallah J, González FA, Nasraoui O (2012) Multimodal representation, indexing, automated annotation and retrieval of image collections via non-negative matrix factorization. Neurocomputing 76(1):50–60
Chen M, Zheng A, Weinberger K (2013) Fast Image Tagging. In: Proceedings of The 30th International Conference on Machine Learning. pp 1274–1282
Ding C, Li T, Jordan MI (2010) Convex and semi-nonnegative matrix factorizations. IEEE Trans Pattern Anal Mach Intell 32(1):45–55
Driesen J (2012) Discovering words in speech using matrix factorization. Ph. D. dissertation, Ph. D. dissertation, KU Leuven, ESAT
Duygulu P, Barnard K, de Freitas JF, Forsyth DA (2002, May) Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In European conference on computer vision. Springer, Berlin, Heidelberg, pp 97–112
Eweiwi A, Cheema MS, Bauckhage C (2013, September) Discriminative joint non-negative matrix factorization for human action classification. In German Conference on Pattern Recognition. Springer, Berlin, Heidelberg, pp 61–70
Grubinger M (2007) Analysis and evaluation of visual information systems performance. Victoria University, Toronto
Guan X, Wang W, Zhang X (2009) Fast intrusion detection based on a non-negative matrix factorization model. J Netw Comput Appl 32(1):31–44
Guan N, Tao D, Luo Z, Yuan B (2011) Manifold regularized discriminative nonnegative matrix factorization with fast gradient descent. IEEE Trans Image Process 20(7):2030–2048
Guillaumin M, Mensink T, Verbeek J, Schmid C (2009) Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In: Computer Vision, 2009 I.E. 12th International Conference on. IEEE, pp 309–316
Johnson J, Ballan L, Fei-Fei L (2015) Love thy neighbors: Image annotation by exploiting image metadata. In: Proceedings of the IEEE International Conference on Computer Vision. pp 4624–4632
Kalayeh MM, Idrees H, Shah M (2014) NMF-KNN: Image Annotation using Weighted Multi-view Non-negative Matrix Factorization. In: Computer Vision and Pattern Recognition (CVPR), 2014 I.E. Conference on. IEEE, pp 184–191
Ke X, Guo W (2016) Multi-scale salient region and relevant visual keywords based model for automatic image annotation. Multimed Tools Appl 75(20):12477–12498
Kejun H, Sidiropoulos ND, Swami A (2014) Non-negative matrix factorization revisited: uniqueness and algorithm for symmetric decomposition. IEEE Trans Signal Process 62(1):211–224. https://doi.org/10.1109/TSP.2013.2285514
Kim H, Park H (2007) Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23(12):1495–1502
Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791
Lin C, Pang M (2015) Graph regularized nonnegative matrix factorization with sparse coding. Math Probl Eng 2015. https://doi.org/10.1155/2015/239589
Lin Z, Ding G, Hu M (2015) Image auto-annotation via tag-dependent random search over range-constrained visual neighbours. Multimed Tools Appl 74(11):4091–4116
Liu Y, Zhang D, Lu G, Ma W-Y (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282
Liu J, Wang C, Gao J, Han J (2013) Multi-view clustering via joint nonnegative matrix factorization. In: Proc. of SDM. SIAM, pp 252–260
Long X, Lu H, Peng Y, Li W (2014) Graph regularized discriminative non-negative matrix factorization for face recognition. Multimed Tools Appl 72(3):2679–2699
Lowe DG (1999) Object recognition from local scale-invariant features. In: Computer vision, 1999. The proceedings of the seventh IEEE international conference on. IEEE, pp 1150–1157
Makadia A, Pavlovic V, Kumar S (2008). A new baseline for image annotation. Computer Vision–ECCV 2008, pp. 316–329
Manning CD, Raghavan P, Schütze H (2008) Introduction to information retrieval, vol 1. Cambridge university press, Cambridge
Mirzaei S, Norouzi Y (2015) Blind audio source counting and separation of anechoic mixtures using the multichannel complex NMF framework. Signal Process 115:27–37
Moran S, Lavrenko V (2014) A sparse kernel relevance model for automatic image annotation. Int J Multimed Inf Retr 3(4):209–229
Murthy VN, Can EF, Manmatha R (2014) A hybrid model for automatic image annotation. In: Proceedings of International Conference on Multimedia Retrieval. ACM, p 369
Prajapati SJ, Jadhav KR (2015) Brain tumor detection by various image segmentation techniques with introduction to non negative matrix factorization. Brain 4(3):600–603
Rad R, Jamzad M (2015) Automatic image annotation by a loosely joint non-negative matrix factorisation. IET Comput Vis 9(6):806–813
Rad R, Jamzad M (2017) Image annotation using multi-view non-negative matrix factorization with different number of basis vectors. J Vis Commun Image Represent 46:1–12
Sun S (2013) A survey of multi-view machine learning. Neural Comput & Applic 23(7–8):2031–2038
Verma Y, Jawahar CV (2012, October) Image annotation using metric learning in semantic neighbourhoods. In European Conference on Computer Vision. Springer, Berlin, Heidelberg, pp 836-849
Virtanen T (2007) Monaural sound source separation by nonnegative matrix factorization with temporal continuity and sparseness criteria. IEEE Trans Audio Speech Lang Process 15(3):1066–1074
Von Ahn L, Dabbish L (2004) Labeling images with a computer game. In: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, pp 319–326
Wang D, Lu H (2013) On-line learning parts-based representation via incremental orthogonal projective non-negative matrix factorization. Signal Process 93(6):1608–1623
Wang Y-X, Zhang Y-J (2013) Nonnegative matrix factorization: a comprehensive review. IEEE Trans Knowl Data Eng 25(6):1336–1353
Xiang Y, Zhou X, Chua T-S (2009) Ngo C-W A revisit of generative model for automatic image annotation using markov random fields. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, pp 1153–1160
Xu C, Tao D, Xu C (2013) A survey on multi-view learning. Neural Comput Appl 23(7–8):2031–2038
Xu H, Pan P, Xu C, Lu Y, Chen D (2016) Image auto-annotation via concept interdependency network. Multimed Tools Appl 75(11):6237–6261
Yang Y, Zhang W, Xie Y (2015) Image automatic annotation via multi-view deep representation. J Vis Commun Image Represent 33:368–377
Zar JH (1998) Spearman rank correlation. In: Armitage P, Colton T (eds-inchief) Encyclopedia of biostatistics, vol 5. John Wiley and Sons, Chichester, pp 4191–4196
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rad, R., Jamzad, M. A multi-view-group non-negative matrix factorization approach for automatic image annotation. Multimed Tools Appl 77, 17109–17129 (2018). https://doi.org/10.1007/s11042-017-5279-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-5279-4