Skip to main content
SpringerLink
Log in

Learning to detect concepts with Approximate Laplacian Eigenmaps in large-scale and online settings

  • Regular Paper
  • Published:
International Journal of Multimedia Information Retrieval Aims and scope Submit manuscript

Abstract

We present a versatile and effective manifold learning approach to tackle the concept detection problem in large-scale and online settings. We demonstrate that Approximate Laplacian Eigenmaps, which constitute a latent representation of the manifold underlying a set of images, offer a compact yet effective feature representation for the problem of concept detection. We expose the theoretical principles of the approach and present an extension that renders the approach applicable in online settings. We evaluate the approach on a number of well-known and two new datasets, coming from the social media domain, and demonstrate that it achieves equal or slightly better detection accuracy compared to supervised methods, while at the same time offering substantial speedup, enabling for instance the training of ten concept detectors using 1.5M images in just 3 min on a commodity server. We also explore a number of factors that affect the detection accuracy of the proposed approach, including the size of training set, the role of unlabelled samples in semi-supervised learning settings, and the performance of the approach across different concepts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://acmmm13.org/submissions/call-for-multimedia-grand-challenge-solutions/yahoo-large-scale-flickr-tag-image-classification-challenge/.

  2. http://www.socialsensor.eu/datasets/mm-concept-detection-dataset-2013/mm-concept-detection-datasets.zip.

  3. http://www.socialsensor.eu/datasets/mm-concept-detection-dataset-2013/mm-concept-detection-twitter2013-images.zip.

  4. MiAP is also known as 11-points interpolated average precision. It is computed with the vl_pr() method of the vlfeat library, http://www.vlfeat.org/matlab/vl_pr.html.

  5. http://www.vlfeat.org/.

  6. https://github.com/socialsensor/multimedia-indexing.

  7. https://github.com/socialsensor/mm-concept-detection-experiments.

References

  1. Balasubramanian M, Schwartz EL (2002) The isomap algorithm and topological stability. Science 295(5552):7

  2. Bart T, Adrian P (2012) Overview of the clef 2012 flickr photo annotation and retrieval task. In: The working notes for the clef 2012 labs and workshop, Rome, Italy

  3. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (surf). Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  4. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434

    MATH  MathSciNet  Google Scholar 

  5. Bengio Y, Delalleau O, Roux N, Paiement J, Vincent P, Ouimet M (2004) Learning eigenfunctions links spectral embedding and kernel pca. Neural Comput 16(10):2197–2219

    Article  MATH  Google Scholar 

  6. Chen X, Mu Y, Yan S, Chua TS (2010) Efficient large-scale image annotation by probabilistic collaborative multi-label propagation. In: Proceedings of the international conference on multimedia, MM ’10ACM, New York, NY, USA, pp 35–44

  7. Chua TS, Tang J, Hong R, Li H, Luo Z, Zheng YT (2009) Nus-wide: a real-world web image database from national university of singapore. In: Proceedings of ACM conference on image and video retrieval (CIVR’09), Santorini, Greece, 8–10 July 2009

  8. Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ (2008) LIBLINEAR: a library for large linear classification. J Mach Learn Res 9:1871–1874

    MATH  Google Scholar 

  9. Fergus R, Weiss Y, Torralba A (2009) Semi-supervised learning in gigantic image collections. Adv Neural Inf Process Syst 22:522–530

    Google Scholar 

  10. Guillaumin M, Verbeek J, Schmid C (2010) Multimodal semi-supervised learning for image classification. In: IEEE conference on computer vision and pattern recognition, pp 902–909

  11. Hofmann T (ed) (1999) Probabilistic latent semantic analysis. In: Proceedings of uncertainty in artificial intelligence. UAI99, Stockholm

  12. Huiskes MJ, Lew MS (2008) The mir flickr retrieval evaluation. In: Proceedings of the 2008 ACM MIR ’08: ACM, New York, NY, USA

  13. Jégou H, Chum O Negative evidences and co-occurrences in image retrieval: the benefit of PCA and whitening. http://hal.inria.fr/hal-00722622

  14. Jégou H, Douze M, Schmid C, Pérez P (2010) Aggregating local descriptors into a compact image representation. In: IEEE conference on, CVPR, pp 3304–3311

  15. Ji M, Yang T, Lin B, Jin R, Han J (2012) A simple algorithm for semi-supervised learning with improved generalization error bound. arXiv:1206.6412

  16. Jia P, Yin J, Huang X, Hu D (2009) Incremental laplacian eigenmaps by preserving adjacent information between data points. Pattern Recognit Lett 30(16):1457–1463

    Article  Google Scholar 

  17. Kong T, Tian Y, Shen H (2011) A fast incremental spectral clustering for large data sets. In: Parallel and distributed computing, applications and technologies (PDCAT), 2011 12th international conference on, pp 1–5, IEEE

  18. Kouropteva O, Okun O, Pietikäinen M (2005) Incremental locally linear embedding. Pattern Recognit 38(10):1764–1767

    Article  MATH  Google Scholar 

  19. Law MH, Jain AK (2006) Incremental nonlinear dimensionality reduction by manifold learning. Pattern Anal Mach Intell IEEE Trans 28(3):377–391

    Article  Google Scholar 

  20. Liu W, He J, Chang SF (2010) Large graph construction for scalable semi-supervised learning. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 679–686. Omnipress, Haifa, Israel

  21. Liu X, Yin J, Feng Z, Dong J (2006) Incremental manifold learning via tangent space alignment. In: Artificial neural networks in pattern recognition. Springer, pp 107–121

  22. Manning CD, Raghavan P, Schütze H (2008) Introduction to information retrieval. Cambridge University Press, New York

    Book  MATH  Google Scholar 

  23. Mantziou E, Papadopoulos S, Kompatsiaris I (2013) Large-scale semi-supervised learning by approximate laplacian eigenmaps, VLAD and pyramids. In WIAMIS

  24. Mantziou E, Papadopoulos S, Kompatsiaris Y (2013) Scalable training with approximate incremental laplacian eigenmaps and pca. In: Proceedings of the 21st ACM international conference on multimedia, MM ’13ACM, New York, NY, USA, pp 381–384

  25. Nadler B, Lafon S, Coifman RR, Kevrekidis IG (2006) Diffusion maps, spectral clustering and reaction coordinates of dynamical systems. Appl Comput Harmon Anal 21(1):113–127

  26. Ning H, Xu W, Chi Y, Gong Y, Huang TS (2007) Incremental spectral clustering with application to monitoring of evolving blog communities. In: SDM, pp 261–272

  27. Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42:145–175

    Article  MATH  Google Scholar 

  28. Papadopoulos S, Sagonas C, Kompatsiaris I, Vakali A (2013) Semi-supervised concept detection by learning the structure of similarity graphs. In: 19th international conference on MMM

  29. Perronnin F, Sánchez J, Liu Y (2010) Large-scale image categorization with explicit data embedding. In: CVPR, pp 2297–2304

  30. van de Sande KEA, Gevers T, Snoek CGM (2011) Empowering visual categorization with the gpu. IEEE Trans Multimedia 13(1):60–70

    Article  Google Scholar 

  31. Saul LK, Roweis ST (2003) Think globally, fit locally: unsupervised learning of low dimensional manifolds. J Mach Learn Res 4:119–155

    MathSciNet  Google Scholar 

  32. Sinha K, Belkin M (2009) Semi-supervised learning using sparse eigenfunction bases. In: Bengio Y, Schuurmans D, Lafferty J, Williams CKI, Culotta A (eds) Advances in Neural Information Processing Systems 22, pp 1687–1695

  33. Spyromitros-Xioufis E, Papadopoulos S, Kompatsiaris I, Tsoumakas G, Vlahavas I (2012) An empirical study on the combination of surf features with vlad vectors for image search. In: 13th international workshop on image analysis for multimedia interactive services (WIAMIS), pp 1–4, IEEE

  34. Talwalkar A, Kumar S, Rowley H (2008) Large-scale manifold learning. In: IEEE CVPR, 2008, pp 1–8

  35. Tang J, Yan S, Hong R, Qi GJ, Chua TS (2009) Inferring semantic concepts from community-contributed images and noisy tags. In: Proceedings of the 17th ACM international conference on multimedia, MM ’09ACM, New York, NY, USA, pp 223–232

  36. Wang H, Huang H, Ding CHQ (2011) Image annotation using bi-relational graph of images and semantic labels. In: CVPR, pp 793–800, IEEE

  37. Wang M, Hua XS (2009) Beyond distance measurement: constructing neighborhood similarity for video annotation. IEEE Trans Multimed 11(3):465–476

    Article  MathSciNet  Google Scholar 

  38. Zhang K, Kwok JT (2010) Clustered nyström method for large scale manifold learning and dimension reduction. IEEE Trans Neural Netw 21(10):1576–1587

    Article  Google Scholar 

  39. Zhang K, Kwok JT, Parvin B (2009) Prototype vector machine for large scale semi-supervised learning. In: Proceedings of the 26th annual international conference on machine learning, ICML ’09ACM, New York, NY, USA, pp 1233–1240

  40. Zhang Z, Zha H (2004) Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. SIAM J Sci Comput 26(1):313–338

    Article  MATH  MathSciNet  Google Scholar 

  41. Zheng J, Yu H, Shen F, Zhao J (2010) An online incremental learning support vector machine for large-scale data. In: International conference on artificial neural networks ICANN, lecture notes in computer science. Springer

  42. Zhou D, Bousquet O, Lal TN, Weston J, Schlkopf B (2004) Learning with local and global consistency. In: Advances in Neural Information Processing Systems 16:321–328, MIT Press

  43. Zhu X (2008) Semi-supervised learning literature survey. Technical report TR-1530, Computer Sciences, University of Wisconsin-Madison

  44. Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: IN ICML, pp 912–919

  45. Zhu X, Kandola J, Laerty J, Ghahramani Z (2006) Graph Kernels by spectral transforms. MIT Press, pp 1–17. http://pages.cs.wisc.edu/~jerryzhu/pub/ssl-book.pdf

Download references

Author information

Authors and Affiliations

  1. Information Technologies Institute (ITI), Centre for Research and Technology Hellas (CERTH), 57001, Thessaloníki, Greece

    Eleni Mantziou, Symeon Papadopoulos & Yiannis Kompatsiaris

Authors
  1. Eleni Mantziou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Symeon Papadopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yiannis Kompatsiaris
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Symeon Papadopoulos.

Additional information

This work was supported by the SocialSensor and REVEAL Projects, partially funded by the European Commission, under the contract numbers FP7-287975 and FP7-610928 respectively.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mantziou, E., Papadopoulos, S. & Kompatsiaris, Y. Learning to detect concepts with Approximate Laplacian Eigenmaps in large-scale and online settings. Int J Multimed Info Retr 4, 95–111 (2015). https://doi.org/10.1007/s13735-015-0079-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13735-015-0079-y

Keywords

Search

Navigation