Skip to main content
SpringerLink
Log in

Toward Optimized Multimodal Concept Indexing

  • Conference paper
  • First Online:
Semantic Keyword-Based Search on Structured Data Sources (IKC 2015)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9398))

  • 491 Accesses

Abstract

Information retrieval on the (social) web moves from a pure term-frequency-based approach to an enhanced method that includes conceptual multimodal features on a semantic level. In this paper, we present an approach for semantic-based keyword search and focus especially on its optimization to scale it to real-world sized collections in the social media domain. Furthermore, we present a faceted indexing framework and architecture that relates content to semantic concepts to be indexed and searched semantically. We study the use of textual concepts in a social media domain and observe a significant improvement from using a concept-based solution for keyword searching. We address the problem of time-complexity that is critical issue for concept-based methods by focusing on optimization to enable larger and more real-world style applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    http://en.wikipedia.org/wiki/List_of_social_networking_websites.

  2. 2.

    http://lucene.apache.org/core.

  3. 3.

    https://code.google.com/p/word2vec/.

  4. 4.

    https://code.google.com/p/semanticvectors/.

  5. 5.

    http://scikit-learn.org/stable/.

References

  1. Arya, S., Mount, D.M., Netanyahu, N.S., Silverman, R., Wu, A.Y.: An optimal algorithm for approximate nearest neighbor searching fixed dimensions. J. ACM (JACM) 45(6), 891–923 (1998)

    Article  MathSciNet  Google Scholar 

  2. Baroni, M., Dinu, G., Kruszewski, G.: Dont count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In Proc. of the 52nd Annual Meeting of the Association for. Comput. Linguist. 1, 238–247 (2014)

    Google Scholar 

  3. Clinchant, S., Ah-Pine, J., Csurka, G.: Semantic combination of textual and visual information in multimedia retrieval. In: Proceedings of the 1st ACM International Conference on Multimedia Retrieval (2011)

    Google Scholar 

  4. Csurka, G., Dance, C.R., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision (at ECCV) (2004)

    Google Scholar 

  5. Dang, V., Bendersky, M., Croft, W.: Two-stage learning to rank for information retrieval. In: Proceedings of European Conference on Information Retrieval (2013)

    Google Scholar 

  6. Deerwester, S.C., Dumais, S.T., Landauer, T.K., Furnas, G.W., Harshman, R.A.: Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. (JASIS) 41, 391–407 (1990)

    Article  Google Scholar 

  7. Depeursinge, A., Müller, H.: Fusion techniques for combining textual and visual information retrieval. In: Müller, H., Clough, P., Deselaers, T., Caputo, B. (eds.) ImageCLEF. The Information Retrieval Series, pp. 95–114. Springer, Berlin (2010)

    Chapter  Google Scholar 

  8. Eskevich, M., Jones, G.J., Aly, R., et al.: Multimedia information seeking through search and hyperlinking. In: Proceedings of the Annual ACM International Conference on Multimedia Retrieval (2013)

    Google Scholar 

  9. Ionescu, B., Popescu, A., Lupu, M., Gînsca, A.L., Boteanu, B., Müller, H.: Div150cred: a social image retrieval result diversification with user tagging credibility dataset. In: ACM Multimedia Systems Conference Series (2015)

    Google Scholar 

  10. Ionescu, B., Radu, A.-L., Menéndez, M., Müller, H., Popescu, A., Loni, B.: Div400: a social image retrieval result diversification dataset. In: Proceedings of ACM Multimedia Systems Conference Series (2014)

    Google Scholar 

  11. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the ACM International Conference on Multimedia, pp. 675–678. ACM (2014)

    Google Scholar 

  12. Liu, C., Wang, Y.-M.: On the connections between explicit semantic analysis and latent semantic analysis. In: Proceedings of Conference on Information and Knowledge Management, New York, NY, USA (2012)

    Google Scholar 

  13. Liu, N., Dellandréa, E., Chen, L., Zhu, C., Zhang, Y., Bichot, C.-E., Bres, S., Tellez, B.: Multimodal recognition of visual concepts using histograms of textual concepts and selective weighted late fusion scheme. Comput. Vis. Image Underst. 117, 493–512 (2013)

    Article  Google Scholar 

  14. Magalhaes, J., Rüger, S.: Information-theoretic semantic multimedia indexing. In: Proceedings of the 6th ACM International Conference on Image and Video Retrieval, pp. 619–626. ACM (2007)

    Google Scholar 

  15. Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H.: Big data: the next frontier for innovation, competition, and productivity. McKinsey Global Institute (2011)

    Google Scholar 

  16. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space (2013). arXiv preprint arXiv:1301.3781

  17. Paramita, M.L., Grubinger, M.: Photographic image retrieval. In: Müller, H., Clough, P., Deselaers, T., Caputo, B. (eds.) ImageCLEF: Experimental Evaluation in Visual Information Retrieval, pp. 141–162. Springer, Berlin (2010)

    Chapter  Google Scholar 

  18. Pham, T.-T., Maillot, N., Lim, J.-H., Chevallet, J.-P.: Latent semantic fusion model for image retrieval and annotation. In: Proceedings of Conference on Information and Knowledge Management (2007)

    Google Scholar 

  19. Rekabsaz, N., Bierig, R., Ionescu, B., Hanbury, A., Lupu, M.: On the use of statistical semantics for metadata-based social image retrieval. In: Proceedings of the 13th International Workshop on Content-Based Multimedia Indexing (CBMI) (2015)

    Google Scholar 

  20. Sabetghadam, S., Lupu, S., Bierig, R., Rauber, A.: A combined approach of structured and non-structured IR in multimodal domain. In: Proceedings of ACM International Conference on Multimedia Retrieval (2014)

    Google Scholar 

  21. Sahlgren, M.: An introduction to random indexing. In: Methods and Applications of Semantic Indexing Workshop in the Proceedings of Terminology and Knowledge Engineering (2005)

    Google Scholar 

  22. Thomee, B., Popescu, A.: Overview of the ImageCLEF 2012 flickr photo annotation and retrieval task. In: Proceedings of Cross-Language Evaluation Forum (CLEF) (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Information and Software Engineering Group, Vienna University of Technology, 1040, Vienna, Austria

    Navid Rekabsaz, Ralf Bierig, Mihai Lupu & Allan Hanbury

Authors
  1. Navid Rekabsaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ralf Bierig
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mihai Lupu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Allan Hanbury
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Navid Rekabsaz .

Editor information

Editors and Affiliations

  1. University of Coimbra, Coimbra, Portugal

    Jorge Cardoso

  2. Huawei European Research Center, Munich, Germany

    Jorge Cardoso

  3. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Francesco Guerra

  4. Delft University of Technology, Delft, Zuid-Holland, The Netherlands

    Geert-Jan Houben

  5. University of Coimbra, Coimbra, Portugal

    Alexandre Miguel Pinto

  6. Università degli Studi di Trento, Trento, Italy

    Yannis Velegrakis

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (http://creativecommons.org/licenses/by-nc/2.5/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Rekabsaz, N., Bierig, R., Lupu, M., Hanbury, A. (2015). Toward Optimized Multimodal Concept Indexing. In: Cardoso, J., Guerra, F., Houben, GJ., Pinto, A.M., Velegrakis, Y. (eds) Semantic Keyword-Based Search on Structured Data Sources. IKC 2015. Lecture Notes in Computer Science(), vol 9398. Springer, Cham. https://doi.org/10.1007/978-3-319-27932-9_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-27932-9_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-27931-2

  • Online ISBN: 978-3-319-27932-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation