Advertisement

I-SEARCH: A Unified Framework for Multimodal Search and Retrieval

  • Apostolos Axenopoulos
  • Petros Daras
  • Sotiris Malassiotis
  • Vincenzo Croce
  • Marilena Lazzaro
  • Jonas Etzold
  • Paul Grimm
  • Alberto Massari
  • Antonio Camurri
  • Thomas Steiner
  • Dimitrios Tzovaras
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7281)

Abstract

In this article, a unified framework for multimodal search and retrieval is introduced. The framework is an outcome of the research that took place within the I-SEARCH European Project. The proposed system covers all aspects of a search and retrieval process, namely low-level descriptor extraction, indexing, query formulation, retrieval and visualisation of the search results. All I-SEARCH components advance the state of the art in the corresponding scientific fields. The I-SEARCH multimodal search engine is dynamically adapted to end-user’s devices, which can vary from a simple mobile phone to a high-performance PC.

Keywords

multimodal search multimodal interfaces adaptive presentation 

References

  1. 1.
    Daras, P., Axenopoulos, A., Darlagiannis, V., Tzovaras, D., Le Bourdon, X., Joyeux, L., Verroust-Blondet, A., Croce, V., Steiner, T., Massari, A., Camurri, A., Morin, S., Mezaour, A.D., Sutton, L., Spiller, S.: Introducing a Unified Framework for Content Object Description. International Journal of Multimedia Intelligence and Security, Special Issue on “Challenges in Scalable Context Aware Multimedia Computing 2(3-4), 351–375 (2011), doi:10.1504/IJMIS.2011.044765Google Scholar
  2. 2.
    Daras, P., Alvarez, F.: A Future Perspective on the 3D Media Internet. Towards the Future Internet - A European Research Perspective (January 2009) ISBN: 978-1-60750-007-0Google Scholar
  3. 3.
  4. 4.
    Etzold, J., Brousseau, A., Grimm, P., Steiner, T.: Context-Aware Querying for Multimodal Search Engines. In: Schoeffmann, K., Merialdo, B., Hauptmann, A.G., Ngo, C.-W., Andreopoulos, Y., Breiteneder, C. (eds.) MMM 2012. LNCS, vol. 7131, pp. 728–739. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  5. 5.
    Steiner, T., Sutton, L., Spiller, S., et al.: I-SEARCH – A Multimodal Search Engine based on Rich Unified Content Description (RUCoD). Submitted to the European Projects Track at the 21st International World Wide Web Conference. Under review, http://www.lsi.upc.edu/~tsteiner/papers/2012/isearch-multimodal-search-www2012.pdf
  6. 6.
    Zagoris, K., Arampatzis, A., Chatzichristofis, S.A.: www.MMRetrieval.Net: a multimodal search engine. In: Proceedings of the Third International Conference on SImilarity Search and APplications (SISAP 2010), pp. 117–118. ACM, New York (2010)CrossRefGoogle Scholar
  7. 7.
    Bederson, B.B.: PhotoMesa: A Zoomable Image Browser Using Quantum Treemaps and Bubblemaps.. In: ACM Symposium on User Interface Software and Technology, UIST 2001, CHI Letters, vol. 3(2), pp. 71–80 (2001)Google Scholar
  8. 8.
    Graham, A., Garcia-Molina, H., Paepcke, A., Winograd, T.: Time as essence for photo browsing through personal digital libraries. In: Proceedings of the 2nd ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL 2002), pp. 326–335. ACM, NY (2002)CrossRefGoogle Scholar
  9. 9.
    Platt, J.C., Czerwinski, M., Field, B.A.: PhotoTOC: automatic clustering for browsing personal photographs. In: Proceedings of the 2003 Joint Conference and the Information, Communications and Signal Processing Fourth Pacific Rim Conference on Multimedia, December 15-18, vol. 1, pp. 6–10 (2003)Google Scholar
  10. 10.
    Zhang, H., Weng, J.: Measuring Multi-modality Similarities Via Subspace Learning for Cross-Media Retrieval. In: Advances in Multimedia Information Processing, PCM (2006)Google Scholar
  11. 11.
    Yang, Y., Xu, D., Nie, F., Luo, J., Zhuang, Y.: Ranking with Local Regression and Global Alignment for Cross Media Retrieval. ACM MM, Beijing, China (2009)Google Scholar
  12. 12.
    Axenopoulos, A., Manolopoulou, S., Daras, P.: Multimodal Search and Retrieval using Manifold Learning and Query Formulation. In: ACM International Conference on 3D Web Technology, Paris,France, June 20-22 (2011)Google Scholar
  13. 13.
    Daras, P., Manolopoulou, S., Axenopoulos, A.: Search and Retrieval of Rich Media Objects Supporting Multiple Multimodal Queries. Accepted on IEEE Transactions on MultimediaGoogle Scholar
  14. 14.
    Axenopoulos, A., Manolopoulou, S., Daras, P.: Optimizing Multimedia Retrieval Using Multimodal Fusion and Relevance Feedback Techniques. In: Schoeffmann, K., Merialdo, B., Hauptmann, A.G., Ngo, C.-W., Andreopoulos, Y., Breiteneder, C. (eds.) MMM 2012. LNCS, vol. 7131, pp. 716–727. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  15. 15.
    Gennaro, C., Amato, G., Bolettieri, P., Savino, P.: An Approach to Content-Based Image Retrieval Based on the Lucene Search Engine Library. In: Lalmas, M., Jose, J., Rauber, A., Sebastiani, F., Frommholz, I. (eds.) ECDL 2010. LNCS, vol. 6273, pp. 55–66. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  16. 16.
    Glowinski, D., Dael, N., Camurri, A., Volpe, G., Mortillaro, M., Scherer, K.: Towards a Minimal Representation of Affective Gestures. IEEE Transactions on Affective Computing 2(2), 106–118 (2011)CrossRefGoogle Scholar
  17. 17.
    Varni, G., Volpe, G., Camurri, A.: A System for Real-Time Multimodal Analysis of Nonverbal Affective Social Interaction in User-Centric Media. IEEE Transactions on Multimedia 12(6), 576–590 (2010)CrossRefGoogle Scholar
  18. 18.
  19. 19.

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Apostolos Axenopoulos
    • 1
  • Petros Daras
    • 1
  • Sotiris Malassiotis
    • 1
  • Vincenzo Croce
    • 2
  • Marilena Lazzaro
    • 2
  • Jonas Etzold
    • 3
  • Paul Grimm
    • 3
  • Alberto Massari
    • 4
  • Antonio Camurri
    • 4
  • Thomas Steiner
    • 5
  • Dimitrios Tzovaras
    • 1
  1. 1.Centre for Research and Technology Hellas (CERTH/ITI)ThessalonikiGreece
  2. 2.Engineering Ingegneria Informatica S.p.a.PalermoItaly
  3. 3.Hochschule FuldaFuldaGermany
  4. 4.Casa Paganini – InfoMus Research CentreUniversity of GenoaGenoaItaly
  5. 5.Google Germany GmbHHamburgGermany

Personalised recommendations