Integration of Exploration and Search: A Case Study of the M\(^3\) Model

  • Snorri Gíslason
  • Björn Þór JónssonEmail author
  • Laurent Amsaleg
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11295)


Effective support for multimedia analytics applications requires exploration and search to be integrated seamlessly into a single interaction model. Media metadata can be seen as defining a multidimensional media space, casting multimedia analytics tasks as exploration, manipulation and augmentation of that space. We present an initial case study of integrating exploration and search within this multidimensional media space. We extend the M\(^3\) model, initially proposed as a pure exploration tool, and show that it can be elegantly extended to allow searching within an exploration context and exploring within a search context. We then evaluate the suitability of relational database management systems, as representatives of today’s data management technologies, for implementing the extended M\(^3\) model. Based on our results, we finally propose some research directions for scalability of multimedia analytics.


Multimedia analytics Exploration-search axis Scalability 


  1. 1.
    Arigon, A.M., Miquel, M., Tchounikine, A.: Multimedia data warehouses: a multiversion model and a medical application. Multi. Tools Apps. 35(1), 91–108 (2007)CrossRefGoogle Scholar
  2. 2.
    Babenko, A., Lempitsky, V.S.: The inverted multi-index. IEEE Trans. Pattern Anal. Mach. Intell. 37(6), 1247–1260 (2015)CrossRefGoogle Scholar
  3. 3.
    Bartolini, I., Ciaccia, P.: Integrating semantic and visual facets for browsing digital photo collections. In: Proceedings of SEBD (2009)Google Scholar
  4. 4.
    Diao, M., Mukherjea, S., Rajput, N., Srivastava, K.: Faceted search and browsing of audio content on spoken web. In: Proceedings of CIKM (2010)Google Scholar
  5. 5.
    Guðmundsson, G.Þ., Amsaleg, L., Jónsson, B.Þ., Franklin, M.J.: Towards engineering a web-scale multimedia service: a case study using Spark. In: Proceedings of MMSys, Taipei, Taiwan (2017)Google Scholar
  6. 6.
    Jégou, H., Tavenard, R., Douze, M., Amsaleg, L.: Searching in one billion vectors: re-rank with source coding. In: Proceedings of ICASSP, Prague, Czech Republic (2011)Google Scholar
  7. 7.
    Jin, X., Han, J., Cao, L., Luo, J., Ding, B., Lin, C.X.: Visual cube and on-line analytical processing of images. In: Proceedings of CIKM (2010)Google Scholar
  8. 8.
    Amsaleg, L., Jónsson, B.Þ., Lejsek, H.: Scalability of the NV-tree: three experiments. In: Marchand-Maillet, S., Silva, Y.N., Chávez, E. (eds.) SISAP 2018. LNCS, vol. 11223, pp. 59–72. Springer, Cham (2018). Scholar
  9. 9.
    Jónsson, B.Þ., Tómasson, G., Sigurþórsson, H., Eiríksdóttir, Á., Amsaleg, L., Lárusdóttir, M.K.: A multi-dimensional data model for personal photo browsing. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8936, pp. 345–356. Springer, Cham (2015). Scholar
  10. 10.
    Shneiderman, B., Bederson, B.B., Drucker, S.M.: Find that photo! interface strategies to annotate, browse, and share. Commun. ACM 49(4), 69–71 (2006)CrossRefGoogle Scholar
  11. 11.
    Worring, M., Koelma, D.C.: Insight in image collections by multimedia pivot tables. In: Proceedings of ACM ICMR, Shanghai, China (2015)Google Scholar
  12. 12.
    Yee, K.P., Swearingen, K., Li, K., Hearst, M.: Faceted metadata for image search and browsing. In: Proceedings of CHI (2003)Google Scholar
  13. 13.
    Zahálka, J., Rudinac, S., Jónsson, B.Þ., Koelma, D.C., Worring, M., : Blackthorn: large-scale interactive multimodal learning. IEEE Trans. Multi. 20(3), 687–698 (2018)CrossRefGoogle Scholar
  14. 14.
    Zahálka, J., Worring, M.: Towards interactive, intelligent, and integrated multimedia analytics. In: Proceedings of IEEE VAST, Paris, France (2014)Google Scholar
  15. 15.
    Zaïane, O.R., Han, J., Li, Z.N., Hou, J.: Mining multimedia data. In: Proceedings of CASCON (1998)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Snorri Gíslason
    • 1
  • Björn Þór Jónsson
    • 1
    Email author
  • Laurent Amsaleg
    • 2
  1. 1.IT University of CopenhagenCopenhagenDenmark
  2. 2.CNRS-IRISARennesFrance

Personalised recommendations