Abstract
Content-based multimedia information retrieval (IR) provides new models and methods for effectively and efficiently “searching” through the huge variety of media that are available in different kinds of repositories (digital libraries, Web portals, social networks, multimedia databases, etc.). In this chapter, we will review the current state of the art of content-based multimedia information retrieval, including the most promising browsing and search paradigms for the several types of multimedia data, and show some cultural heritage applications.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Albanese, M., d’Acierno, A., Moscato, V., Persia, F., Picariello, A.: A multimedia semantic recommender system for cultural heritage applications. In: 2011 Fifth IEEE International Conference on Semantic Computing (ICSC), pp. 403–410. IEEE, Palo Alto (2011)
Amato, F., Mazzeo, A., Penta, A., Picariello, A.: Building rdf ontologies from semi-structured legal documents. In: CISIS, pp. 997–1002 (2008)
Amato, F., Chianese, A., Moscato, V., Picariello, A., Sperli, G.: Snops: a smart environment for cultural heritage applications. In: Proceedings of the Twelfth International Workshop on Web Information and Data Management, pp. 49–56. ACM, Maui (2012)
Amato, F., Mazzeo, A., Moscato, V., Picariello, A.: Building and retrieval of 3d objects in cultural heritage domain. In: 2012 Sixth International Conference on Complex, Intelligent and Software Intensive Systems (CISIS), pp. 816–821. IEEE, Palermo (2012)
Amato, F., Chianese, A., Mazzeo, A., Moscato, V., Picariello, A., Piccialli, F.: The talking museum project. Proc. Comput. Sci. 21, 114–121 (2013)
Ardizzone, E., La Cascia, M.: Automatic video database indexing and retrieval. In: Zhang, H., Aigrain, P., Petkovic, D. (eds.) Representation and Retrieval of Video Data in Multimedia Systems, pp. 29–56. Springer, New York (1997). doi:10.1007/978-0-585-31786-13. http://dx.doi.org/10.1007/978-0-585-31786-13
Arman, F., Hsu, A., Chiu, M.Y.: Image processing on encoded video sequences. Multimedia Syst. 1(5), 211–219 (1994). doi:10.1007/BF01268945. http://dx.doi.org/10.1007/BF01268945
Bach, J.R., Fuller, C., Gupta, A., Hampapur, A., Horowitz, B., Humphrey, R., Jain, R.C., Shu, C.F.: Virage image search engine: an open framework for image management. In: Storage and Retrieval for Image and Video Databases (1996). doi:10.1117/12.234785. http://dx.doi.org/10.1117/12.234785
Bahrami, A., Jun, Y.: Methods and systems for context based query formulation and information retrieval. U.S. Patent No. 7,970,786. U.S. Patent and Trademark Office, Washington, DC, June 2011
Barrington, L., Yazdani, M., Turnbull, D., Lanckriet, G.R.: Combining feature kernels for semantic music retrieval. In: ISMIR, pp. 614–619 (2008)
Bolasco, S.: Statistica testuale e text mining: alcuni paradigmi applicativi. Quaderni di Statistica 7, 17–53 (2005)
Bordogna, G., Pasi, G.: A fuzzy linguistic approach generalizing boolean information retrieval: a model and its evaluation. J. Am. Soc. Inf. Sci. 44(2), 70–82 (1993)
Bullock, J.: libxtract: a lightweight library for audio feature extraction. In: Proceedings of the 2007 International Computer Music Conference, vol. 2, pp. 25–28. ICMA, Copenhagen (2007)
Carpineto, C., Romano, G.: A survey of automatic query expansion in information retrieval. ACM Comput. Surv. (CSUR) 44(1), 1 (2012)
Chang, S.K., Liu, S.H.: Picture indexing and abstraction techniques for pictorial databases. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-6(4), 475–484 (1984). doi:10.1109/TPAMI.1984.4767552
Chang, S., Chen, W., Meng, H.J., Sundaram, H., Zhong, D.: Videoq: an automated content based video search system using visual cues. In: Proceedings of ACM Multimedia, pp. 313–324 (1997)
Chen, J.Y., Taskiran, C., Delp, E., Bouman, C.: Vibe: a new paradigm for video database browsing and search. In: Proceedings of IEEE Workshop on Content-Based Access of Image and Video Libraries, 1998, pp. 96–100 (1998). doi:10.1109/IVL.1998.694510
Chung, Y.Y., Choi, E.H.C., Liu, L., Shukran, M.A.M., Shi, D.Y., Chen, F.: A new hybrid audio classification algorithm based on svm weight factor and euclidean distance. In: Proceedings of the 2007 Annual Conference on International Conference on Computer Engineering and Applications, CEA’07, pp. 152–157. World Scientific and Engineering Academy and Society, Stevens Point (2007). http://dl.acm.org/citation.cfm?id=1348258.1348286
Clarizia, F., Colace, F., De Santo, M., Greco, L., Napoletano, P.: Mixed graph of terms for query expansion. In: 11th International Conference on Intelligent Systems Design and Applications (ISDA), 2011, pp. 581–586 (2011). doi:10.1109/ISDA.2011.6121718
Colace, F., Santo, M.D., Greco, L.: An adaptive product configurator based on slow intelligence approach. Int. J. Metadata Semant. Ontol. 9(2), 128–137 (2014). doi:10.1504/IJMSO.2014.060340. http://dx.doi.org/10.1504/IJMSO.2014.060340
Colace, F., Santo, M.D., Greco, L., Napoletano, P.: Text classification using a few labeled examples. Comput. Hum. Behav. (2014). doi:10.1016/j.chb.2013.07.043. http://www.sciencedirect.com/science/article/pii/S0747563213002823
Cummins, R.: A standard document score for information retrieval. In: Proceedings of the 2013 Conference on the Theory of Information Retrieval, p. 24. ACM, New York (2013)
D’Agostino, E., Elia, A., Vietri, S.: Lexicon-grammar, electronic dictionaries and local grammars of Italian. In: Leclère, Ch., Laporte, É., Piot, M., Silberztein, M. (eds.) Lexique, Syntaxe et Lexique-Grammaire. Papers in Honour of Maurice Gross, Lingvisticae Investigationes Supplementa, vol. 24. IGML, Amsterdam (2004)
Deliege, F., Chua, B.Y., Pedersen, T.B.: High-level audio features: distributed extraction and similarity search. In: Ninth International Conference on Music Information Retrieval, Philadelphia, pp. 565–570, September 2008
Downie, J.S.: Music information retrieval. Annu. Rev. Inf. Sci. Technol. 37(1), 295–340 (2003). doi:10.1002/aris.1440370108. http://dx.doi.org/10.1002/aris.1440370108
Eakins, J., Graham, M.: Content-based image retrieval, Technical Report (1999)
Edmunds, A., Morris, A.: The problem of information overload in business organisations: a review of the literature. Int. J. Inf. Manag. 20(1), 17–28 (2000)
Egozi, O., Markovitch, S., Gabrilovich, E.: Concept-based information retrieval using explicit semantic analysis. ACM Trans. Inf. Syst. (TOIS) 29(2), 8 (2011)
Elia, A., Vietri, S., Postiglione, A., Monteleone, M., Marano, F.: Data mining modular software system. In: SWWS, pp. 127–133 (2010)
Elia, A., Guglielmo, D., Maisto, A., Pelosi, S.: A linguistic-based method for automatically extracting spatial relations from large non-structured data. In: Algorithms and Architectures for Parallel Processing, pp. 193–200. Springer, Heidelberg (2013)
Fablet, R., Bouthemy, P., Pérez, P.: Statistical motion-based video indexing and retrieval. In: International Conference on Content-Based Multimedia Information Access, pp. 602–619 (2000)
Faloutsos, C., Oard, D.W.: A survey of information retrieval and filtering methods. Technical Report, CS-TR-3514, University of Maryland (1995)
Faloutsos, C., Barber, R., Flickner, M., Hafner, J., Niblack, W., Petkovic, D., Equitz, W.: Efficient and effective querying by image content. J. Intell. Inf. Syst. 3(3-4), 231–262 (1994). doi:10.1007/BF00962238. http://dx.doi.org/10.1007/BF00962238
Fan, J., Elmagarmid, A.K., Zhu, X., Aref, W.G., Wu, L.: Classview: hierarchical video shot classification, indexing, and accessing. Trans. Multimedia 6(1), 70–86 (2004). doi:10.1109/TMM.2003.819583. http://dx.doi.org/10.1109/TMM.2003.819583
Fernández, M., Cantador, I., López, V., Vallet, D., Castells, P., Motta, E.: Semantically enhanced information retrieval: an ontology-based approach. Web Semant. Sci. Serv. Agents World Wide Web 9(4), 434–452 (2011)
Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., Yanker, P.: Query by image and video content: the qbic system. Computer 28(9), 23–32 (1995). doi:10.1109/2.410146
Gabrilovich, E., Markovitch, S.: Computing semantic relatedness using wikipedia-based explicit semantic analysis. In: IJCAI, vol. 7, pp. 1606–1611 (2007)
Gantz, J., Reinsel, D.: Extracting value from chaos. IDC iview, pp. 1–12 (2011)
Grossman, D.A.: Information Retrieval: Algorithms and Heuristics, vol. 15. Springer, New York (2004)
Guo, G., Li, S.Z.: Content-based audio classification and retrieval by support vector machines. IEEE Trans. Neural Netw. 14(1), 209–215 (2003)
Gupta, A., Jain, R.: Visual information retrieval. Commun. ACM 40(5), 70–79 (1997). doi:10.1145/253769.253798. http://doi.acm.org/10.1145/253769.253798
Han, S., Yoon, K., Kweon, I.: A new technique for shot detection and key frames selection in histogram space. In: 12th Workshop on Image Processing and Image Understanding, pp. 475–479 (2000)
Harris, Z.S.: Linguistic transformations for information retrieval. In: Papers in Structural and Transformational Linguistics, pp. 458–471. Springer, Dordrecht (1970)
Hliaoutakis, A., Varelas, G., Voutsakis, E., Petrakis, E.G., Milios, E.: Information retrieval by semantic similarity. Int. J. Semant. Web Inf. Syst. 2(3), 55–73 (2006)
Hopcroft, J.E.: Introduction to Automata Theory, Languages, and Computation. Pearson Education India, New Delhi (1979)
Hossein, M.: Automatic audio classification using modified two dimensional root cepstral features. In: The International Conference on Electrical Engineering (2008)
Ishikawa, Y., Subramanya, R., Faloutsos, C.: Mindreader: querying databases through multiple examples. Computer Science Department, p. 551 (1998)
Jain, R.: Dynamic vision. In: 9th International Conference on Pattern Recognition, 1988, vol. 1, pp. 226–235 (1988). doi:10.1109/ICPR.1988.28212
Kara, S., Alan, Ö., Sabuncu, O., Akpınar, S., Cicekli, N.K., Alpaslan, F.N.: An ontology-based retrieval system using semantic indexing. Inf. Syst. 37(4), 294–305 (2012)
Klapuri, A., Davy, M. (eds.): Signal Processing Methods for Music Transcription. Springer, New York (2006)
Kurth, F., Gehrmann, T., Müller, M.: The cyclic beat spectrum: tempo-related audio features for time-scale invariant audio identification. In: Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR), Victoria, pp. 35–40, 8–12 October 2006
Landauer, T.K., Dumais, S.T.: A solution to plato’s problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev. 104(2), 211 (1997)
Landauer, R., Swanson, J.: Frequency factors in the thermally activated process. Phys. Rev. 121(6), 1668 (1961)
Lartillot, O., Toiviainen, P.: A matlab toolbox for musical feature extraction from audio. In: International Conference on Digital Audio Effects, pp. 237–244 (2007)
Lew, M.S., Sebe, N., Djeraba, C., Jain, R.: Content-based multimedia information retrieval: state of the art and challenges. ACM Trans. Multimedia Comput. Commun. Appl. 2(1), 1–19 (2006). doi:10.1145/1126004.1126005. http://doi.acm.org/10.1145/1126004.1126005
Liu, M., Wan, C., Wang, L.: Content-based audio classification and retrieval using a fuzzy logic system: towards multimedia search engines. Soft Comput. 6(5), 357–364 (2002). doi:10.1007/s00500-002-0189-3. http://dx.doi.org/10.1007/s00500-002-0189-3
Ma, W., Manjunath, B.: Netra: a toolbox for navigating large image databases. In: Proceedings of International Conference on Image Processing, 1997, vol. 1, pp. 568–571 (1997). doi:10.1109/ICIP.1997.647976
Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval, vol. 1. Cambridge University Press, Cambridge (2008)
Massoudi, K., Tsagkias, M., de Rijke, M., Weerkamp, W.: Incorporating query expansion and quality indicators in searching microblog posts. In: Advances in Information Retrieval, pp. 362–367. Springer, Heidelberg (2011)
Mckinney, M., Breebaart, J.: Features for audio and music classification. In: Proceedings of the International Symposium on Music Information Retrieval, pp. 151–158 (2003)
Mehrotra, S., Rui, Y., Ortega-Binderberger, M., Huang, T.: Supporting content-based queries over images in mars. In: Proceedings of IEEE International Conference on Multimedia Computing and Systems ’97, pp. 632–633 (1997). doi:10.1109/MMCS.1997.609791
Mitrovic, D., Zeppelzauer, M., Breiteneder, C.: Features for content-based audio retrieval. Adv. Comput. 78, 71–150 (2010). http://dblp.uni-trier.de/db/journals/ac/ac78.htmlMitrovicZB10
Mojsilovic, A., Rogowitz, B.: Capturing image semantics with low-level descriptors. In: Proceedings of 2001 International Conference on Image Processing, vol. 1, pp. 18–21 (2001). doi:10.1109/ICIP.2001.958942
Ogle, V.E., Stonebraker, M.: Chabot: retrieval from a relational database of images. Computer 28(9), 40–48 (1995)
Ohishi, Y., Goto, M., Itou, K., Takeda, K.: A stochastic representation of the dynamics of sung melody. In: Dixon, S., Bainbridge, D., Typke, R. (eds.) ISMIR, pp. 371–372. Austrian Computer Society, Vienna (2007). http://dblp.uni-trier.de/db/conf/ismir/ismir2007.htmlOhishiGIT07
Papathomas, T., Conway, T., Cox, I., Ghosn, J., Miller, M., Minka, T., Yianilos, P.: Psychophysical studies of the performance of an image database retrieval system. In: IS&T/SPIE Symposium on Electronic Imaging: Science and Technology, Conference on Human Vision and Electronic Imaging III, pp. 591–602 (1998)
Patel, A.J.: Systems and methods for highlighting search results. U.S. Patent No. 6,839,702, 4 Jan 2005
Pentland, A., Picard, R., Sclaroff, S.: Photobook: content-based manipulation of image databases. Int. J. Comput. Vis. 18(3), 233–254 (1996). doi:10.1007/BF00123143. http://dx.doi.org/10.1007/BF00123143
Picard, R., Minka, T.: Vision texture for annotation. Multimedia Syst. 3(1), 3–14 (1995). doi:10.1007/BF01236575. http://dx.doi.org/10.1007/BF01236575
Pickard, A.J.: Research Methods in Information. Facet, London (2013)
Pinquier, J., Arias, J., André-Obrecht, R.: Audio classification by search of primary components. In: International Workshop on Image, Video and Audio Retrieval and Mining, Sherbrooke (2004)
Pohle, T., Knees, P., Seyerlehner, K., Widmer, G.: A high-level audio feature for music retrieval and sorting. In: DAFx-10 (2010)
Quinlan, J.R.: Combining instance-based and model-based learning. In: ICML, pp. 236–243 (1993)
Rabiner, L., Juang, B.H.: Fundamentals of Speech Recognition. Prentice-Hall, Upper Saddle River (1993)
Ranchhod, E.: Lexique-grammaire du portugais: prédicats nominaux supportés par estar. Lingvisticae Investigationes 13(2), 351–367 (1989)
Reed, J.: A study on music genre classification based on universal acoustic models. In: ISMIR 2006, pp. 89–94 (2006)
Roa-Valverde, A.J., Sicilia, M.A.: A survey of approaches for ranking on the web of data. Inf. Retr. 17, 295–325 (2014)
Robertson, S.E., Jones, K.S.: Relevance weighting of search terms. J. Am. Soc. Inf. Sci. 27(3), 129–146 (1976)
Rogowitz, B.E., Frese, T., Smith, J., Bouman, C.A., Kalin, E.: Perceptual image similarity experiments. In: Proceedings of the SPIE Human Vision and Electronic Imaging III, vol. 3299 (1998)
Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Inf. Process. Manag. 24(5), 513–523 (1988)
Salton, G., Wong, A., Yang, C.S.: A vector space model for automatic indexing. Commun. ACM 18(11), 613–620 (1975)
Schnitzer, D.: High-performance music similarity computation and automatic playlist generation. Technical Report, University of Technology, Vienna (2007)
Scholtes, J.C.: Unsupervised learning and the information retrieval problem. In: 1991 IEEE International Joint Conference on Neural Networks, pp. 95–100. IEEE, Seattle (1991)
Sethi, I.K., Coman, I.L., Stan, D.: Mining association rules between low-level image features and high-level concepts (2001). doi:10.1117/12.421083. http://dx.doi.org/10.1117/12.421083
Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1349–1380 (2000)
Smith, J.R., Chang, S.F.: Visualseek: a fully automated content-based image query system. In: Proceedings of the Fourth ACM International Conference on Multimedia, MULTIMEDIA ’96, pp. 87–98. ACM, New York (1996). doi: 10.1145/244130.244151. http://doi.acm.org/10.1145/244130.244151
Tamine-Lechani, L., Boughanem, M., Daoud, M.: Evaluation of contextual information retrieval effectiveness: overview of issues and research. Knowl. Inf. Syst. 24(1), 1–34 (2010)
Tzanetakis, G., Ermolinskyi, A., Cook, P.: Pitch histograms in audio and symbolic music information retrieval. In: Proceedings of the Third International Conference on Music Information Retrieval: ISMIR, pp. 31–38 (2002)
Wang, J., Li, J., Wiederhold, G.: Simplicity: semantics-sensitive integrated matching for picture libraries. IEEE Trans. Pattern Anal. Mach. Intell. 23(9), 947–963 (2001). doi:10.1109/34.955109
Wang, A., et al.: An industrial strength audio search algorithm. In: ISMIR, pp. 7–13 (2003)
Wang, X.H., Zhang, D.Q., Gu, T., Pung, H.K.: Ontology based context modeling and reasoning using owl. In: Proceedings of the Second IEEE Annual Conference on Pervasive Computing and Communications Workshops, pp. 18–22. IEEE, Orlando (2004)
Wendlandt, E.B., Driscoll, J.R.: Incorporating a semantic analysis into a document retrieval strategy. In: Proceedings of the 14th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 270–279. ACM, New York (1991)
Wold, E., Blum, T., Keislar, D., Wheaten, J.: Content-based classification, search, and retrieval of audio. IEEE MultiMedia 3(3), 27–36 (1996). doi:10.1109/93.556537
Zabih, R., Miller, J., Mai, K.: A feature-based algorithm for detecting and classifying scene breaks. In: Proceedings of the Third ACM International Conference on Multimedia, MULTIMEDIA ’95, pp. 189–200. ACM, New York (1995). doi:10.1145/217279.215266. http://doi.acm.org/10.1145/217279.215266
Zhou, X.S., Huang, T.S.: Cbir: from low-level features to high-level semantics. In: Proceedings of SPIE Image and Video Communication and Processing (2000). doi:10.1117/12.382975. http://dx.doi.org/10.1117/12.382975
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Amato, F., Greco, L., Persia, F., Poccia, S.R., De Santo, A. (2015). Content-Based Multimedia Retrieval. In: Colace, F., De Santo, M., Moscato, V., Picariello, A., Schreiber, F., Tanca, L. (eds) Data Management in Pervasive Systems. Data-Centric Systems and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-20062-0_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-20062-0_14
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-20061-3
Online ISBN: 978-3-319-20062-0
eBook Packages: Computer ScienceComputer Science (R0)