Skip to main content
SpringerLink
Log in

Extended Bayesian generalization model for understanding user’s intention in semantics based images retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Learning concepts from examples presented in user’s query and infer the other items that belong to this query is still a significant challenge for images retrieval systems. Existing models from cognitive science namely Bayesian models of generalization mainly focus on this challenge where they remarkably succeed at explaining how to generalize from few examples in a wide range of domains. However their success largely depends on the validity of examples. They require that each example is a good representative, which is not always the case in the context of images retrieval. In this paper, we will extend the Bayesian models of generalization to identify the appropriate level of generalization for a given query in the context of query by semantic example systems. Our model uses an ontology as the basis of its hypothesis space which allows us to take advantages of its semantic richness and inference capacity. Experimental study using the ImageNet benchmark verifies the efficiency of our model in comparison to the state-of-the-art models of generalization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Abbott JT, Austerweil JL, Griffiths TL (2012) Constructing a hypothesis space from the Web for large-scale Bayesian word learning. In Proceedings of the 34th Annual Conference of the Cognitive Science Society

  2. Allani O, Zghal HB, Mellouli N, Akdag H (2016) A knowledge-based image retrieval system integrating semantic and visual features. Procedia Comput Sci 96(September):1428–1436

    Article  Google Scholar 

  3. Austerweil JL, Griffiths TL (2001) Learning hypothesis spaces and dimensions through concept learning. Citeseer, 73–78

  4. Austerweil JL, Griffiths TL (2011) Seeking confirmation is rational for deterministic hypotheses. Cogn Sci 35(3):499–526

    Article  Google Scholar 

  5. Barnard K, Forsyth D (2001) Learning the semantics of words and pictures. In ICCV, volume 2, pages 408–415, Vancouver

  6. Ben-Haim N, Babenko B, Belongie S (2006) Improving Web- Based Image Search via Content Based Clustering, Proc. Int’lWorkshop Semantic Learning Applications in Multimedia

  7. Blei D, Jordan M (2003) Modeling annotated data. In Proc. ACM SIGIR

  8. Carneiro G, Chan A, Moreno P, Vasconcelos N (2007) Supervised learning of semantic classes for image annotation and retrieval. IEEE PAMI 29(3):394–410

    Article  Google Scholar 

  9. Celik C, Bilge HS (2017) Content based image retrieval with sparse representations and local feature descriptors : a comparative study. Pattern Recogn 68:1–13

    Article  Google Scholar 

  10. Datta D, Varma S, Chowdary CR, Singh SK (2017) Multimodal retrieval using mutual information based textual query reformulation. Expert Syst Appl 68:81–92

    Article  Google Scholar 

  11. Deepa C (2017) SABC-SBC: a hybrid ontology based image and webpage retrieval for datasets. Autom Control Comput Sci 51(2):108–113

    Article  Google Scholar 

  12. Deng JDJ, Dong WDW, Socher R, Li L.-J. L. L.-J, Li K. L. K, and Fei-Fei L. F.-F. L (2009) ImageNet: A large-scale hierarchical image database, 2009 IEEE Conf. Comput. Vis. Pattern Recognit., pp. 2–9

  13. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) DeCAF: a deep convolutional activation feature for generic visual recognition. Icml 32:647–655

    Google Scholar 

  14. Faloutsos C, Barber R, Flickner M, Hafner J, Niblack W, Petkovic D, Equitz W (1994) Efficient and effective querying by image content. J Intell Inf Syst 3(3–4):231–262

    Article  Google Scholar 

  15. Filali J, Zghal HB, Martinet J (2016) Towards visual vocabulary and ontology-based image retrieval system. Proc 8th Int Conf Agents Artif Intell 2(Icaart):560–565

    Google Scholar 

  16. Gao L, Guo Z, Zhang H et al (2017) Video captioning with attention-based lstm and semantic consistency. IEEE Transactions on Multimedia 19(9):2045–2055

    Article  Google Scholar 

  17. Gao L, Song J, Liu X et al (2017) Learning in high-dimensional multimedia data: the state of the art. Multimedia Systems 23(3):303–313

    Article  Google Scholar 

  18. Ghahramani Z, Heller KA (2006) Bayesian Sets. Adv Neural Inf Proces Syst 18:435–442

    Google Scholar 

  19. Hannan MA, Arebey M, Begum RA, Basri H, Al Mamun MA (2016) Content-based image retrieval system for solid waste bin level detection and performance evaluation. Waste Manag 50:10–19

    Article  Google Scholar 

  20. Hartvedt C (2010) Using context to understand user intentions in image retrieval. 2nd Int Conf Adv Multimedia, MMEDIA 2010:130–133

    Article  Google Scholar 

  21. Heller KA (2008) Efficient Bayesian Methods for Clustering. Ph.D. thesis, University College London, Gatsby Computational Neuroscience Unit

  22. Heller KA, Ghahramani Z (2006) A simple Bayesian framework for content-based image retrieval. IEEE Conference on Computer Vision and Pattern Recognition

  23. Hinton G, Deng L, Yu D, Dahl GE, Mohamed A-r, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN et al (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. Signal Processing Magazine, IEEE 29(6):82–97

    Article  Google Scholar 

  24. Hsu WH, Kennedy LS, Chang S-F (2006) Video Search Reranking via Information Bottleneck Principle, Proc. 14th Ann. ACM Int’l Conf. Multimedia

  25. Jia Y, Abbott J, Austerweil JL, Griffiths TL, Darrell T (2013) Visual Concept Learning: Combining Machine Vision and Bayesian Generalization on Concept Hierarchies, Adv. Neural Inf. Process. Syst. 27 (NIPS 2013), vol. 1, no. 1, pp. 1–9

  26. Jing Y, Baluja S (2008) Pagerank for Product Image Search, Proc. Int’l Conf. World Wide Web

  27. Kherfi ML (2008). Review of Human-Computer Interaction Issues in Image Retrieval, Advances in Human Computer Interaction, Shane Pinder (Ed.), InTech, DOI: https://doi.org/10.5772/5929. Available from: http://www.intechopen.com/books/advances_in_human_computer_interaction/review_of_human-computer_interaction_issues_in_image_retrieval

  28. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In NIPS, pages 1106–1114

  29. Krizhevsky A, Sutskever I, Hinton, Geoffrey E. (2012) Imagenet classification with deep convolutional neural networks. In : Advances in neural information processing systems. p. 1097–1105

  30. Liaqat M, Khan S, Majid M (2017). Image retrieval based on fuzzy ontology. Multimedia Tools and Applications

  31. Miller GA (1995) WordNet: a lexical database for English. Commun ACM 38(11):39–41

    Article  Google Scholar 

  32. Natsev AP, Haubold A, Tešić J, Xie L, Yan R (2007) Semantic concept-based query expansion and re-ranking for multimedia retrieval. Proceedings of the 15th International Conference on Multimedia

  33. Nematzadeh A, Grant E, Stevenson SA (2015) Computational Cognitive Model of Novel Word Generalization. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, 17–21, (September), 1795–1804

  34. Niblack W et al. (1993) The QBIC project: Querying images by content using color, texture, and shape. In Storage and Retrieval for Image and Video Databases, pages 173–181, SPIE

  35. Park G, Baek Y, Lee H (2003) Majority Based Ranking Approach in Web Image Retrieval, Proc. Second Int’l Conf. Image and Video Retrieval

  36. Rasiwasia N, Vasconcelos N. (2008) A study of query by semantic example. 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops

  37. Rasiwasia N, Moreno PJ, Vasconcelos N (2007) Bridging the gap: query by semantic example. IEEE Trans Multimed 9(5):923–938

    Article  Google Scholar 

  38. SHEPARD RN et al (1987) Toward a universal law of generalization for psychological science. Science 237(4820):1317–1323

    Article  MathSciNet  Google Scholar 

  39. Silva R, Heller KA, Ghahramani Z (2007) Analogical reasoning with relational bayesian sets. International Conference on AI and Statistics

  40. Song J, He T, Gao L et al. (2017) Deep region hashing for efficient large-scale instance search from images. arXiv preprint arXiv:1701.07901

  41. Song J, Gao L, Liu L et al (2018) Quantization-based hashing: a general framework for scalable image and video retrieval. Pattern Recogn 75:175–187

    Article  Google Scholar 

  42. Tang X, Liu K, Cui J, Wen F, Wang X (2012) IntentSearch:capturing user intention for one-click internet image search. IEEE Trans Pattern Anal Mach Intell 34(7):1342–1353

    Article  Google Scholar 

  43. Tenenbaum JB, Griffiths TL (2001) Generalization, similarity, and Bayesian inference, behavioral and brain sciences. pp. 629–630

  44. Tenenbaum JB, Xu F (2000) Word learning as Bayesian inference, Proc. TwentySecond Annu. Conf. Cogn. Sci. Soc

  45. The BBC Wildlife Ontology. Available from: http://www.bbc.co.uk/ontologies/wo

  46. Tolias G, Jégou H (2014) Visual query expansion with or without geometry: refining local descriptors by feature aggregation. Pattern Recogn 47(10):3466–3476

    Article  Google Scholar 

  47. Torralba A, Murphy K, Freeman W, Rubin M (2003) Context Based Vision System for Place and Object Recognition, Proc. Int’l Conf. Computer Vision

  48. Wan, Ji, Wang, Dayong, Hoi, Steven, Chu Hong, et al. (2014) Deep learning for content-based image retrieval: A comprehensive study. In : Proceedings of the 22nd ACM international conference on Multimedia. ACM. p. 157–166

  49. Wang X, Gao L, Wang P et al (2018) Two-stream 3-D convNet fusion for action recognition in videos with arbitrary size and length. IEEE Transactions on Multimedia 20(3):634–644

    Article  Google Scholar 

  50. Wang J, Zhang T, Sebe N et al (2018) A survey on learning to hash. IEEE Trans Pattern Anal Mach Intell 40(4):769–790

    Article  Google Scholar 

  51. Yu D, Seltzer ML, Li J, Huang J-T, and Seide F (2013) Feature learning in deep neural networks - a study on speech recognition tasks. CoRR, abs/1301.3605

  52. Zeiler MD, Fergus R (2013) Visualizing and understanding convolutional networks. CoRR, abs/1311.2901

  53. Zha Z-J, Yang L, Mei T, Wang M, & Wang Z. (2009). Visual query suggestion. Proceedings of the Seventeen ACM International Conference on Multimedia - MM ‘09, 6(3), 15

  54. Zhao, Fang, Huang, Yongzhen, Wang, Liang, et al. (2015) Deep semantic ranking based hashing for multi-label image retrieval. In : Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on. IEEE, p. 1556–1564

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, University of Constantine 2 - Abdelhamid Mehri, Constantine, Algeria

    Meriem Korichi & Mohamed Batouche

  2. LAMIA Laboratory, University du Québec à Trois-Rivières, Trois-Rivières, Canada

    Mohamed Lamine Kherfi

  3. LINATI Laboratory, University of Ouargla - Kasdi Merbah, Ouargla, Algeria

    Mohamed Lamine Kherfi

  4. Department of Mathematics, University of Ouargla - Kasdi Merbah, Ouargla, Algeria

    Khadra Bouanane

Authors
  1. Meriem Korichi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Lamine Kherfi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Batouche
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Khadra Bouanane
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meriem Korichi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Korichi, M., Kherfi, M.L., Batouche, M. et al. Extended Bayesian generalization model for understanding user’s intention in semantics based images retrieval. Multimed Tools Appl 77, 31115–31138 (2018). https://doi.org/10.1007/s11042-018-6205-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6205-0

Keywords

Search

Navigation