Skip to main content
Book cover

National Conference of Theoretical Computer Science

NCTCS 2017: Theoretical Computer Science pp 233–247Cite as

A Survey of Personalised Image Retrieval and Recommendation

Part of the Communications in Computer and Information Science book series (CCIS,volume 768)

Abstract

With the advent of web2.0 era, it has been becoming increasingly easy to create and share Internet content. Plenty of pictures are uploaded to the Internet every day. A primary challenge against traditional image retrieval technologies is how to help users quickly discover the images they need. Personalised image retrieval is a new trend in the field of image retrieval. It not only improves the accuracy of the existing retrieval systems, but also better meets the users’ needs. Personalised image retrieval and recommendation (PIRR) can be grouped into two main categories, content-based PIRR and collaborative filtering (CF)-based PIRR. This paper first summarises the development of image retrieval and introduces different image retrieval solutions. Then the key technologies of content-based PIRR are analysed from three aspects, user interest acquisition, user interest representation and personalised implementation. Different techniques are compared and analysed. Regarding CF-based PIRR, the user-based, item-based and model-based CF-based PIRR are introduced and compared. At the end of the paper, we compare and summarise content-based PIRR and CF-based PIRR.

Keywords

  • Personalised image retrieval
  • Content-based
  • Collaborative filtering

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-981-10-6893-5_18
  • Chapter length: 15 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   69.99
Price excludes VAT (USA)
  • ISBN: 978-981-10-6893-5
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   89.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.

References

  1. Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: ideas, influences, and trends of the new age. ACM Comput. Surv. (Csur) 40(2), 5 (2008)

    CrossRef  Google Scholar 

  2. Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D.: Query by image and video content: the QBIC system. Computer 28(9), 23–32 (1995)

    CrossRef  Google Scholar 

  3. Mehrotra, S., Rui, Y., Chakrabarti, K., Ortega, M., Huang, T.S.: Multimedia analysis and retrieval system. In: Proceedings of the 3rd International Workshop on Information Retrieval Systems (1997)

    Google Scholar 

  4. Pentland, A., Picard, R.W., Sclaroff, S.: Photobook: content-based manipulation of image databases. Int. J. Comput. Vision 18(3), 233–254 (1996)

    CrossRef  Google Scholar 

  5. Wilensky, R.: UC Berkeley’s digital library project. Commun. ACM 38(4), 60 (1995)

    CrossRef  Google Scholar 

  6. Smith, J.R., Chang, S.-F.: VisualSEEk: a fully automated content-based image query system. In: Proceedings of the Fourth ACM International Conference on Multimedia, pp. 87–98. ACM (1997)

    Google Scholar 

  7. Xu, X., Peng, B., Sun, Z.: A semantic-based image retrieval system: VisEngine. Comput. Eng. 4, 021 (2004)

    Google Scholar 

  8. Zhang, H., Wenyin, L., Hu, C.: IFIND—A system for semantics and feature based image retrieval over Internet. In: Proceedings of the Eighth ACM International Conference on Multimedia, pp. 477–478. ACM (2000)

    Google Scholar 

  9. Wang, W., Wu, Y., Zhang, A.: SemView: a semantic-sensitive distributed image retrieval system. In: Proceedings of the 2003 Annual National Conference on Digital Government Research, pp. 1–4. Digital Government Society of North America (2003)

    Google Scholar 

  10. Ghorab, M.R., Zhou, D., O’Connor, A., Wade, V.: Personalised information retrieval: survey and classification. User Model. User-Adap. Inter. 23(4), 381–443 (2013)

    CrossRef  Google Scholar 

  11. Skowron, M., Tkalčič, M., Ferwerda, B., Schedl, M.: Fusing social media cues: personality prediction from twitter and instagram. In: Proceedings of the 25th International Conference Companion on World Wide Web, pp. 107–108. International World Wide Web Conferences Steering Committee (2016)

    Google Scholar 

  12. Liu, D., Hua, X.-S., Wang, M., Zhang, H.: Boost search relevance for tag-based social image retrieval. In: IEEE International Conference on Multimedia and Expo, ICME 2009, pp. 1636–1639. IEEE (2009)

    Google Scholar 

  13. Cheung, M., She, J.: Bag-of-features tagging approach for a better recommendation with social big data. In: Proceedings of the 4th International Conference on Advances in Information Mining and Management (IMMM 2014), pp. 83–88 (2014)

    Google Scholar 

  14. Sang, J., Xu, C., Lu, D.: Learn to personalized image search from the photo sharing websites. IEEE Trans. Multimedia 14(4), 963–974 (2012)

    CrossRef  Google Scholar 

  15. Qiu, Z.W., Zhang, T.W.: Individuation image retrieval based on user multimedia data management model. Acta Electron. Sin. 36(9), 1746–1749 (2008)

    Google Scholar 

  16. Fan, J., Keim, D.A., Gao, Y., Luo, H., Li, Z.: JustClick: personalized image recommendation via exploratory search from large-scale Flickr images. IEEE Trans. Circuits Syst. Video Technol. 19(2), 273–288 (2009)

    CrossRef  Google Scholar 

  17. Yu, J., Tao, D., Wang, M., Rui, Y.: Learning to rank using user clicks and visual features for image retrieval. IEEE Trans. Cybern. 45(4), 767–779 (2015)

    CrossRef  Google Scholar 

  18. Zhang, H., Zha, Z.-J., Yang, Y., Yan, S., Gao, Y., Chua, T.-S.: Attribute-augmented semantic hierarchy: towards bridging semantic gap and intention gap in image retrieval. In: Proceedings of the 21st ACM International Conference on Multimedia, pp. 33–42. ACM (2013)

    Google Scholar 

  19. Jayech, K., Mahjoub, M.A.: New approach using Bayesian Network to improve content based image classification systems. IJCSI Int. J. Comput. Sci. Issues 7(6), 53–62 (2010)

    Google Scholar 

  20. Hu, T., Yu, J.: Max-margin based Bayesian classifier. Front. Inf. Technol. Electron. Eng. 17(10), 973–981 (2016)

    CrossRef  Google Scholar 

  21. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  22. Lin, C.-H., Chen, C.-C., Lee, H.-L., Liao, J.-R.: Fast K-means algorithm based on a level histogram for image retrieval. Expert Syst. Appl. 41(7), 3276–3283 (2014)

    CrossRef  Google Scholar 

  23. Liu, Y., Zhang, D., Lu, G., Ma, W.-Y.: A survey of content-based image retrieval with high-level semantics. Pattern Recogn. 40(1), 262–282 (2007)

    CrossRef  MATH  Google Scholar 

  24. Wu, J., Xiao, Z.-B., Wang, H.-S., Shen, H.: Learning with both unlabeled data and query logs for image search. Comput. Electr. Eng. 40(3), 964–973 (2014)

    CrossRef  Google Scholar 

  25. Su, J.-H., Huang, W.-J., Philip, S.Y., Tseng, V.S.: Efficient relevance feedback for content-based image retrieval by mining user navigation patterns. IEEE Trans. Knowl. Data Eng. 23(3), 360–372 (2011)

    CrossRef  Google Scholar 

  26. Adomavicius, G., Tuzhilin, A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005)

    CrossRef  Google Scholar 

  27. Paik, J.H.: A novel TF-IDF weighting scheme for effective ranking. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 343–352. ACM (2013)

    Google Scholar 

  28. Whissell, J.S., Clarke, C.L.: Improving document clustering using Okapi BM25 feature weighting. Inf. Retrieval 14(5), 466–487 (2011)

    CrossRef  Google Scholar 

  29. Zhang, J., Zhuo, L., Shen, L., He, L.: A personalized image retrieval based on user interest model. Int. J. Pattern Recogn. Artif. Intell. 24(03), 401–419 (2010)

    CrossRef  Google Scholar 

  30. Nie, W., Li, X., Liu, A., Su, Y.: 3D object retrieval based on Spatial+LDA model. Multimedia Tools Appl. 76(3), 4091–4104 (2017)

    CrossRef  Google Scholar 

  31. Li, X., Ouyang, J., Lu, Y.: Topic modeling for large-scale text data. Front. Inf. Technol. Electron. Eng. 16(6), 457–465 (2015)

    CrossRef  Google Scholar 

  32. Zhang, Y., Jin, R., Zhou, Z.H.: Understanding bag-of-words model: a statistical framework. Int. J. Mach. Learn. Cybern. 1(1), 43–52 (2010)

    CrossRef  Google Scholar 

  33. Tu, N.A., Dinh, D.-L., Rasel, M.K., Lee, Y.-K.: Topic modeling and improvement of image representation for large-scale image retrieval. Inf. Sci. 366, 99–120 (2016)

    MathSciNet  CrossRef  Google Scholar 

  34. Shekhar, R., Jawahar, C.: Word image retrieval using bag of visual words. In: 2012 10th IAPR International Workshop on Document Analysis Systems (DAS), pp. 297–301. IEEE (2012)

    Google Scholar 

  35. Yang, J., Jiang, Y.-G., Hauptmann, A.G., Ngo, C.-W.: Evaluating bag-of-visual-words representations in scene classification. In: Proceedings of the International Workshop on Multimedia Information Retrieval, pp. 197–206. ACM (2007)

    Google Scholar 

  36. Liu, L.: Contextual topic model based image recommendation system. In: 2015 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), pp. 239–240. IEEE (2015)

    Google Scholar 

  37. Jia, D., Berg, A.C., Li, F.F.: Hierarchical semantic indexing for large scale image retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 785–792 (2011)

    Google Scholar 

  38. Jiang, X., Tan, A.-H.: Learning and inferencing in user ontology for personalized Semantic Web search. Inf. Sci. 179(16), 2794–2808 (2009)

    CrossRef  MATH  Google Scholar 

  39. Geng, X., Zhang, H., Bian, J., Chua, T.-S.: Learning image and user features for recommendation in social networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4274–4282 (2015)

    Google Scholar 

  40. Lei, C., Liu, D., Li, W., Zha, Z.J., Li, H.: Comparative deep learning of hybrid representations for image recommendations. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2545–2553 (2016)

    Google Scholar 

  41. Song, G., Jin, X., Chen, G., Nie, Y.: Two-level hierarchical feature learning for image classification. Front. Inf. Technol. Electron. Eng. 17(9), 897–906 (2016)

    CrossRef  Google Scholar 

  42. Burdescu, D.D., Mihai, C.G., Stanescu, L., Brezovan, M.: Automatic image annotation and semantic based image retrieval for medical domain. Neurocomputing 109, 33–48 (2013)

    CrossRef  Google Scholar 

  43. Kurtz, C., Depeursinge, A., Napel, S., Beaulieu, C.F., Rubin, D.L.: On combining image-based and ontological semantic dissimilarities for medical image retrieval applications. Med. Image Anal. 18(7), 1082–1100 (2014)

    CrossRef  Google Scholar 

  44. Kurtz, C., Beaulieu, C.F., Napel, S., Rubin, D.L.: A hierarchical knowledge-based approach for retrieving similar medical images described with semantic annotations. J. Biomed. Inform. 49, 227–244 (2014)

    CrossRef  Google Scholar 

  45. Rui, Y., Huang, T.S.: A novel relevance feedback technique in image retrieval. In: Proceedings of the Seventh ACM International Conference on Multimedia (Part 2), pp. 67–70. ACM (1999)

    Google Scholar 

  46. Rocchio, J.J.: Relevance feedback in information retrieval. In: Salton, G. (ed.) The Smart Retrieval System: Experiments in Automatic Document Processing, pp. 313–323. Prentice Hall Inc., Englewood Cliffs (1971)

    Google Scholar 

  47. Porkaew, K., Chakrabarti, K.: Query refinement for multimedia similarity retrieval in MARS. In: Proceedings of the Seventh ACM International Conference on Multimedia (Part 1), pp. 235–238. ACM (1999)

    Google Scholar 

  48. Tao, D., Tang, X., Li, X., Wu, X.: Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 28(7), 1088–1099 (2006)

    CrossRef  Google Scholar 

  49. Johnson, M., Shotton, J., Cipolla, R.: Semantic texton forests for image categorization and segmentation. In: Criminisi, A., Shotton, J. (eds.) Decision Forests for Computer Vision and Medical Image Analysis. Advances in Computer Vision and Pattern Recognition, pp. 211–227. Springer, London (2013)

    CrossRef  Google Scholar 

  50. Tong, S., Chang, E.: Support vector machine active learning for image retrieval. In: Proceedings of the Ninth ACM International Conference on Multimedia, pp. 107–118. ACM (2001)

    Google Scholar 

  51. Zhao, S., Du, N., Nauerz, A., Zhang, X., Yuan, Q., Fu, R.: Improved recommendation based on collaborative tagging behaviors. In: Proceedings of the 13th International Conference on Intelligent User Interfaces, pp. 413–416. ACM (2008)

    Google Scholar 

  52. Gong, S.J.: Personalized recommendation system based on association rules mining and collaborative filtering. In: Wang, Y. (ed.) Applied Mechanics and Materials, pp. 540–544. Trans Tech Publ, Zürich (2011)

    Google Scholar 

  53. Ju, B., Qian, Y., Ye, M.: Preference transfer model in collaborative filtering for implicit data. Front. Inf. Technol. Electron. Eng. 17(6), 489–500 (2016)

    CrossRef  Google Scholar 

  54. Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web, pp. 285–295. ACM (2001)

    Google Scholar 

  55. Zhou, K., Yang, S.-H., Zha, H.: Functional matrix factorizations for cold-start recommendation. In: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 315–324. ACM (2011)

    Google Scholar 

  56. Yuan, Z., Huang, C., Sun, X., Li, X., Xu, D.: A microblog recommendation algorithm based on social tagging and a temporal interest evolution model. Front. Inf. Technol. Electron. Eng. 16(7), 532–540 (2015)

    CrossRef  Google Scholar 

  57. Tiraweerakhajohn, C., Pinngern, O.: A combination of content-based filtering and item-based collaborative filtering using association rules. In: The 1st International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology. ECTI, Thailand (2004)

    Google Scholar 

  58. Lei, W., Qing, F., Zhou, J.: Improved personalized recommendation based on causal association rule and collaborative filtering. Int. J. Distance Educ. Technol. (IJDET) 14(3), 21–33 (2016)

    CrossRef  Google Scholar 

  59. Ye, H.: A personalized collaborative filtering recommendation using association rules mining and self-organizing map. JSW 6(4), 732–739 (2011)

    CrossRef  Google Scholar 

  60. Thorat, P.B., Goudar, R., Barve, S.: Survey on collaborative filtering, content-based filtering and hybrid recommendation system. Int. J. Comput. Appl. 110(4), 31–36 (2015)

    Google Scholar 

  61. Ma, Z., Leijon, A.: A model-based collaborative filtering method for bounded support data. In: 2012 3rd IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC), pp. 545–548. IEEE (2012)

    Google Scholar 

  62. Fernández-Tobías, I., Braunhofer, M., Elahi, M., Ricci, F., Cantador, I.: Alleviating the new user problem in collaborative filtering by exploiting personality information. User Model. User-Adap. Inter. 26(2–3), 221–255 (2016)

    CrossRef  Google Scholar 

  63. Yang, C., Zhou, Y., Chen, L., Zhang, X., Chiu, D.M.: Social-group-based ranking algorithms for cold-start video recommendation. Int. J. Data Sci. Anal. 1(3–4), 165–175 (2016)

    CrossRef  Google Scholar 

  64. Candillier, L., Meyer, F., Boullé, M.: Comparing state-of-the-art collaborative filtering systems. In: Perner, P. (ed.) MLDM 2007. LNCS, vol. 4571, pp. 548–562. Springer, Heidelberg (2007). doi:10.1007/978-3-540-73499-4_41

    CrossRef  Google Scholar 

  65. Sanchez, F., Barrilero, M., Uribe, S., Alvarez, F., Tena, A., Menendez, J.M.: Social and content hybrid image recommender system for mobile social networks. Mob. Netw. Appl. 17(6), 782–795 (2012)

    CrossRef  Google Scholar 

  66. Lekakos, G., Caravelas, P.: A hybrid approach for movie recommendation. Multimedia Tools Appl. 36(1), 55–70 (2008)

    CrossRef  Google Scholar 

  67. Widisinghe, A., Ranasinghe, D., Kulathilaka, K., Kaluarachchi, R., Wimalawarne, K.A.D.N.K.: picSEEK: collaborative filtering for context-based image recommendation. In: International Conference on Information and Automation for Sustainability, pp. 225–232 (2010)

    Google Scholar 

  68. Liu, X., Tsai, M.H., Huang, T.: Analyzing user preference for social image recommendation. arXiv:1604.07044 [cs.IR] (2016)

Download references

Acknowledgements

This project was supported by NSFC (61272353).

Author information

Authors and Affiliations

  1. Beijing Jiaotong University, Beijing, 100044, China

    Zhenyan Ji, Weina Yao, Huaiyu Pi & Wei Lu

  2. Victoria University, Melbourne, VIC, 3011, Australia

    Jing He

  3. Washington University in St. Louis, St. Louis, MO, 63130, USA

    Haishuai Wang

Authors
  1. Zhenyan Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weina Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huaiyu Pi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jing He
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Haishuai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenyan Ji .

Editor information

Editors and Affiliations

  1. The University of Texas at Dallas, Richardson, Texas, USA

    Dingzhu Du

  2. Hefei University of Technology, Hefei, Anhui Province, China

    Lian Li

  3. National University of Defense Technology, Changsha, Hunan, China

    En Zhu

  4. Huazhong University of Science and Technology, Wuhan, Hubei, China

    Kun He

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper

Ji, Z., Yao, W., Pi, H., Lu, W., He, J., Wang, H. (2017). A Survey of Personalised Image Retrieval and Recommendation. In: Du, D., Li, L., Zhu, E., He, K. (eds) Theoretical Computer Science. NCTCS 2017. Communications in Computer and Information Science, vol 768. Springer, Singapore. https://doi.org/10.1007/978-981-10-6893-5_18

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-6893-5_18

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-6892-8

  • Online ISBN: 978-981-10-6893-5

  • eBook Packages: Computer ScienceComputer Science (R0)