Skip to main content
SpringerLink
Log in

Hypergraph-based image search reranking with elastic net regularized regression

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image search reranking is emerging as an effective technique to refine the text-based image search results using visual information. In this paper, we introduce a novel hypergraph-based image search reranking method that accounts for both relevance and diversity of search results. Namely, the text-based image search results are taken as vertices in a probabilistic regression-based hypergraph model and reranking is formulated as a hypergraph ranking problem with absorbing nodes. More specifically, to discover related samples and characterize the relationships among them, we bring the Elastic Net regularized regression model into the hypergraph construction. Exceeding the conventional hypergraph construction schemes, our scheme is able to describe the high-order relationships and the local manifold structure among visual samples while ensuring the datum-adaptiveness. Afterward, we apply a hypergraph-based ranking with absorbing nodes to ensure a diversified reranking. That is, during the reranking process, previously-ranked samples are transformed into absorbing nodes at each iteration, thereby redundant ones are prevented from receiving high ranking scores. Extensive experiments on real-world data from Flickr suggest our proposed reranking method achieves promising results compared to existing reranking methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. www.flickr.com

References

  1. Boteanu B, Mironică I, Ionescu B (2015) Hierarchical clustering pseudo-relevance feedback for social image search result diversification. In: 13th international workshop on content-based multimedia indexing (CBMI)

  2. Boteanu B, Mironică I, Ionescu B (2017) Pseudo-relevance feedback diversification of social image retrieval results. Multimed Tools Appl 76 (9):11,889–11, 916

    Article  Google Scholar 

  3. Bouchakwa M, Ayadi Y, Amous I (2020) Multi-level diversification approach of semantic-based image retrieval results. Progress in Artificial Intelligence 9(1):1–30

    Article  Google Scholar 

  4. Bouchrika T, Zaied M, Jemai O, Ben Amar C (2012) Ordering computers by hand gestures recognition based on wavelet networks. In: CCCA12, pp 1–6

  5. Boughrara H, Chtourou M, Ben Amar C (2012) MLP neural network based face recognition system using constructive training algorithm. In: 2012 international conference on multimedia computing and systems, pp 233–238

  6. Bouhlel N, Ksibi A, Ben Ammar A, Ben Amar C (2016) Semantic-aware framework for mobile image search. In: International conference on intelligent systems design and applications, ISDA, pp 479–484

  7. Bouhlel N, Feki G, Ben Ammar A, Ben Amar C (2017) A hypergraph-based reranking model for retrieving diverse social images. In: International conference on computer analysis of images and patterns, pp 279–291

  8. Bouhlel N, Feki G, Ben Ammar A, Ben Amar C (2020) Hypergraph learning with collaborative representation for image search reranking. International Journal of Multimedia Information Retrieval

  9. Brin S, Page L (2012) The anatomy of a large-scale hypertextual web search engine. In: Computer networks and ISDN systems, Amsterdam, The Netherlands, vol 56, pp 3825–3833

  10. Cai J, Zha ZJ, Wang M, Zhang S, Tian Q (2015) An attribute-assisted reranking model for web image search. IEEE Trans Image Process 24 (1):261–272

    Article  MathSciNet  Google Scholar 

  11. Cheng B, Yang J, Yan S, Fu Y, Huang TS (2010) Learning with l1-graph for image analysis. IEEE Trans Image Process 19(4):858–866

    Article  MathSciNet  Google Scholar 

  12. Cheng XQ, Du P, Guo J, Zhu X, Chen Y (2013) Ranking on data manifold with sink points. IEEE Trans Knowl Data Eng 25(1):177–191

    Article  Google Scholar 

  13. Dang-Nguyen DT, Piras L, Giacinto G, Boato G, De Natale FG (2015) A hybrid approach for retrieving diverse social images of landmarks. In: International conference on multimedia and expo, pp 1–6

  14. Dang-Nguyen DT, Piras L, Giacinto G, Boato G, De Natale FG (2017) Multimodal retrieval with diversification and relevance feedback for tourist attraction images. ACM Transactions on Multimedia Computing, Communications and Applications 13(4). https://doi.org/10.1145/3103613

  15. ElAdel A, Ejbali R, Zaied M, Ben Amar C (2016) A hybrid approach for content-based image retrieval based on fast beta wavelet network and fuzzy decision support system. Mach Vis Appl 27(6):781–799

    Article  Google Scholar 

  16. Feki G, Fakhfakh R, Ben Ammar A, Ben Amar C (2016) Knowledge structures: which one to use for the query disambiguation?. In: International conference on intelligent systems design and applications, ISDA, pp 499–504

  17. Feki G, Fakhfakh R, Bouhlel N, Ben Ammar A, Ben Amar C (2016) Regim@2016 retrieving diverse social images task. In: Mediaeval 2016 workshop, vol 1739

  18. Friedman J, Hastie T, Tibshirani R (2010) Regularization paths for generalized linear models via coordinate descent. Tech. Rep. 1

  19. Gao Y, Dai Q (2014) Efficient view-based 3-D object retrieval via hypergraph learning. Tsinghua Sci Technol 19(3):250–256

    Article  Google Scholar 

  20. Gao Y, Wang M, Luan H, Shen J, Yan S, Tao D (2011) Tag-based social image search with visual-text joint hypergraph learning. Multimedia Conference and Co-Located Workshops, pp 1517–1520

  21. Hong C, Zhu J (2013) Hypergraph-based multi-example ranking with sparse representation for transductive learning image retrieval. Neurocomputing 101:94–103

    Article  Google Scholar 

  22. Huang BYY (2010) Hypergraph based visual categorization and segmentation. PhD thesis

  23. Huang Y, Liu Q, Zhang S, Metaxas DN (2010) Image retrieval via probabilistic hypergraph ranking. In: IEEE computer society conference on computer vision and pattern recognition, pp 3376–3383

  24. Ionescu B, Popescu A, Müller H, Menendez M, Radu AL (2014) Benchmarking result diversification in social image retrieval. IEEE International Conference on Image Processing, ICIP 75(2):3072–3076

    Google Scholar 

  25. Ionescu B, Popescu A, Lupu M, GÎnscă AL, Boteanu B, Müller H (2015) Div150cred: a social image retrieval result diversification with user tagging credibility dataset. In: 6th ACM multimedia systems conference, MMSys 2015, pp 207–212

  26. Jing Y, Baluja S (2008) Visualrank: applying pagerank to large-scale image search. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(11):1877–1890

    Article  Google Scholar 

  27. Lin GL, Peng H, Ma QL, Wei J, Qin JW (2010) Improving diversity in Web search results re-ranking using absorbing random walks. International Conference on Machine Learning and Cybernetics, ICMLC 2010(5):2416–2421

    Google Scholar 

  28. Lin W (2019) Aggregation of multiple pseudo relevance feedbacks for image search re-ranking. IEEE Access 7:147, 553–147, 559

    Article  Google Scholar 

  29. Mei T, Rui Y, Li S, Tian Q (2014) Multimedia search reranking: a literature survey. ACM Comput Surv 46(3)

  30. Mejdoub M, Fonteles L, Ben Amar C, Antonini M (2008) Fast indexing method for image retrieval using tree-structured lattices. In: 2008 international workshop on content-based multimedia indexing, pp 365–372

  31. Mejdoub M, Fonteles L, Ben Amar C, Antonini M (2009) Embedded lattices tree: an efficient indexing scheme for content based retrieval on image databases. J Vis Comun Image Represent 20(2):145–156

    Article  Google Scholar 

  32. Othmani M, Bellil W, Ben Amar C, Alimi AM (2010) A new structure and training procedure for multi-mother wavelet networks. International Journal of Wavelets, Multiresolution and Information Processing 08(01):149–175

    Article  MathSciNet  Google Scholar 

  33. Pedronette DCG, Torres RDS (2012) Exploiting contextual information for image re-ranking and rank aggregation. International Journal of Multimedia Information Retrieval 1(2):115–128

    Article  Google Scholar 

  34. Sabetghadam S, Palotti J, Rekabsaz N, Lupu M, Hanbury A (2015) TUW@MediaEval 2015 retrieving diverse social images task. In: Mediaeval 2015 workshop, vol 1436

  35. Spyromitros-Xioufis E, Papadopoulos S, Ginsca AL, Popescu A, Kompatsiaris Y, Vlahavas I (2015) Improving diversity in image search via supervised relevance scoring. In: The 2015 ACM international conference on multimedia retrieval, pp 323–330

  36. Sunderrajan S, Manjunath BS (2016) Context-aware hypergraph modeling for re-identification and summarization. IEEE Trans Multimed 18(1):51–63

    Article  Google Scholar 

  37. Teyeb I, Jemai O, Zaied M, Ben Amar C (2014) A novel approach for drowsy driver detection using head posture estimation and eyes recognition system based on wavelet network. In: IISA 2014, The 5th international conference on information, intelligence, systems and applications, pp 379–384

  38. Tian X, Yang L, Wang J, Wu X, Hua XS (2012) Bayesian visual reranking. IEEE Transactions on Multimedia 14(2):490

    Article  Google Scholar 

  39. Wang M, Yang K, Hua XS, Zhang HJ (2010) Towards a relevant and diverse search of social images. IEEE Transactions on Multimedia 12(8):829–842

    Article  Google Scholar 

  40. Wang M, Li H, Tao D, Lu K, Wu X (2012) Multimodal graph-based reranking for web image search. IEEE Trans Image Process 21(11):4649–4661

    Article  MathSciNet  Google Scholar 

  41. Wang M, Liu X, Wu X (2015) Visual classification by l1-hypergraph modeling. IEEE Trans Knowl Data Eng 27(9):2564–2574

    Article  Google Scholar 

  42. Wang X, Qiu S, Liu K, Tang X (2014) Web image re-ranking usingquery-specific semantic signatures. IEEE Transactions on Pattern Analysis and Machine Intelligence 36(4):810–823

    Article  Google Scholar 

  43. Weijer JVD, Schmid C, Verbeek J, Larlus D, Weijer JVD, Schmid C, Verbeek J, Larlus D, Weijer JVD, Schmid C, Verbeek J, Larlus D (2009) Learning color names for real-world applications. IEEE Transactions on Image Processing

  44. Xu J, An W, Zhang L, Zhang D (2019) Sparse, collaborative, or nonnegative representation: which helps pattern classification? Pattern Recogn 88:679–688

    Article  Google Scholar 

  45. Yan R, Hauptmann A, Jin R (2003) Multimedia search with pseudo-relevance feedback. In: The 2nd international conference on image and video retrieval, Berlin, Heidelberg, pp 238–247

  46. Zhou D, Huang J, Schölkopf B (2007) Learning with hypergraphs: clustering, classification, and embedding. Advances in Neural Information Processing Systems 19:1601–1608

    Google Scholar 

  47. Zhu L, Shen J, Jin H, Zheng R, Xie L (2015) Content-based visual landmark search via multimodal hypergraph learning. IEEE Transactions on Cybernetics 45(12):2756–2769

    Article  Google Scholar 

  48. Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B: Statistical Methodology 67(2):301–320

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The research leading to these results has received funding from the Ministry of Higher Education and Scientific Research of Tunisia under the grant agreement number LR11ES48.

Author information

Authors and Affiliations

  1. REGIM: Research Groups in Intelligent Machines, University of Sfax, National Engineering School of Sfax (ENIS), BP 1173, Sfax, 3038, Tunisia

    Noura Bouhlel, Ghada Feki & Chokri Ben Amar

Authors
  1. Noura Bouhlel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ghada Feki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chokri Ben Amar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Noura Bouhlel.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bouhlel, N., Feki, G. & Amar, C.B. Hypergraph-based image search reranking with elastic net regularized regression. Multimed Tools Appl 79, 30257–30280 (2020). https://doi.org/10.1007/s11042-020-09418-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09418-z

Keywords

Search

Navigation