Hybrid Diffusion: Spectral-Temporal Graph Filtering for Manifold Ranking

  • Ahmet IscenEmail author
  • Yannis Avrithis
  • Giorgos Tolias
  • Teddy Furon
  • Ondřej Chum
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11362)


State of the art image retrieval performance is achieved with CNN features and manifold ranking using a k-NN similarity graph that is pre-computed off-line. The two most successful existing approaches are temporal filtering, where manifold ranking amounts to solving a sparse linear system online, and spectral filtering, where eigen-decomposition of the adjacency matrix is performed off-line and then manifold ranking amounts to dot-product search online. The former suffers from expensive queries and the latter from significant space overhead. Here we introduce a novel, theoretically well-founded hybrid filtering approach allowing full control of the space-time trade-off between these two extremes. Experimentally, we verify that our hybrid method delivers results on par with the state of the art, with lower memory demands compared to spectral filtering approaches and faster compared to temporal filtering.



This work was supported by MSMT LL1303 ERC-CZ grant and the OP VVV funded project CZ.02.1.01/0.0/0.0/16_019/0000765 “Research Center for Informatics”.


  1. 1.
    Arandjelovic, R., Zisserman, A.: Three things everyone should know to improve object retrieval. In: CVPR, June 2012Google Scholar
  2. 2.
    Bai, S., Bai, X., Tian, Q., Latecki, L.J.: Regularized diffusion process on bidirectional context for object retrieval. IEEE Trans. PAMI (2018)Google Scholar
  3. 3.
    Bai, S., Zhou, Z., Wang, J., Bai, X., Jan Latecki, L., Tian, Q.: Ensemble diffusion for retrieval. In: ICCV (2017)Google Scholar
  4. 4.
    Chum, O., Mikulik, A., Perdoch, M., Matas, J.: Total recall II: query expansion revisited. In: CVPR, June 2011Google Scholar
  5. 5.
    Chum, O., Philbin, J., Sivic, J., Isard, M., Zisserman, A.: Total recall: automatic query expansion with a generative feature model for object retrieval. In: ICCV, October 2007Google Scholar
  6. 6.
    Chung, F.R.: Spectral Graph Theory, vol. 92. American Mathematical Society (1997)Google Scholar
  7. 7.
    Delvinioti, A., Jégou, H., Amsaleg, L., Houle, M.: Image retrieval with reciprocal and shared nearest neighbors. In: VISAPP, January 2014Google Scholar
  8. 8.
    Donoser, M., Bischof, H.: Diffusion processes for retrieval revisited. In: CVPR (2013)Google Scholar
  9. 9.
    Gordo, A., Almazan, J., Revaud, J., Larlus, D.: End-to-end learning of deep visual representations for image retrieval. IJCV 124(2) (2017)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Hackbusch, W.: Iterative Solution of Large Sparse Systems of Equations. Springer, New York (1994). Scholar
  11. 11.
    Iscen, A., Avrithis, Y., Tolias, G., Furon, T., Chum, O.: Fast spectral ranking for similarity search. In: CVPR (2018)Google Scholar
  12. 12.
    Iscen, A., Tolias, G., Avrithis, Y., Furon, T., Chum, O.: Efficient diffusion on region manifolds: recovering small objects with compact CNN representations. In: CVPR (2017)Google Scholar
  13. 13.
    Jégou, H., Harzallah, H., Schmid, C.: A contextual dissimilarity measure for accurate and efficient image search. In: CVPR, June 2007Google Scholar
  14. 14.
    Joly, A., Buisson, O.: Logo retrieval with a contrario visual query expansion. In: ACM Multimedia, October 2009Google Scholar
  15. 15.
    Kalantidis, Y., Mellina, C., Osindero, S.: Cross-dimensional weighting for aggregated deep convolutional features. arXiv preprint arXiv:1512.04065 (2015)
  16. 16.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: CVPR, June 2007Google Scholar
  17. 17.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Lost in quantization: improving particular object retrieval in large scale image databases. In: CVPR, June 2008Google Scholar
  18. 18.
    Qin, D., Gammeter, S., Bossard, L., Quack, T., Van Gool, L.: Hello neighbor: accurate object retrieval with k-reciprocal nearest neighbors. In: CVPR (2011)Google Scholar
  19. 19.
    Radenović, F., Iscen, A., Tolias, G., Avrithis, Y., Chum, O.: Revisiting Oxford and Paris: large-scale image retrieval benchmarking. In: CVPR (2018)Google Scholar
  20. 20.
    Radenović, F., Tolias, G., Chum, O.: Fine-tuning CNN image retrieval with no human annotation. IEEE Trans. PAMI (2018)Google Scholar
  21. 21.
    Shen, X., Lin, Z., Brandt, J., Wu, Y.: Spatially-constrained similarity measure for large-scale object retrieval. IEEE Trans. PAMI 36(6), 1229–1241 (2014)CrossRefGoogle Scholar
  22. 22.
    Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: ICCV (2003)Google Scholar
  23. 23.
    Tolias, G., Jégou, H.: Visual query expansion with or without geometry: refining local descriptors by feature aggregation. Pattern Recogn. 47(10), 3466–3476 (2014)CrossRefGoogle Scholar
  24. 24.
    Tolias, G., Sicre, R., Jégou, H.: Particular object retrieval with integral max-pooling of CNN activations. ICLR (2016)Google Scholar
  25. 25.
    Tong, H., Faloutsos, C., Pan, J.Y.: Fast random walk with restart and its applications. In: Proceedings of the IEEE International Conference on Data Mining, pp. 613–622 (2006)Google Scholar
  26. 26.
    Trefethen, L.N., Bau III, D.: Numerical Linear Algebra. SIAM (1997)Google Scholar
  27. 27.
    Zhang, S., Yang, M., Cour, T., Yu, K., Metaxas, D.N.: Query specific fusion for image retrieval. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, pp. 660–673. Springer, Heidelberg (2012). Scholar
  28. 28.
    Zhou, D., Weston, J., Gretton, A., Bousquet, O., Schölkopf, B.: Ranking on data manifolds. In: NIPS (2003)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Ahmet Iscen
    • 1
    Email author
  • Yannis Avrithis
    • 2
  • Giorgos Tolias
    • 1
  • Teddy Furon
    • 2
  • Ondřej Chum
    • 1
  1. 1.VRG, FEECTU in PraguePragueCzech Republic
  2. 2.Univ Rennes, Inria, CNRS, IRISARennesFrance

Personalised recommendations