Logo localization and recognition in natural images using homographic class graphs

Abstract

We propose a method for localization and classification of brand logos in natural images. The system has to overcome multiple challenges such as perspective deformations, warping, variations of the shape and colors, occlusions, background variations. To deal with perspective variation, we rely on homography matching between the SIFT keypoints of logo instances of the same class. To address the changes in color, we construct a weighted graph of logo interconnections that is further analyzed to extract potentially multiple instances of the class. The main instance is built by grouping the keypoints of the graph connected logos onto the central image. The secondary instance is needed for color inverted logos and is obtained by inverting the orientation of the main instance. The constructed logo recognition system is tested on two databases (FlickrLogos-32 and BelgaLogos), outperforming state of the art with more than 10 % accuracy.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Notes

  1. 1.

    Confusion matrix, and other supplementary results may be retrieved from the project page http://imag.pub.ro/common/staff/rboia/logoRecognition/.

References

  1. 1.

    Bagdanov, A., Ballan, L., Bertini, M., Del Bimbo, A.: Trademark matching and retrieval in sports video databases. In: ACM MIR, pp. 79–86 (2007)

  2. 2.

    Boia, R., Florea, C.: Homographic class template for logo localization and recognition. In: Proc. of IbPRIA (2015)

  3. 3.

    Brown, M., Lowe, D.: Recognising panoramas. In: ICCV, pp. 1218–1225 (2003)

  4. 4.

    Brown, M., Lowe, D.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007)

    Article  Google Scholar 

  5. 5.

    Chan, D., Ge, R., Gershony, O., Hesterberg, T., Lambert, D.: Evaluating online ad campaigns in a pipeline: causal models at scale. In: ACM SIGKDD Int. Conf. on Knowledge discovery and data mining, pp. 7–16 (2010)

  6. 6.

    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR, pp. 886–893 (2005)

  7. 7.

    Dorko, G., Schmid, C.: Object class recognition using discriminative local features. Tech. rep, INRIA (2005)

  8. 8.

    Dubout, C., Fleuret, F.: Exact acceleration of linear object detectors. In: ECCV, pp. 310 – 311 (2012)

  9. 9.

    Dubrofsky, E.: Homography estimation. Master’s thesis, Carleton University (2009)

  10. 10.

    Everingham, M., Van Gool, L., Williams, C., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vis. 1, 303–338 (2010)

    Article  Google Scholar 

  11. 11.

    Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scale-invariant learning. In: CVPR, pp. 264–271 (2003)

  12. 12.

    Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with application to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  13. 13.

    Florea, L., Florea, C., Vranceanu, R., Vertan, C.: Can your eyes tell me how you think? a gaze directed estimation of the mental activity. In: BMVC (2013)

  14. 14.

    Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press (2004)

  15. 15.

    Hauagge, D.C., Snavely, N.: Image matching using local symmetry features. In: CVPR, pp. 206–213 (2012)

  16. 16.

    Jin, Y., Tao, L., Di, H., Rao, N., Xu, G.: Background modeling from a free-moving camera by multi-layer homography algorithm. In: ICIP, pp. 1572–1575 (2008)

  17. 17.

    Joly, A., Buisson, O.: Logo retrieval with a contrario visual query expansion. In: ACM MM, pp. 581–584 (2009)

  18. 18.

    Kleban, J., Xie, X., Ma, W.Y.: Spatial pyramid mining for logo detection in natural scenes. In: IEEE ICME, pp. 1470–1477 (2008)

  19. 19.

    Klein, G., Murray, D.: Parallel tracking and mapping on a camera phone. In: Int. Symp. on mixed and augmented reality, pp. 83–86 (2009)

  20. 20.

    Krapac, J., Perronnin, F., Furon, T., Jegou, H.: Instance classification with prototype selection. In: ACM ICMR, pp. 431–434 (2014)

  21. 21.

    Kumar, S., Hebert, M.: Discriminative fields for modeling spatial dependencies in natural images. In: NIPS, pp. 1531–1538 (2004)

  22. 22.

    Lewis, R., Rao, J., Reiley, D.: Measuring the effects of advertising: the digital frontier. In: Economics of digitization, pp. 1–5 (2014)

  23. 23.

    Li, K., Chen, S., Su, S., Duh, D., Zhang, H., Li, S.: Logo detection with extendibility and discrimination. Multimed. Tools Appl. 72, 1285–1310 (2014)

    Article  Google Scholar 

  24. 24.

    Lin, Y., Medioni, G.: Map-enhanced uav image sequence registration and synchronization of multiple image sequences. In: CVPR, pp. 1–7 (2007)

  25. 25.

    Lowe, D.: Object recognition from local scale-invariant features. In: ICCV, pp. 1150–1157 (1999)

  26. 26.

    Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV 62(2), 91–110 (2004)

    Article  Google Scholar 

  27. 27.

    Lu, V., Endres, I., Stroila, M., Hart, J.: Accelerating arrays of linear classifiers using approximate range queries. In: IEEE WACV, pp. 255–260 (2014)

  28. 28.

    McLauchlan, P.F., Jaenicke, A.: Image mosaicing using sequential bundle adjustment. Image Vis. Comput., pp. 751–759 (2002)

  29. 29.

    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Gool, L.V.: A comparison of affine region detectors. IJCV 62(1–2), 43–72 (2005)

  30. 30.

    Ng, A., Jordan, M.I.: On discriminative vs. generative classifiers: a comparison of logistic regression and naive bayes. In: NIPS, pp. 841–849 (2001)

  31. 31.

    Opelt, A., Fusseneger, M., Pinz, A., Auer, P.: Generic object recognition with boosting. IEEE T PAMI 28(3), 416–431 (2006)

    Article  Google Scholar 

  32. 32.

    Psyllos, A.P., Anagnostopoulos, C.N.E., Kayafas, E.: Vehicle logo recognition using a sift-based enhanced matching scheme. IEEE Trans. Intell. Transp. Syst. 11(2), 322–328 (2010)

    Article  Google Scholar 

  33. 33.

    Raguram, R., Frahm, J.M., Pollefeys, M.: A comparative analysis of ransac techniques leading to adaptive real-time random sample consensus. In: Proceedings of the 10th European conference on computer vision: part II, Berlin, Heidelberg, ECCV ’08, pp. 500–513 (2008)

  34. 34.

    Revaud, J., Douze, M., Schmid, C.: Correlation-based burstiness for logo retrieval. In: ACM MM, pp. 965–968 (2012)

  35. 35.

    Ries, C., Richter, F., Romberg, S., Lienhart, R.: Automatic object annotation from weakly labeled data with latent structured svm. In: CBMI, pp. 1–4 (2014)

  36. 36.

    Romberg, S., Lienhart, R.: Bundle min-hashing for logo recognition. In: ACM ICMR (2013)

  37. 37.

    Romberg, S., Garcia Pueyo, L., Lienhart, R., van Zwol, R.: Scalable logo recognition in real-world images. In: ACM ICMR, pp. 965–968 (2011)

  38. 38.

    Schneiderman, H., Kanade, T.: A statistical method for 3d ob-ject detection applied to faces and cars. In: CVPR, pp. 746–751 (2003)

  39. 39.

    Sheikh, Y., Javed, O., Kanade, T.: Background subtraction for freely moving cameras. In: ICCV, pp. 1219–1225 (2009)

  40. 40.

    Simon, G., Fitzgibbon, A., Zisserman, A.: Markerless tracking using planar structures in the scene. In: Augmented reality, pp. 120–128 (2000)

  41. 41.

    Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from internet photo collections. Int. J. Comput. Vis. pp. 189–210 (2008)

  42. 42.

    Torii, A., Sivic, J., Pajdla, T., Okutomi, M.: Visual place recognition with repetitive structures. In: CVPR (2013)

  43. 43.

    Torralba, A., Murphy, K., Freeman, W.: Sharing visual fea-tures for multiclass and multiview object detection. In: CVPR, pp. 762–769 (2004)

  44. 44.

    Vedaldi, A., Fulkerson, B.: Vlfeat: an open and portable library of computer vision algorithms. In: ACM MM, pp. 1469–1472 (2010)

  45. 45.

    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. CVPR 1, 511–518 (2004)

    Google Scholar 

  46. 46.

    Zhu, G., Doermann, D.: Automatic document logo detection. In: ICDAR, pp. 864–868 (2007)

Download references

Acknowledgments

This work was supported by the Romanian Sectoral Operational Programme Human Resources Development 2007–2013 through the European Social Fund Financial Agreements POSDRU/159/1.5/S/132395 and POSDRU /159/1.5/S/134398.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Laura Florea.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Boia, R., Florea, C., Florea, L. et al. Logo localization and recognition in natural images using homographic class graphs. Machine Vision and Applications 27, 287–301 (2016). https://doi.org/10.1007/s00138-015-0741-7

Download citation

Keywords

  • Logo
  • Localization
  • Recognition
  • Class model
  • Homography