Advertisement

Machine Vision and Applications

, Volume 29, Issue 2, pp 361–373 | Cite as

Beyond Eleven Color Names for Image Understanding

  • Lu Yu
  • Lichao Zhang
  • Joost van de Weijer
  • Fahad Shahbaz Khan
  • Yongmei Cheng
  • C. Alejandro Parraga
Original Paper

Abstract

Color description is one of the fundamental problems of image understanding. One of the popular ways to represent colors is by means of color names. Most existing work on color names focuses on only the eleven basic color terms of the English language. This could be limiting the discriminative power of these representations, and representations based on more color names are expected to perform better. However, there exists no clear strategy to choose additional color names. We collect a dataset of 28 additional color names. To ensure that the resulting color representation has high discriminative power we propose a method to order the additional color names according to their complementary nature with the basic color names. This allows us to compute color name representations with high discriminative power of arbitrary length. In the experiments we show that these new color name descriptors outperform the existing color name descriptor on the task of visual tracking, person re-identification and image classification.

Keywords

Color name Discriminative descriptors Image classification Re-identification Tracking 

Notes

Acknowledgements

We acknowledge Dimitris Mylonas for his helpful suggestion on extending the color name set. We also acknowledge Nicole Walasek who has been of great help in the dataset collection and PLSA code preparation. Lu Yu acknowledges the Chinese Scholarship Council (CSC) grant No.201506290126. This work was supported by TIN2013-41751-P and TIN2016-79717-R of the Spanish Ministry and the CERCA Programme / Generalitat de Catalunya.

References

  1. 1.
    Benavente, R., Vanrell, M., Bladrich, R.: A data set for fuzzy colour naming. COLOR Res. Appl. 31(1), 48–56 (2006)CrossRefGoogle Scholar
  2. 2.
    Benavente, R., Vanrell, M., Baldrich, R.: Parametric fuzzy sets for automatic color naming. J. Opt. Soc. Am. A 25(10), 2582–2593 (2008)CrossRefGoogle Scholar
  3. 3.
    Benavente, R., Van de Weijer, J., Vanrell, M., Schmid, C., Baldrich, R., Verbeek, J., Larlus, D.: Color names. In: Gevers, T., Gijsenij, A., van de Weijer, J., Geusebroek, J.M. Color in Computer Vision. Wiley, New York (2012)Google Scholar
  4. 4.
    Berlin, B., Kay, P.: Basic Color Terms: Their Universality and Evolution. University of California, Berkeley (1969)Google Scholar
  5. 5.
    Boynton, R.M., Olson, C.X.: Locating basic colors in the OSA space. Color Res. Appl. 12(2), 94–105 (1987)CrossRefGoogle Scholar
  6. 6.
    Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV, Prague, vol. 1, pp. 1–2 (2004)Google Scholar
  7. 7.
    Danelljan, M., Khan, F.S., Felsberg, M., van de Weijer, J.: Adaptive color attributes for real-time visual tracking. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1090–1097. IEEE (2014)Google Scholar
  8. 8.
    Danelljan, M., Häger, G., Khan, F.S., Felsberg, M.: Learning spatially regularized correlation filters for visual tracking. In: ICCV (2015)Google Scholar
  9. 9.
    Finlayson G.D., Schiele, B., Crowley, J.L.: Comprehensive colour image normalization. In ECCV ’98: Proceedings of the 5th European Conference on Computer Vision, vol. I, pp. 475–490. Springer (1998). ISBN 3-540-64569-1Google Scholar
  10. 10.
    Geusebroek, J.M., van den Boomgaard, R., Smeulders, A.W.M., Geerts, H.: Color invariance. PAMI 23(12), 1338–1350 (2001)CrossRefGoogle Scholar
  11. 11.
    Gevers, T., Smeulders, A.: Color based object recognition. Pattern Recognit. 32, 453–464 (1999)CrossRefGoogle Scholar
  12. 12.
    Hardin, C.L., Maffi, L. (eds.): Color Categories in Thought and Language. Cambridge University Press, Cambridge (1997)Google Scholar
  13. 13.
    Healey, G.: Segmenting images using normalized color. IEEE Trans. Syst. Man Cybern. 22, 64–73 (1992)CrossRefGoogle Scholar
  14. 14.
    Henriques, J.F., Caseiro, R., Martins, P., Batista, J.: Exploiting the circulant structure of tracking-by-detection with kernels. In: European Conference on Computer Vision, pp. 702–715. Springer (2012)Google Scholar
  15. 15.
    Hofmann, T.: Probabilistic latent semantic indexing. In: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 50–57. ACM (1999)Google Scholar
  16. 16.
    Hong, Z., Chen, Z., Wang, C., Mei, X., Prokhorov, D., Tao, D.: Multi-store tracker (muster): a cognitive psychology inspired approach to object tracking. In: CVPR (2015)Google Scholar
  17. 17.
    Huynh, C.P., Robles-Kelly, A.: A solution of the dichromatic model for multispectral photometric invariance. Int. J. Comput. Vis 90(1), 1–27 (2010)CrossRefGoogle Scholar
  18. 18.
    Kelly, K.L., Judd, D.B.: Color: Universal Language and Dictionary of Names, vol. 440. US Department of Commerce, National Bureau of Standards, Gaithersburg (1976)CrossRefGoogle Scholar
  19. 19.
    Khan, F.S., Weijer, J., Bagdanov, A.D., Vanrell, M.: Portmanteau vocabularies for multi-cue image representation. In: Advances in Neural Information Processing Systems, pp. 1323–1331 (2011)Google Scholar
  20. 20.
    Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A.D., Vanrell, M., Lopez, A.M.: Color attributes for object detection. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3306–3313. IEEE (2012)Google Scholar
  21. 21.
    Khan, F.S., Van de Weijer, J., Vanrell, M.: Modulating shape features by color attention for object recognition. Int. J. Comput. Vis. 98(1), 49–64 (2012)CrossRefGoogle Scholar
  22. 22.
    Khan, F.S., Anwer, R.M., van de Weijer, J., Bagdanov, A.D., Lopez, A.M., Felsberg, M.: Coloring action recognition in still images. Int. J. Comput. Vis. 105(3), 205–221 (2013a)CrossRefGoogle Scholar
  23. 23.
    Khan, R., Van de Weijer, J., Khan, F.S., Muselet, D., Ducottet, C., Barat, C.: Discriminative color descriptors. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2866–2873. IEEE (2013)Google Scholar
  24. 24.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  25. 25.
    LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)CrossRefGoogle Scholar
  26. 26.
    Li, Y., Zhu, J.: A scale adaptive kernel correlation filter tracker with feature integration. In: ECCV Workshop (2014)Google Scholar
  27. 27.
    Liu, X., Wang, H., Wu, Y., Yang, J., Yang, H.: An ensemble color model for human re-identification. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 868–875. IEEE (2015)Google Scholar
  28. 28.
    Liu, Y., Zhang, D., Lu, G., Ma, W-Y.: Region-based image retrieval with high-level semantic color names. In: Proceedings of the 11th International Multimedia Modelling Conference, 2005 (MMM 2005), pp. 180–187. IEEE (2005)Google Scholar
  29. 29.
    Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3730–3738 (2015)Google Scholar
  30. 30.
    Mojsilovic, A.: A computational model for color naming and describing color composition of images. IEEE Trans. Image Process. 14(5), 690–699 (2005)CrossRefGoogle Scholar
  31. 31.
    Mylonas, D., Griffin, D., Purver, L.D., Katemake, P., Davidoff J.: The role of primary colours in colour naming. (2016) (Under review) Google Scholar
  32. 32.
    Mylonas, D., MacDonald, L.: Augmenting basic colour terms in english. Color Res. Appl. 41, 32–42 (2015)CrossRefGoogle Scholar
  33. 33.
    Nilsback, M.-E., Zisserman, A.: Automated flower classification over a large number of classes. In: Sixth Indian Conference on Computer Vision, Graphics & Image Processing, 2008 (ICVGIP’08), pp. 722–729. IEEE (2008)Google Scholar
  34. 34.
    Párraga, C., Benavente, R., Baldrich, R., Vanrell, M.: Psychophysical measurements to model intercolor regions of color-naming space. J. Imaging Sci. Technol. 53(3), 31106-1 (2009)CrossRefGoogle Scholar
  35. 35.
    Parraga, C.A., Akbarinia, A.: NICE: A computational solution to close the gap from colour perception to colour categorization. PLoS ONE 11(3), e0149538 (2016)CrossRefGoogle Scholar
  36. 36.
    Schwartz, G., Nishino, K.: Discovering perceptual attributes in a deep local material recognition network. arXiv preprint arXiv:1604.01345 (2016)
  37. 37.
    Sharif Razavian, A., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 806–813 (2014)Google Scholar
  38. 38.
    Sturges, J., Whitfield, T.W.: Locating basic colours in the munsell space. Color Res. Appl. 20(6), 364–376 (1995)CrossRefGoogle Scholar
  39. 39.
    van de Sande, K.E.A., Theo, G., Cees, G., Snoek, M.: Evaluating color descriptors for object and scene recognition. PAMI 32(9), 1582–1596 (2010)CrossRefGoogle Scholar
  40. 40.
    Van De Weijer, J., Khan, F.S.: An overview of color name applications in computer vision. In: International Workshop on Computational Color Imaging, pp. 16–22. Springer (2015)Google Scholar
  41. 41.
    van de Weijer, J., Schmid, C.: Applying color names to image description. In: IEEE International Conference on Image Processing (ICIP), San Antonio, USA (2007)Google Scholar
  42. 42.
    Van De Weijer, J., Schmid, C., Verbeek, J., Larlus, D.: Learning color names for real-world applications. IEEE Trans. Image Process. 18(7), 1512–1523 (2009)MathSciNetCrossRefMATHGoogle Scholar
  43. 43.
    van de Weijer, J., Schmid, C., Verbeek, J., Larlus, D.: Learning color names for real-world applications. IEEE Trans. Image Process. 18(7), 1512–1524 (2009)MathSciNetCrossRefMATHGoogle Scholar
  44. 44.
    Wang, Y., Li, S., Kot, A.C.: On branded handbag recognition. IEEE Trans. Multimedia 18(9), 1869–1881 (2016)CrossRefGoogle Scholar
  45. 45.
    Wang, Y., Liu, J., Wang, J., Li, Y., Lu, H.: Color names learning using convolutional neural networks. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 217–221. IEEE (2015)Google Scholar
  46. 46.
    Wu, Y, Lim, J., Yang, M-H.: Online object tracking: a benchmark. In: 2013 IEEE Conference on Computer Vision and Pattern Recogniton (CVPR). IEEE (2013)Google Scholar
  47. 47.
    Yang, Y., Yang, J., Yan, J., Liao, S., Yi, D., Li, S.: Salient color names for person re-identification. In: 2014 European Conference on Computer Vision (ECCV), pp. 536–551. Springer (2014)Google Scholar
  48. 48.
    Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: a benchmark. In: IEEE International Conference on Computer Vision (ICCV) (2015)Google Scholar
  49. 49.
    Zollinger, H.: Why just turquoise? Remarks on the evolution of color terms. Psychol. Res. 46(4), 403–409 (1984)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Key Laboratory of Information Fusion TechnologyNorthwestern Polytechnical UniversityXi’anChina
  2. 2.Computer Vision LaboratoryLinköping UniversityLinköpingSweden
  3. 3.Computer Vision Center/Computer Science DepartmentUniversitat Autonoma de BarcelonaBarcelonaSpain

Personalised recommendations