Natural Scene Image Modeling Using Color and Texture Visterms

  • Pedro Quelhas
  • Jean-Marc Odobez
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4071)


This paper presents a novel approach for visual scene representation, combining the use of quantized color and texture local invariant features (referred to here as visterms) computed over interest point regions. In particular we investigate the different ways to fuse together local information from texture and color in order to provide a better visterm representation. We develop and test our methods on the task of image classification using a 6-class natural scene database. We perform classification based on the bag-of-visterms (BOV) representation (histogram of quantized local descriptors), extracted from both texture and color features. We investigate two different fusion approaches at the feature level: fusing local descriptors together and creating one representation of joint texture-color visterms, or concatenating the histogram representation of both color and texture, obtained independently from each local feature. On our classification task we show that the appropriate use of color improves the results w.r.t. a texture only representation.


Support Vector Machine Interest Point Natural Scene Local Descriptor Fusion Scheme 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Mikolajczyk, K., Schmid, C.: Scale and affine interest point detectors. International Journal of Computer Vision 60, 63–86 (2004)CrossRefGoogle Scholar
  2. 2.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 91–110 (2004)CrossRefGoogle Scholar
  3. 3.
    Willamowski, J., Arregui, D., Csurka, G., Dance, C., Fan, L.: Categorizing nine visual classes using local appearance descriptors. In: Proc. of LAVS Workshop, in ICPR 2004, Cambridge (2004)Google Scholar
  4. 4.
    Quelhas, P., Monay, F., Odobez, J.M., Gatica-Perez, D., Tuytelaars, T., Gool, L.V.: Modeling scenes with local descriptors and latent aspects. In: Proc. of IEEE Int. Conf. on Computer Vision, Beijing (2005)Google Scholar
  5. 5.
    Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: Proc. of IEEE Int. Conf. on Computer Vision And Pattern Recognition, San Diego (2005)Google Scholar
  6. 6.
    Sivic, J., Zisserman, A.: Video google: A text retrieval approach to object matching in videos. In: Proc. of IEEE Int. Conf. on Computer Vision, Nice (2003)Google Scholar
  7. 7.
    Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T.: Discovering object categories in image collections. In: Proc. of IEEE Int. Conf. on Computer Vision, Beijing (2005)Google Scholar
  8. 8.
    Dorko, G., Schmid, C.: Selection of scale invariant parts for object class recognition. In: Proc. of IEEE Int. Conference on Computer Vision, Nice (2003)Google Scholar
  9. 9.
    Vailaya, A., Figueiredo, M., Jain, A., Zhang, H.: Image classification for content-based indexing. IEEE Trans. on Image Processing 10, 117–130 (2001)MATHCrossRefGoogle Scholar
  10. 10.
    Szummer, M., Picard, R.: Indoor-outdoor image classification. In: IEEE International Workshop CAIVD, in ICCV 1998, Bombay (1998)Google Scholar
  11. 11.
    Oliva, A., Torralba, A., Guerin-Dugue, A., Herault, J.: Global semantic classification of scenes using power spectrum templates. In: Proc. of the Challenge of Image Retrieval, Newcastle upon Tyne, UK (1999)Google Scholar
  12. 12.
    Paek, S., Chang, S.-F.: A knowledge engineering approach for image classification based on probabilistic reasoning systems. In: Proc. of IEEE Int. Conference on Multimedia and Expo., New York (2000)Google Scholar
  13. 13.
    Smeulders, A., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. on Pattern Analysis and Machine Intelligence 22, 1349–1380 (2000)CrossRefGoogle Scholar
  14. 14.
    Serrano, N., Savakis, A., Luo, J.: A computationally efficent approach to indoor/outdoor scene classification. In: Int. Conf. on Pattern Recognition (2002)Google Scholar
  15. 15.
    Vogel, J., Schiele, B.: A semantic typicality measure for natural scene categorization. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 195–203. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Boutell, M., Luo, J., Shen, X., Brown, C.M.: Learning multi-label scene classification. Pattern Recognition 37, 1757–1771 (2004)CrossRefGoogle Scholar
  17. 17.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. In: Proc. of IEEE Int. Conf. on Comp. Vision and Pattern Recognition (2003)Google Scholar
  18. 18.
    Matas, J., Chum, O., Martin, U., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proc. of the British Machine Vision Conference, Cardiff (2002)Google Scholar
  19. 19.
    Kittler, J., Hatef, M., Duin, R., Matas, J.: On combining classifiers. IEEE PAMI 20, 226–239 (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Pedro Quelhas
    • 1
    • 2
  • Jean-Marc Odobez
    • 1
    • 2
  1. 1.IDIAP Research Institute 
  2. 2.Ecole Polytechnique Federale de Lausanne (EPFL) 

Personalised recommendations