International Journal of Computer Vision

, Volume 7, Issue 1, pp 11–32 | Cite as

Color indexing

  • Michael J. Swain
  • Dana H. Ballard


Computer vision is embracing a new research focus in which the aim is to develop visual skills for robots that allow them to interact with a dynamic, realistic environment. To achieve this aim, new kinds of vision algorithms need to be developed which run in real time and subserve the robot's goals. Two fundamental goals are determining the location of a known object. Color can be successfully used for both tasks.

This article demonstrates that color histograms of multicolored objects provide a robust, efficient cue for indexing into a large database of models. It shows that color histograms are stable object representations in the presence of occlusion and over change in view, and that they can differentiate among a large number of objects. For solving the identification problem, it introduces a technique calledHistogram Intersection, which matches model and image histograms and a fast incremental version of Histogram Intersection, which allows real-time indexing into a large database of stored models. For solving the location problem it introduces an algorithm calledHistogram Backprojection, which performs this task efficiently in crowded scenes.


Computer Vision Identification Problem Location Problem Large Database Research Focus 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aloimonos, J. 1990. “Purposive and qualitative active vision.”Proc. Int. Conf. Pat. Rec., pp. 346–360.Google Scholar
  2. Aloimonos, J., Weiss, I., and Bandyopadhay, A. 1988 “Active vision.”Intern. J. Comput. Vision 1:436–440.Google Scholar
  3. Bajcsy, R. 1985. “Active perception vs. passive perception,”Workshop on Computer Vision: Representation and Control, pp. 55–59.Google Scholar
  4. Bajcsy, R. 1988, “Active perception.”Proc. IEEE 76:996–1005.Google Scholar
  5. Ballard, D.H. 1987. “Interpolation coding: A representation for numbers in neural models.”Biological Cybernetics, 57:389–402.Google Scholar
  6. Ballard, D.H. 1989. “Reference frames for animate vision.”Intern. Joint Conf. Artif. Intell., pp. 1635–1641.Google Scholar
  7. Ballard, D.H. 1991. “Animate vision.”Artificial Intelligence 48:57–86.Google Scholar
  8. Ballard, D.H., and Brown, C.M. 1982.Computer Vision. Prentice Hall: New York.Google Scholar
  9. Biederman, I. 1985. “Human image understanding: Recent research and a theory.”Comput. Vision, Graph. Image Process 32(1):29–73.Google Scholar
  10. Brainard, D.H., Wandell, B.A., and Cowan, W.B. 1989. “Black light: How sensors filter spectral variation of the illuminant.”IEEE Trans. Biomed. Engineer. 36:140–149.Google Scholar
  11. Chapman, D. 1990. “Vision, instruction, and action.” Technical Report 1204, Massachusetts Institute of Technology, Artificial Intelligence Laboratory, Cambridge, MA.Google Scholar
  12. Coombs, D.J. 1989. “Tracking objects with eye movements.”Proc. Topical Meet. Image Understand. Mach. Vision.Google Scholar
  13. Dickmanns, E.D. 1988. “An integrated approach to feature based dynamic vision.”Proc. IEEE Conf. Comput. Vision and Patt. Recog., pp. 820–825.Google Scholar
  14. Feldman, J.A. 1985. “Four frames suffice: A provisional model of vision and space.”Behav. Brain Sci. 8:265–289.Google Scholar
  15. Feldman, J.A., and Yakimovsky, Y. 1984. “Decision theory and artificial intelligence: I. A semantics-based region analyzer.”Artificial Intelligence 5:349–371.Google Scholar
  16. Forsyth, D.A. 1990. “A novel algorithm for color constancy.”Intern. J. Comput. Vision 5:5–35.Google Scholar
  17. Freeman, W.T., and Adelson, E.H. 1990. “Steerable filters for early vision, image analysis, and wavelet decomposition.”Proc. 3rd Intern. Conf. Comput. Vision, Osaka, pp. 406–415.Google Scholar
  18. Garvey, T.D. 1986. “Perceptual strategies for purposive vision.” SRI International, Technical Note 117.Google Scholar
  19. Hernstein, R.J. 1982. “Objects, categories, and discriminative stimuli.”Animal Cogn. Proc. Harry Frank Guggenheim Conf. Google Scholar
  20. Klinker, G.J., Shafer, S.A., and Kanade, T. 1988. “The measurement of highlights in color images.”Intern. J. Comput. Vision, 2:7–32.Google Scholar
  21. Koenderink, J.J., and van, Doorn, A.J. 1976. “The singularities of the visual mapping.”Biological Cybernetics 24:51–59.Google Scholar
  22. Lennie, P., and D'Zmura, M. 1988. “Mechanisms of color vision.”CRC Crit. Rev. Neurobiol. 3:333–400.Google Scholar
  23. Malik, J., and Perona, P. 1990. “Preattentive texture discrimination with early vision mechanisms.”J. Opt. Soc. Amer. A. 7:923–932.Google Scholar
  24. Maloney, L.T., and Wandell, B. 1986. “Color constancy: A method for recovering surface spectral reflectance.”J. Opt. Soc. Amer. A 3(1):29–33.Google Scholar
  25. Maunsell, J.H.R., and Newsome, W.T. 1987. “Visual Processing in monkey extrastriate cortex.”Annu. Rev. Neurosci. 10:363–401.Google Scholar
  26. Mishkin, M., and Appenzeller, T. 1987. “The anatomy of memory.”Scientific American, June, pp. 80–89.Google Scholar
  27. Nelson, R.C. 1989. “Obstacle avoidance using flow field divergence.”IEEE Trans. Patt. Anal. Mach. Intell. 11:1102–1106.Google Scholar
  28. Nelson, R.C. 1991. “Qualitative detection of motion by a moving observer.” In this issue.Google Scholar
  29. Novak, C.L., and Shafer, S.A. 1990. “Supervised color constancy using a color chart.” School of Computer Science, Carnegie Mellon University, Technical Report CUM-CS-90-140.Google Scholar
  30. Ohlander, R., Price, K., and Reddy, D.R. 1978. “Picture segmentation using a recursive region splitting method.”Comput. Graph. Image Process. 8:313–333.Google Scholar
  31. Olson, T.J., and Coombs, D.J. 1991. “Real-time vergence control for binocular robots.” In this issue.Google Scholar
  32. Rubner, J., and Schulten, K. 1989. “A regularized approach to color constancy.”Biological Cybernetics 61:29–36.Google Scholar
  33. Strat, T.M. 1990, personal communication.Google Scholar
  34. Swain, M.J. 1990a. “Color indexing.” Department of Computer Science, University of Rochester, Technical report 360.Google Scholar
  35. Swain, M.J. 1990b. “Companion videotape to ‘color indexing’”.Google Scholar
  36. Thompson, W.B. 1986. “Inexact vision.”Workshop on Motion, Representation, and Analysis, pp. 15–22.Google Scholar
  37. Treisman, A. 1985. “Preattentive processing in vision.”Comput. Vision, Graph. Image Process. 31:156–177.Google Scholar
  38. Ullman, S. 1986. “An approach to object recognition: Aligning pictorial descriptions.” Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Technical Report 931.Google Scholar
  39. Yarbus, A.L. 1967.Eye Movements and Vision. Plenum Press: New York.Google Scholar
  40. Young, T.Y., and Fu, K.S. eds. 1986.Handbook of Pattern Recognition and Image Processing. Academic Press: San Diego, CA.Google Scholar

Copyright information

© Kluwer Academic Publishers 1991

Authors and Affiliations

  • Michael J. Swain
    • 1
  • Dana H. Ballard
    • 2
  1. 1.Department of Computer ScienceUniversity of ChicagoChicago
  2. 2.Department of Computer ScienceUniversity of RochesterRochester

Personalised recommendations