Skip to main content
SpringerLink
Log in

A physical approach to color image understanding

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper, we present an approach to color image understanding that can be used to segment and analyze surfaces with color variations due to highlights and shading. The work is based on a theory—the Dichromatic Reflection Model—which describes the color of the reflected light as a mixture of light from surface reflection (highlights) and body reflection (object color). In the past, we have shown how the dichromatic theory can be used to separate a color image into two intrinsic reflection images: an image of just the highlights, and the original image with the highlights removed. At that time, the algorithm could only be applied to hand-segmented images. This paper shows how the same reflection model can be used to include color image segmentation into the image analysis. The result is a color image understanding system, capable of generating physical descriptions of the reflection processes occurring in the scene. Such descriptions include the intrinsic reflection images, an image segmentation, and symbolic information about the object and highlight colors. This line of research can lead to physicsbased image understanding methods that are both more reliable and more useful than traditional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. D.H. Ballard and C.M. Brown, Computer Vision, Englewood Cliffs, NJ: Prentice-Hall, 1982.

    Google Scholar 

  2. H.G. Barrow and J.M. Tenenbaum, “Recovering intrinsic scene characteristics from images, Computer Vision Systems. New York: Academic Press, pp. 3–26, 1978.

    Google Scholar 

  3. R.L. Cook and K.E. Torrance, “A reflectance model for computer graphics,” ACM Trans. on Graphics 1 (1):7–24, 1982 (Also published in Computer Graphics 15(3), SIGGRAPH'81).

    Google Scholar 

  4. M. D'Zmura and P. Lennie, “Mechanisms of color constaney,” J. Opt. Soc. Amer. A 3(10):1662–1672, 1986.

    Google Scholar 

  5. L. Dreschler and H.-H. Nagel, “Volumetric model and 3D Trajectory of a moving car derived from monocular TV frame sequences of a street scene,” Comput. Graphics Image Process. 20:199–228, 1982.

    Google Scholar 

  6. R. Gershon, “The use of color in computational vision,” Ph.D. thesis, Department of Computer Science, University of Toronto, 1987.

  7. R. Gershon, A.D. Jepson, and J.K. Tsotsos, “Highlight identification using chromatic information,” Proc. 1st Intern. Conf. Computer Vision, pp. 161–171. IEEE, London, June 1987.

    Google Scholar 

  8. H. Grassmann, “On the theory of compound colors,” Phil. Mag., April 1854.

  9. A.R. Hanson and E.M. Riseman, “VISIONS: A computer system for interpreting scenes,” In A.R. Hanson and E.M. Riseman (eds.), Computer Vision Systems, New York: Academic Press, pp. 303–333, 1978.

    Google Scholar 

  10. H.H. Harman, Modern Factor Analysis, 2nd. ed. Chicago: University of Chicago Press, 1967.

    Google Scholar 

  11. G. Healey and T.O. Binford, “Local shape from specularity.” In L.S. Bauman (ed.), DARPA-Image Understanding (IUS) Workshop, pp. 874–887. Los Angeles, CA, February 1987.

  12. G. Healey and T.O. Binford, “Local shape from specularity,” Proc. 1st Intern. Conf. Computer Vision, pp. 151–161, IEEE, London, June 1987.

    Google Scholar 

  13. G. Healey and T.O. Binford, “The role and use of color in a general vision system.” In L.S. Bauman (ed.), DARPA-Image Understanding (IUS) Workshop, pp. 599–613. Los Angeles, CA, February 1987.

  14. B.K.P. Horn, “Understanding image intensities,” Artificial Intelligence 8(11):201–231, 1977.

    Google Scholar 

  15. B.K.P. Horn, “Exact reproduction of colored images,” Comput. Vision, Graphics, Image Process. 26:135–167, 1984.

    Google Scholar 

  16. B.K.P. Horn and B.G. Schunk, “Determining optical flow,” Artificial Intelligence 17:185–203, 1981.

    Google Scholar 

  17. T. Kanade, “Region segmentation: Signal vs. semantics.” Proc. 4th Intern. Joint Conf. Pattern Recog. pp. 95–105, IEEE, Kyoto, Japan, November 1978.

    Google Scholar 

  18. J.R. Kender, “Shape from texture.” Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, November 1980. Appeared also as technical report CMU-CS-81-102, Computer Science Department, Carnegie-Mellon University, Pittsburgh, PA.

  19. J.R. Kender and E.M. Smith, “Shape from darkness: Deriving surface information from dynamic shadows,” Proc. 1st Intern. Conf. Computer Vision, pp. 539–546. IEEE, London, June 1987.

    Google Scholar 

  20. G.J. Klinker, “A physical approach to color image understanding.” Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, May 1988. Available as technical report CMU-CS-88-161.

  21. G.J. Klinker, S.A. Shafer, and T. Kanade, “Using a color reflection model to separate highlights from object color,” Proc. 1st Intern. Conf. Comp. Vision, pp. 145–150, IEEE, London, June 1987.

    Google Scholar 

  22. G.J. Klinker, S.A. Shafer, and T. Kanade, “The measurement of highlights in color images,” Intern. J. Comput. Vision 2(1):7–32, 1988.

    Google Scholar 

  23. H.-C. Lee, “Method for computing the scene-illuminant chromaticity from specular highlights,” J. Opt. Soc. Amer. 3(10):1694–1699, 1986.

    Google Scholar 

  24. L.T. Maloney and B.A. Wandell, “Color constancy: A method for recovering surface spectral reflectance,” J. Opt. Soc. Amer. A 3(1):29–33, 1986.

    Google Scholar 

  25. R. Ohlander, K. Price, and D.R. Reddy, “Picture segmentation using a recursive region splitting method,” Comput. Graphics Image Process. 8:313–333, 1978.

    Google Scholar 

  26. Y. Ohta, T. Kanade, and T. Sakai, “Color information for region segmentation,” Comput. Graphics Image Process. 13:222–231, 1980.

    Google Scholar 

  27. A. Rosenfeld and A.C. Kak, Digital Picture Processing, 2nd ed. New York: Academic Press, 1982.

    Google Scholar 

  28. J.M. Rubin and W.A. Richards, “Color vision and image intensities: When are changes material?” Biological Cybernetics 45:215–226, 1982.

    Google Scholar 

  29. S.A. Shafer, “Describing light mixtures through linear algebra,” J. Opt. Soc. Amer. 72(2):299–300, 1982.

    Google Scholar 

  30. S.A. Shafer, “Using color to separate reflection components,” COLOR research and application 10(4):210–218, 1985. Also available as technical report TR-136, Computer Science Department, University of Rochester, NY, April 1984.

    Google Scholar 

  31. S.A. Shafer and T. Kanade, “Recursive region segmentation by analysis of histograms,” Proc. Intern. Conf. Acoustics, Speech, and Signal Process., pp. 1166–1171, IEEE, Paris, France, May, 1982.

    Google Scholar 

  32. S.A. Shafer and T. Kanade, “Using shadows in finding surface orientations,” Comput. Vision. Graphics, Image Process. 22:145–176, 1983.

    Google Scholar 

  33. C.E. Thorpe, “FIDO: Vision and navigation for a robot rover.” Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, December 1984. Available as technical report CMU-CS-84-168.

  34. F. Tong and B.V. Funt, “Specularity removal for shape from shading,” Proc. Conf. Vision Interface, Edmonton, Alberta, Canada, 1988.

  35. S. Ullman, “The interpretation of structure from motion,” Al Memo 476, MIT AI Laboratory, Cambridge, MA, October 1976.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cambridge Research Lab, Digital Equipment Corporation, One Kendall Square, Bldg. 700, 02139, Cambridge, MA

    Gudrun J. Klinker

  2. Computer Science Department, Carnegie Mellon University, 15213, Pittsburgh, PA

    Steven A. Shafer & Takeo Kanade

Authors
  1. Gudrun J. Klinker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steven A. Shafer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Takeo Kanade
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Klinker, G.J., Shafer, S.A. & Kanade, T. A physical approach to color image understanding. Int J Comput Vision 4, 7–38 (1990). https://doi.org/10.1007/BF00137441

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00137441

Keywords

Search

Navigation