International Journal of Computer Vision

, Volume 4, Issue 1, pp 7–38 | Cite as

A physical approach to color image understanding

  • Gudrun J. Klinker
  • Steven A. Shafer
  • Takeo Kanade
Article

Abstract

In this paper, we present an approach to color image understanding that can be used to segment and analyze surfaces with color variations due to highlights and shading. The work is based on a theory—the Dichromatic Reflection Model—which describes the color of the reflected light as a mixture of light from surface reflection (highlights) and body reflection (object color). In the past, we have shown how the dichromatic theory can be used to separate a color image into two intrinsic reflection images: an image of just the highlights, and the original image with the highlights removed. At that time, the algorithm could only be applied to hand-segmented images. This paper shows how the same reflection model can be used to include color image segmentation into the image analysis. The result is a color image understanding system, capable of generating physical descriptions of the reflection processes occurring in the scene. Such descriptions include the intrinsic reflection images, an image segmentation, and symbolic information about the object and highlight colors. This line of research can lead to physicsbased image understanding methods that are both more reliable and more useful than traditional methods.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    D.H. Ballard and C.M. Brown, Computer Vision, Englewood Cliffs, NJ: Prentice-Hall, 1982.Google Scholar
  2. 2.
    H.G. Barrow and J.M. Tenenbaum, “Recovering intrinsic scene characteristics from images, Computer Vision Systems. New York: Academic Press, pp. 3–26, 1978.Google Scholar
  3. 3.
    R.L. Cook and K.E. Torrance, “A reflectance model for computer graphics,” ACM Trans. on Graphics 1 (1):7–24, 1982 (Also published in Computer Graphics 15(3), SIGGRAPH'81).Google Scholar
  4. 4.
    M. D'Zmura and P. Lennie, “Mechanisms of color constaney,” J. Opt. Soc. Amer. A 3(10):1662–1672, 1986.Google Scholar
  5. 5.
    L. Dreschler and H.-H. Nagel, “Volumetric model and 3D Trajectory of a moving car derived from monocular TV frame sequences of a street scene,” Comput. Graphics Image Process. 20:199–228, 1982.Google Scholar
  6. 6.
    R. Gershon, “The use of color in computational vision,” Ph.D. thesis, Department of Computer Science, University of Toronto, 1987.Google Scholar
  7. 7.
    R. Gershon, A.D. Jepson, and J.K. Tsotsos, “Highlight identification using chromatic information,” Proc. 1st Intern. Conf. Computer Vision, pp. 161–171. IEEE, London, June 1987.Google Scholar
  8. 8.
    H. Grassmann, “On the theory of compound colors,” Phil. Mag., April 1854.Google Scholar
  9. 9.
    A.R. Hanson and E.M. Riseman, “VISIONS: A computer system for interpreting scenes,” In A.R. Hanson and E.M. Riseman (eds.), Computer Vision Systems, New York: Academic Press, pp. 303–333, 1978.Google Scholar
  10. 10.
    H.H. Harman, Modern Factor Analysis, 2nd. ed. Chicago: University of Chicago Press, 1967.Google Scholar
  11. 11.
    G. Healey and T.O. Binford, “Local shape from specularity.” In L.S. Bauman (ed.), DARPA-Image Understanding (IUS) Workshop, pp. 874–887. Los Angeles, CA, February 1987.Google Scholar
  12. 12.
    G. Healey and T.O. Binford, “Local shape from specularity,” Proc. 1st Intern. Conf. Computer Vision, pp. 151–161, IEEE, London, June 1987.Google Scholar
  13. 13.
    G. Healey and T.O. Binford, “The role and use of color in a general vision system.” In L.S. Bauman (ed.), DARPA-Image Understanding (IUS) Workshop, pp. 599–613. Los Angeles, CA, February 1987.Google Scholar
  14. 14.
    B.K.P. Horn, “Understanding image intensities,” Artificial Intelligence 8(11):201–231, 1977.Google Scholar
  15. 15.
    B.K.P. Horn, “Exact reproduction of colored images,” Comput. Vision, Graphics, Image Process. 26:135–167, 1984.Google Scholar
  16. 16.
    B.K.P. Horn and B.G. Schunk, “Determining optical flow,” Artificial Intelligence 17:185–203, 1981.Google Scholar
  17. 17.
    T. Kanade, “Region segmentation: Signal vs. semantics.” Proc. 4th Intern. Joint Conf. Pattern Recog. pp. 95–105, IEEE, Kyoto, Japan, November 1978.Google Scholar
  18. 18.
    J.R. Kender, “Shape from texture.” Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, November 1980. Appeared also as technical report CMU-CS-81-102, Computer Science Department, Carnegie-Mellon University, Pittsburgh, PA.Google Scholar
  19. 19.
    J.R. Kender and E.M. Smith, “Shape from darkness: Deriving surface information from dynamic shadows,” Proc. 1st Intern. Conf. Computer Vision, pp. 539–546. IEEE, London, June 1987.Google Scholar
  20. 20.
    G.J. Klinker, “A physical approach to color image understanding.” Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, May 1988. Available as technical report CMU-CS-88-161.Google Scholar
  21. 21.
    G.J. Klinker, S.A. Shafer, and T. Kanade, “Using a color reflection model to separate highlights from object color,” Proc. 1st Intern. Conf. Comp. Vision, pp. 145–150, IEEE, London, June 1987.Google Scholar
  22. 22.
    G.J. Klinker, S.A. Shafer, and T. Kanade, “The measurement of highlights in color images,” Intern. J. Comput. Vision 2(1):7–32, 1988.Google Scholar
  23. 23.
    H.-C. Lee, “Method for computing the scene-illuminant chromaticity from specular highlights,” J. Opt. Soc. Amer. 3(10):1694–1699, 1986.Google Scholar
  24. 24.
    L.T. Maloney and B.A. Wandell, “Color constancy: A method for recovering surface spectral reflectance,” J. Opt. Soc. Amer. A 3(1):29–33, 1986.Google Scholar
  25. 25.
    R. Ohlander, K. Price, and D.R. Reddy, “Picture segmentation using a recursive region splitting method,” Comput. Graphics Image Process. 8:313–333, 1978.Google Scholar
  26. 26.
    Y. Ohta, T. Kanade, and T. Sakai, “Color information for region segmentation,” Comput. Graphics Image Process. 13:222–231, 1980.Google Scholar
  27. 27.
    A. Rosenfeld and A.C. Kak, Digital Picture Processing, 2nd ed. New York: Academic Press, 1982.Google Scholar
  28. 28.
    J.M. Rubin and W.A. Richards, “Color vision and image intensities: When are changes material?” Biological Cybernetics 45:215–226, 1982.Google Scholar
  29. 29.
    S.A. Shafer, “Describing light mixtures through linear algebra,” J. Opt. Soc. Amer. 72(2):299–300, 1982.Google Scholar
  30. 30.
    S.A. Shafer, “Using color to separate reflection components,” COLOR research and application 10(4):210–218, 1985. Also available as technical report TR-136, Computer Science Department, University of Rochester, NY, April 1984.Google Scholar
  31. 31.
    S.A. Shafer and T. Kanade, “Recursive region segmentation by analysis of histograms,” Proc. Intern. Conf. Acoustics, Speech, and Signal Process., pp. 1166–1171, IEEE, Paris, France, May, 1982.Google Scholar
  32. 32.
    S.A. Shafer and T. Kanade, “Using shadows in finding surface orientations,” Comput. Vision. Graphics, Image Process. 22:145–176, 1983.Google Scholar
  33. 33.
    C.E. Thorpe, “FIDO: Vision and navigation for a robot rover.” Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, December 1984. Available as technical report CMU-CS-84-168.Google Scholar
  34. 34.
    F. Tong and B.V. Funt, “Specularity removal for shape from shading,” Proc. Conf. Vision Interface, Edmonton, Alberta, Canada, 1988.Google Scholar
  35. 35.
    S. Ullman, “The interpretation of structure from motion,” Al Memo 476, MIT AI Laboratory, Cambridge, MA, October 1976.Google Scholar

Copyright information

© Kluwer Academic Publishers 1990

Authors and Affiliations

  • Gudrun J. Klinker
    • 1
  • Steven A. Shafer
    • 2
  • Takeo Kanade
    • 2
  1. 1.Cambridge Research LabDigital Equipment CorporationCambridge
  2. 2.Computer Science DepartmentCarnegie Mellon UniversityPittsburgh

Personalised recommendations