In this paper, we present an approach to color image understanding that can be used to segment and analyze surfaces with color variations due to highlights and shading. The work is based on a theory—the Dichromatic Reflection Model—which describes the color of the reflected light as a mixture of light from surface reflection (highlights) and body reflection (object color). In the past, we have shown how the dichromatic theory can be used to separate a color image into two intrinsic reflection images: an image of just the highlights, and the original image with the highlights removed. At that time, the algorithm could only be applied to hand-segmented images. This paper shows how the same reflection model can be used to include color image segmentation into the image analysis. The result is a color image understanding system, capable of generating physical descriptions of the reflection processes occurring in the scene. Such descriptions include the intrinsic reflection images, an image segmentation, and symbolic information about the object and highlight colors. This line of research can lead to physicsbased image understanding methods that are both more reliable and more useful than traditional methods.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
D.H. Ballard and C.M. Brown, Computer Vision, Englewood Cliffs, NJ: Prentice-Hall, 1982.
H.G. Barrow and J.M. Tenenbaum, “Recovering intrinsic scene characteristics from images, Computer Vision Systems. New York: Academic Press, pp. 3–26, 1978.
R.L. Cook and K.E. Torrance, “A reflectance model for computer graphics,” ACM Trans. on Graphics 1 (1):7–24, 1982 (Also published in Computer Graphics 15(3), SIGGRAPH'81).
M. D'Zmura and P. Lennie, “Mechanisms of color constaney,” J. Opt. Soc. Amer. A 3(10):1662–1672, 1986.
L. Dreschler and H.-H. Nagel, “Volumetric model and 3D Trajectory of a moving car derived from monocular TV frame sequences of a street scene,” Comput. Graphics Image Process. 20:199–228, 1982.
R. Gershon, “The use of color in computational vision,” Ph.D. thesis, Department of Computer Science, University of Toronto, 1987.
R. Gershon, A.D. Jepson, and J.K. Tsotsos, “Highlight identification using chromatic information,” Proc. 1st Intern. Conf. Computer Vision, pp. 161–171. IEEE, London, June 1987.
H. Grassmann, “On the theory of compound colors,” Phil. Mag., April 1854.
A.R. Hanson and E.M. Riseman, “VISIONS: A computer system for interpreting scenes,” In A.R. Hanson and E.M. Riseman (eds.), Computer Vision Systems, New York: Academic Press, pp. 303–333, 1978.
H.H. Harman, Modern Factor Analysis, 2nd. ed. Chicago: University of Chicago Press, 1967.
G. Healey and T.O. Binford, “Local shape from specularity.” In L.S. Bauman (ed.), DARPA-Image Understanding (IUS) Workshop, pp. 874–887. Los Angeles, CA, February 1987.
G. Healey and T.O. Binford, “Local shape from specularity,” Proc. 1st Intern. Conf. Computer Vision, pp. 151–161, IEEE, London, June 1987.
G. Healey and T.O. Binford, “The role and use of color in a general vision system.” In L.S. Bauman (ed.), DARPA-Image Understanding (IUS) Workshop, pp. 599–613. Los Angeles, CA, February 1987.
B.K.P. Horn, “Understanding image intensities,” Artificial Intelligence 8(11):201–231, 1977.
B.K.P. Horn, “Exact reproduction of colored images,” Comput. Vision, Graphics, Image Process. 26:135–167, 1984.
B.K.P. Horn and B.G. Schunk, “Determining optical flow,” Artificial Intelligence 17:185–203, 1981.
T. Kanade, “Region segmentation: Signal vs. semantics.” Proc. 4th Intern. Joint Conf. Pattern Recog. pp. 95–105, IEEE, Kyoto, Japan, November 1978.
J.R. Kender, “Shape from texture.” Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, November 1980. Appeared also as technical report CMU-CS-81-102, Computer Science Department, Carnegie-Mellon University, Pittsburgh, PA.
J.R. Kender and E.M. Smith, “Shape from darkness: Deriving surface information from dynamic shadows,” Proc. 1st Intern. Conf. Computer Vision, pp. 539–546. IEEE, London, June 1987.
G.J. Klinker, “A physical approach to color image understanding.” Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, May 1988. Available as technical report CMU-CS-88-161.
G.J. Klinker, S.A. Shafer, and T. Kanade, “Using a color reflection model to separate highlights from object color,” Proc. 1st Intern. Conf. Comp. Vision, pp. 145–150, IEEE, London, June 1987.
G.J. Klinker, S.A. Shafer, and T. Kanade, “The measurement of highlights in color images,” Intern. J. Comput. Vision 2(1):7–32, 1988.
H.-C. Lee, “Method for computing the scene-illuminant chromaticity from specular highlights,” J. Opt. Soc. Amer. 3(10):1694–1699, 1986.
L.T. Maloney and B.A. Wandell, “Color constancy: A method for recovering surface spectral reflectance,” J. Opt. Soc. Amer. A 3(1):29–33, 1986.
R. Ohlander, K. Price, and D.R. Reddy, “Picture segmentation using a recursive region splitting method,” Comput. Graphics Image Process. 8:313–333, 1978.
Y. Ohta, T. Kanade, and T. Sakai, “Color information for region segmentation,” Comput. Graphics Image Process. 13:222–231, 1980.
A. Rosenfeld and A.C. Kak, Digital Picture Processing, 2nd ed. New York: Academic Press, 1982.
J.M. Rubin and W.A. Richards, “Color vision and image intensities: When are changes material?” Biological Cybernetics 45:215–226, 1982.
S.A. Shafer, “Describing light mixtures through linear algebra,” J. Opt. Soc. Amer. 72(2):299–300, 1982.
S.A. Shafer, “Using color to separate reflection components,” COLOR research and application 10(4):210–218, 1985. Also available as technical report TR-136, Computer Science Department, University of Rochester, NY, April 1984.
S.A. Shafer and T. Kanade, “Recursive region segmentation by analysis of histograms,” Proc. Intern. Conf. Acoustics, Speech, and Signal Process., pp. 1166–1171, IEEE, Paris, France, May, 1982.
S.A. Shafer and T. Kanade, “Using shadows in finding surface orientations,” Comput. Vision. Graphics, Image Process. 22:145–176, 1983.
C.E. Thorpe, “FIDO: Vision and navigation for a robot rover.” Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, December 1984. Available as technical report CMU-CS-84-168.
F. Tong and B.V. Funt, “Specularity removal for shape from shading,” Proc. Conf. Vision Interface, Edmonton, Alberta, Canada, 1988.
S. Ullman, “The interpretation of structure from motion,” Al Memo 476, MIT AI Laboratory, Cambridge, MA, October 1976.
About this article
Cite this article
Klinker, G.J., Shafer, S.A. & Kanade, T. A physical approach to color image understanding. Int J Comput Vision 4, 7–38 (1990). https://doi.org/10.1007/BF00137441