International Journal of Computer Vision

, Volume 21, Issue 3, pp 163–186 | Cite as

Separation of Reflection Components Using Color and Polarization

  • Shree K. Nayar
  • Xi-Sheng Fang
  • Terrance Boult


Specular reflections and interreflections produce strong highlights in brightness images. These highlights can cause vision algorithms for segmentation, shape from shading, binocular stereo, and motion estimation to produce erroneous results. A technique is developed for separating the specular and diffuse components of reflection from images. The approach is to use color and polarization information, simultaneously, to obtain constraints on the reflection components at each image point. Polarization yields local and independent estimates of the color of specular reflection. The result is a linear subspace in color space in which the local diffuse component must lie. This subspace constraint is applied to neighboring image points to determine the diffuse component. In contrast to previous separation algorithms, the proposed method can handle highlights on surfaces with substantial texture, smoothly varying diffuse reflectance, and varying material properties. The separation algorithm is applied to several complex scenes with textured objects and strong interreflections. The separation results are then used to solve three problems pertinent to visual perception; determining illumination color, estimating illumination direction, and shape recovery.


Motion Estimation Image Point Specular Reflection Shape Recovery Diffuse Component 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Batchelor, B.G., Hill, D.A., and Hodgson, D.C. 1985. Automated Visual Inspection. IFS Publications Ltd.: Bedford, England.Google Scholar
  2. Bajcsy, R., Lee, S.W., and Leonardis, A. 1990. Color image segmentation with detection of highlights and local illumination induced by interreflections. Proc. of Intl. Conf. on Pattern Recognition (ICPR), Atlantic City, NJ.Google Scholar
  3. Beckmann, P. and Spizzichino, A. 1963. The Scattering of Electromagnetic Waves from Rough Surfaces. Pergamon: New York.Google Scholar
  4. Born, M. and Wolf, E. 1965. Principles of Optics. Pergamon: London.Google Scholar
  5. Boult, T.E. and Wolff, L.B. 1991. Physically based edge labeling. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Maui, Hawaii, pp. 656–663.Google Scholar
  6. Brelstaff, G. and Blake, A. 1992. Detecting specular reflections using Lambertian constraints. Proc. of Intl. Conf. on Computer Vision (ICCV), Tampa, Florida, pp. 297–302.Google Scholar
  7. Gershon, R. 1987. The use of color in computational vision. Ph.D. thesis, University of Toronto.Google Scholar
  8. Handbook of Laser Science and Technology. M.J. Marvin (Ed.), CRC Press Inc., Vol. 4, 1988.Google Scholar
  9. Handbook of Optics. W.G. Driscoll (Ed.), McGraw Hill Inc., 1978.Google Scholar
  10. Klinker, G.J. 1993. A Physical Approach to Color Image Understanding. A.K. Peters, Wellesley: Massachusetts.Google Scholar
  11. Klinker, G.J., Shafer, S.A., and Kanade, T. 1990. The measurement of highlights in color images. International Journal of Computer Vision, 2(1):7–32.Google Scholar
  12. Koshikawa, K. 1979. A polarimetric approach to shape understanding. Proc. of Intl. Joint Conf. on Artificial Intelligence (IJCAI), pp. 493–495.Google Scholar
  13. Lee, S.W. 1991. Understanding of surface reflection in computer vision by color and multiple views. Ph.D. thesis, University of Pennsylvania.Google Scholar
  14. Mersch, S. 1984. Polarized lighting for machine vision applications. Proc. of RI/SME Third Annual Applied Machine Vision Conf., Schaumburg, pp. 40–54.Google Scholar
  15. Nayar, S.K., Ikeuchi, K., and Kanade, T. 1991. Surface reflection: Physical and geometrical perspectives. IEEE Trans. on Pattern Analysis and Machine Intelligence, 13(7):611–634.Google Scholar
  16. Novak, C. and Shafer, S. 1992. Anatomy of a color histogram. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 599–605.Google Scholar
  17. Oren, M. and Nayar, S.K. 1994. Seeing beyond Lambert's law. Proc. Third European Conference on Computer Vision, 2:269–280.Google Scholar
  18. Polarization and Remote Sensing. W.G. Egan (Ed.), SPIE, Vol. 1747, July, 1992.Google Scholar
  19. Sato, Y. and Ikeuchi, K. 1993. Temporal-color space analysis of reflection. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 570–576.Google Scholar
  20. Shafer, S. 1985. Using color to separate reflection components. Color Research and Applications, 10:210–218.Google Scholar
  21. Shafer, S., Kanade, T., Klinker, G.J., and Novak, C.L. 1993. Physics-based models for early vision by machine. Proc. SPIE Conf. on Perceiving, Measuring and Using Color, vol. 1250.Google Scholar
  22. Torrance, K. and Sparrow, E. 1967. Theory for off-specular reflection from roughened surfaces. Journal of the Optical Society of America, 57:1105–1114.Google Scholar
  23. Wolff, L.B. and Boult, T.E. 1991. Constraining object features using a polarization reflectance model. IEEE Trans. on Pattern Analysis and Machine Intelligence, 13(7):635–657.Google Scholar
  24. Wolff, L.B. 1992. A diffuse reflectance model for dielectric surfaces. Proc. SPIE Conf. on Optics, Illumination, and Image Sensing for Machine Vision, vol. 1822, pp. 60–73.Google Scholar
  25. Woodham, R.J. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering, 19(1):139–144.Google Scholar

Copyright information

© Kluwer Academic Publishers 1997

Authors and Affiliations

  • Shree K. Nayar
  • Xi-Sheng Fang
  • Terrance Boult

There are no affiliations available

Personalised recommendations