International Journal of Computer Vision

, Volume 17, Issue 3, pp 219–240 | Cite as

Reflectance based object recognition

  • Shree K. Nayar
  • Ruud M. Bolle


Neighboring points on a smoothly curved surface have similar surface normals and illumination conditions. Therefore, their brightness values can be used to compute the ratio of their reflectance coefficients. Based on this observation, we develop an algorithm that estimates a reflectance ratio for each region in an image with respect to its background. The algorithm is efficient as it computes ratios for all image regions in just two raster scans. The region reflectance ratio represents a physical property that is invariant to illumination and imaging parameters. Several experiments are conducted to demonstrate the accuracy and robustness of ratio invariant.

The ratio invariant is used to recognize objects from a single brightness image of a scene. Object models are automatically acquired and represented using a hash table. Recognition and pose estimation algorithms are presented that use ratio estimates of scene regions as well as their geometric properties to index the hash table. The result is a hypothesis for the existence of an object in the image. This hypothesis is verified using the ratios and locations of other regions in the scene. This approach to recognition is effective for objects with printed characters and pictures. Recognition experiments are conducted on images with illumination variations, occlusions, and shadows. The paper is concluded with a discussion on the simultaneous use of reflectance and geometry for visual perception.


object representation physical properties retinex theory reflectance ratio photometric invariant region ratios indexing model acquisition object recognition pose estimation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aho, A.V., Hopcroft, J.E., and Ullman, J.D. 1974. The Design and Analysis of Computer Algorithms. Addison-Wesley: Reading, MA.Google Scholar
  2. Ballard, D.H. and Brown, C.M. 1982. Computer Vision. Prentice Hall.Google Scholar
  3. Besl, P.J. and Jain, R.C. 1985. Three-dimensional object recognition. ACM Computing Surveys, 17(1):75–145.Google Scholar
  4. Blake, A. 1985. Boundary conditions for lightness computation in Mondrian world. Computer Vision, Graphics, and Image Processing, 32:314–327.Google Scholar
  5. Born, A. and Wolf, E. 1965. Principles of Optics. Pergamon: London.Google Scholar
  6. Burns, J.B., Weiss, R., and Riseman, E. 1990. View variation of point-set and line segment features. In Proc. Image Understanding Workshop, pp. 650–659.Google Scholar
  7. Califano, A. and Mohan, R. 1991. Multidimensional indexing for recognizing visual shapes. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 28–34.Google Scholar
  8. Chin, R.T. and Dyer, C.R. 1986. Model-based recognition in robot vision. ACM Computing Surveys, 18(1):67–108.Google Scholar
  9. Finlayson, G.D. 1992. Colour Object Recognition. M.S. Thesis, Simon Fraser University.Google Scholar
  10. Forsyth, D. and Zisserman, A. 1989. Mutual Illumination. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 466–473.Google Scholar
  11. Gilchrist, A.L. 1979. The perception of surfaces blacks and whites. Scientific American, pp. 112–124.Google Scholar
  12. Gray, S.B. 1971. Local properties of binary images in two dimensions. IEEE Trans. on Computers, 20(5):551–561.Google Scholar
  13. Healey, G.E., Shafer, S.A., and Wolff, L.b. (Eds.) 1992. Physics Based Vision: Color, Jones and Bartlett.Google Scholar
  14. Horn, B.K.P. 1974. Determining lightness from an image. Computer Graphics and Image Processing, 3(1):277–299.Google Scholar
  15. Horn, B.K.P. 1986. Robot Vision. The MIT Press: Cambridge, MA.Google Scholar
  16. Hurlbert, A.C. 1989. The computation of color. Ph.D. thesis, Massachusetts Institute of Technology.Google Scholar
  17. Huttenlocher, D.P. and Ullman, S. 1990. Recognizing solid objects by alignment with an image. International Journal of Computer Vision, 5(2):195–212.Google Scholar
  18. Koenderink, J.J. and vanDoorn, A.J. 1980. Photometric invariants related to solid shape. Optica Acta, 27(7):981–996.Google Scholar
  19. Koenderink, J.J. and vanDoorn, A.J. 1983. Geometrical modes as a general method to treat diffuse interreflections in radiometry. Journal of Optical Society of America, 73(6):843–850.Google Scholar
  20. Lambdam, Y. and Wolfson, H.J. 1988. Geometric hashing: A general and efficient model-based recognition scheme. In Proc. International Conference on Computer Vision, pp. 238–249.Google Scholar
  21. Land, E.H. 1964. The retinex. American Scientist, 52(2):247–264.Google Scholar
  22. Land, E.H. and McCann, J.J. 1971. Lightness and retinex theory. Journal of Optical Society of America, 61(1):1–11.Google Scholar
  23. Land, E.H. 1983. Recent advances in retinex theory and some implications for cortical computations: Color vision and the natural image. Proc. National Academy of Science, 80:5163–5169.Google Scholar
  24. Land, E.H. 1986a. An alternative technique for the computation of the designator in the retinex theory of color vision. Proc. National Academy of Science, 83:3078–3080.Google Scholar
  25. Land, E.H. 1986b. Recent advances in retinex theory. Vision Research, 26:7–21.Google Scholar
  26. Lumia, R., Shapiro, L., and Zuniga, O. 1983. A new connected components algorithm for virtual memory computers. Computer Vision, Graphics, and Image Processing, 22:287–300.Google Scholar
  27. Nagy, G. 1969. Feature Extraction on Binary Patterns. IEEE Trans. on Systems, Science, and Cybernetics, 5(4):273–278.Google Scholar
  28. Nayar, S.K. and Bolle, R.M. 1993. Computing reflectance ratios from an image. Pattern Recognition, 26(10):1529–1542.Google Scholar
  29. Nayar, S.K., Ikeuchi, K., and Kanade, T. 1990. Determining shape and reflectance of hybrid surfaces by photometric sampling. IEEE Transactions on Robotics and Automation, 6(4): 418–431.Google Scholar
  30. Nayar, S.K., Ikeuchi, K. and Kanade, T. 1991a. Shape from Interreflections. International Journal of Computer Vision, 6(3):173–195.Google Scholar
  31. Nayar, S.K., Ikeuchi, K., and Kanade, T. 1991b. Surface reflection: Physical and geometrical perspective, IEEE Trans. on Pattern Analysis and Machine Intell., 13(7):611–634.Google Scholar
  32. Nayar, S.K., Fang, X., and Boult, T.E. 1993. Removal of specularities using color and polarization. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 583–590.Google Scholar
  33. Nicol, C.J. 1993. A systolic architecture for labeling the connected components of multi-valued images in real time. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 136–141.Google Scholar
  34. Oren, M. and Nayar, S.K. 1993. Generalization of the Lambertian Model. In Proc. Image Understanding Workshop, pp. 1037–1048.Google Scholar
  35. Rosenfeld, A. and Pfaltz, J.L. 1966. Sequential operations in digital picture processing. ACM, 13:471–494.Google Scholar
  36. Sinha, P. and Adelson, E. 1993. Recovering reflectance and illumination in a world of painted polyhedra. In Proc. International Conference on Computer Vision.Google Scholar
  37. Strickler, M.A. 1992. Color and geometry as cues for indexing. Technical Report CS92-22, University of Chicago.Google Scholar
  38. Swain, M.J. and Ballard, D.H. 1991. Color Indexing. International Journal of Computer Vision, pp. 11–32.Google Scholar
  39. Torrance, K. and Sparrow, E. 1967. Theory for off-specular reflection from rough surfaces. Journal of Optical Society of America, 57:1057–1114.Google Scholar
  40. Weinshall, D. 1991. Model-based invariants for 3d vision. Technical Report RC 17705, IBM Thomas J. Watson Research Center.Google Scholar

Copyright information

© Kluwer Academic Publishers 1996

Authors and Affiliations

  • Shree K. Nayar
    • 1
  • Ruud M. Bolle
    • 2
  1. 1.Department of Computer ScienceColumbia UniversityNew York
  2. 2.Exploratory Computer Vision GroupIBM T.J. Watson Research CenterYorktown Heights

Personalised recommendations