International Journal of Computer Vision

, Volume 52, Issue 1, pp 25–43 | Cite as

Observing Shape from Defocused Images

  • Paolo Favaro
  • Andrea Mennucci
  • Stefano Soatto
Article

Abstract

Accommodation cues are measurable properties of an image that are associated with a change in the geometry of the imaging device. To what extent can three-dimensional shape be reconstructed using accommodation cues alone? This question is fundamental to the problem of reconstructing shape from focus (SFF) and shape from defocus (SFD) for applications in inspection, microscopy, image restoration and visualization. We address it by studying the “observability” of accommodation cues in an analytical framework that reveals under what conditions shape can be reconstructed from defocused images. We do so in three steps: (1) we characterize the observability of any surface in the presence of a controlled radiance (“weak observability”), (2) we conjecture the existence of a radiance that allows distinguishing any two surfaces (“sufficient excitation”) and (3) we show that in the absence of any prior knowledge on the radiance, two surfaces can be distinguished up to the degree of resolution determined by the complexity of the radiance (“strong observability”). We formulate the problem of reconstructing the shape and radiance of a scene as the minimization of the information divergence between blurred images, and propose an algorithm that is provably convergent and guarantees that the solution is admissible, in the sense of corresponding to a positive radiance and imaging kernel.

shape surface geometry low-level vision shape from defocus shape from focus 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Asada, N., Fujiwara, H., and Matsuyama, T. 1998. Seeing behind the scene: Analysis of photometric properties of occluding edges by reversed projection blurring model. IEEE Trans. Pattern Analysis and Machine Intelligence, 20:155–167.Google Scholar
  2. Chan, T.F. and Wong, C.K. 1998. Total variation blind deconvolution. IEEE Transactions on Image Processing, 7(3):370–375.Google Scholar
  3. Chaudhuri, S. and Rajagopalan, A.N. 1999. Depth from Defocus: A Real Aperture Imaging Approach. Springer-Verlag: Berlin.Google Scholar
  4. Cover, T.M. and Thomas, J.A. 1991. Elements of Information Theory.Wiley Interscience.Google Scholar
  5. Csiszár, I. 1991. Why least squares and maximum entropy? An axiomatic approach to inverse problems. Ann. of Stat., 19:2033– 2066.Google Scholar
  6. Ens, J. and Lawrence, P. 1993. An investigation of methods for determining depth from focus. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(2):97–108.Google Scholar
  7. Farid, H. and Simoncelli, E.P. 1998. Range estimation by optical differentiation. Journal of the Optical Society of America A, 15(7):1777–1786.Google Scholar
  8. Favaro, P. and Soatto, S. 2000. Shape and radiance estimation from the information divergence of blurred images. In Proc. European Conference on Computer Vision, 1:755–768.Google Scholar
  9. Friedlander, G. and Joshi, M. 1998. Introduction to the Theory of Distributions. Cambridge University Press: Cambridge.Google Scholar
  10. Girod, B. and Scherock, S. 1989. Depth from focus of structured light. In SPIE, pp. 209–215.Google Scholar
  11. Gokstorp, M. 1994. Computing depth from out-of-focus blur using a local frequency representation. In International Conference on Pattern Recognition, Vol. A, pp. 153–158.Google Scholar
  12. Harikumar, G. and Bresler, Y. 1999. Perfect blind restoration of images blurred by multiple filters: Theory and efficient algorithms. IEEE Transactions on Image Processing, 8(2):202–219.Google Scholar
  13. Hopkins, H.H. 1955. The frequency response of a defocused optical system. Proc. R. Soc. London Ser. A, 231:91–103.Google Scholar
  14. Hwang, T.L., Clark, J.J., and Yuille, A.L. 1989. A depth recovery algorithm using defocus information. In Computer Vision and Pattern Recognition, pp. 476–482.Google Scholar
  15. Kalifa, J., Mallat, S., and Rouge, B. 1998. Image deconvolution in mirror wavelet bases. In Proc. International Conference on Image Processing, p. 98.Google Scholar
  16. Kundur, D. and Hatzinakos, D. 1998. A novel blind deconvolution scheme for image restoration using recursive filtering. IEEE Transactions on Signal Processing, 46(2):375–390.Google Scholar
  17. Luemberger, D. 1968. Optimization by Vector Space Methods.Wiley: New York.Google Scholar
  18. Marshall, J.A., Burbeck, C.A., Ariely, D., Rolland, J.P., and Martin, K.E. 1996. Occlusion edge blur: A cue to relative visual depth. Journal of the Optical Society of America A, 13(4):681–688.Google Scholar
  19. Mennucci, A. and Soatto, S. 1999. The accommodation cue, part 1: Modeling. Essrl Technical Report 99-001, Washington University.Google Scholar
  20. Murase, H. and Nayar, S. 1995. Visual learning and recognition or 3d object from appearance. Intl. J. Computer Vision, 14(1):5–24.Google Scholar
  21. Nair, H.N. and Stewart, C.V. 1992. Robust focus ranging. In Computer Vision and Pattern Recognition, pp. 309–314.Google Scholar
  22. Namba, M. and Ishida, Y. 1998. Wavelet transform domain blind deconvolution. Signal Processing, 68(1):119–124.Google Scholar
  23. Nayar, S.K. and Nakagawa, Y. 1990. Shape from focus: An effective approach for rough surfaces. In IEEE International Conference on Robotics and Automation, pp. 218–225.Google Scholar
  24. Nayar, S.K., Watanabe, M., and Noguchi, M. 1995. Real-time focus range sensor. In Proc. International Conference on Computer Vision, pp. 995–1001.Google Scholar
  25. Neelamani, R., Choi, H., and Baraniuk, R. 1999. Wavelet-domain regularized deconvolution for ill-conditioned systems. In Proc.International Conference on Image Processing, pp. 58–72.Google Scholar
  26. Noguchi, M. and Nayar, S.K. 1994. Microscopic shape from focus using active illumination. In International Conference on Pattern Recognition, pp. 147–152.Google Scholar
  27. Pentland, A. 1987. A new sense for depth of field. IEEE Trans.Pattern Anal. Mach. Intell., 9:523–531.Google Scholar
  28. Pentland, A., Darrell, T., Turk, M., and Huang, W. 1989. A simple, real-time range camera. In Computer Vision and Pattern Recognition, pp. 256–261.Google Scholar
  29. Pentland, A., Scherock, S., Darrell, T., and Girod, B. 1994. Simple range cameras based on focal error. Journal of the Optical Society of America A, 11(11):2925–2934.Google Scholar
  30. Rajagopalan, A.N. and Chaudhuri, S. 1995. A block shift-variant blur model for recovering depth from defocused images. In Proc. International Conference on Image Processing, pp. 636– 639.Google Scholar
  31. Rajagopalan, A.N. and Chaudhuri, S. 1997. Optimal selection of camera parameters for recovery of depth from defocused images.In Computer Vision and Pattern Recognition, pp. 219– 224.Google Scholar
  32. Rajagopalan, A.N. and Chaudhuri, S. 1998. Optimal recovery of depth from defocused images using an mrf model. In Proc. International Conference on Computer Vision, pp. 1047–1052.Google Scholar
  33. Schechner, Y.Y. and Kiryati, N. 1999. The optimal axial interval in estimating depth from defocus. In IEEE Proc. International Conference on Computer Vision, Vol. II, pp. 843–848.Google Scholar
  34. Schechner, Y.Y., Kiryati, N., and Basri, R. 1998. Separation of transparent layers using focus. In IEEE Proc. International Conference on Computer Vision, pp. 1061–1066.Google Scholar
  35. Scherock, S. 1981. Depth from focus of structured light. In Technical Report-167, Media-Lab, MIT.Google Scholar
  36. Schneider, G., Heit, B., Honig, J., and Bremont, J. 1994. Monocular depth perception by evaluation of the blur in defocused images.In Proc. International Conference on Image Processing, Vol. 2, pp. 116–119.Google Scholar
  37. Simoncelli, E.P. and Farid, H. 1996. Direct differential range estimation using optical masks. In European Conference on Computer Vision, Vol. II, pp. 82–93.Google Scholar
  38. Snyder, D.L., Schulz, T.J., and O'Sullivan, J.A. 1992. Deblurring subject to nonnegativity constraints. IEEE Transactions on Signal Processing, 40(5):1142–1150.Google Scholar
  39. Soatto, S. and Favaro, P. 2000. A geometric approach to blind deconvolution with application to shape from defocus. Proc. IEEE Computer Vision and Pattern Recognition, 2:10–17.Google Scholar
  40. Subbarao, M. and Surya, G. 1994. Depth from defocus: A spatial domain approach. International Journal of Computer Vision, 13(3):271–294.Google Scholar
  41. Subbarao, M. and Wei, T.C. 1992. Depth from defocus and rapid autofocusing: A practical approach. In Computer Vision and Pattern Recognition, pp. 773–776Google Scholar
  42. Taylor, M. 1996. Partial Differential Equations (volume i: Basic Theory). Springer Verlag: Berlin.Google Scholar
  43. Watanabe, M. and Nayar, S.K. 1996a. Minimal operator set for passive depth from defocus. In Computer Vision and Pattern Recognition, pp. 431–438.Google Scholar
  44. Watanabe, M. and Nayar, S.K. 1996b. Telecentric optics for computational vision. In European Conference on Computer Vision, Vol. II, pp. 439–445.Google Scholar
  45. Xiong, Y. and Shafer, S.A. 1995. Moment filters for high precision computation of focus and stereo. In Proc. of International Conference on Intelligent Robots and Systems, pp. 108–113.Google Scholar
  46. Ziou, D. 1998. Passive depth from defocus using a spatial domain approach. In Proc. International Conference on Computer Vision, pp. 799–804.Google Scholar

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Paolo Favaro
    • 1
  • Andrea Mennucci
    • 2
  • Stefano Soatto
    • 3
  1. 1.Electrical Engineering DepartmentWashington University in St. LouisSt. Louis
  2. 2.Scuola Normale SuperiorePisaItaly
  3. 3.Computer Science DepartmentUniversity of California, Los AngelesLos Angeles

Personalised recommendations