Skip to main content
SpringerLink
Log in

Programmable Imaging: Towards a Flexible Camera

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this paper, we introduce the notion of a programmable imaging system. Such an imaging system provides a human user or a vision system significant control over the radiometric and geometric characteristics of the system. This flexibility is achieved using a programmable array of micro-mirrors. The orientations of the mirrors of the array can be controlled with high precision over space and time. This enables the system to select and modulate rays from the scene’s light field based on the needs of the application at hand.

We have implemented a programmable imaging system that uses a digital micro-mirror device (DMD), which is used in digital light processing. Although the mirrors of this device can only be positioned in one of two states, we show that our system can be used to implement a wide variety of imaging functions, including, high dynamic range imaging, feature detection, and object recognition. We also describe how a micro-mirror array that allows full control over the orientations of its mirrors can be used to instantly change the field of view and resolution characteristics of the imaging system. We conclude with a discussion on the implications of programmable imaging for computer vision.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Castracane, J. and Gutin, M. A. 1999. DMD-based bloom control for intensified imaging systems. In Diffractive and Holographic Tech., Syst., and Spatial Light Modulators VI, vol. 3633, pp. 234–242. SPIE.

  • Christensen, M.P., Euliss, G.W., McFadden, M.J., Coyle, K.M., Milojkovic, P., Haney, M.W., Gracht, J. van der, and Athale, R. A. October 2002. Active-eyes: An adaptive pixel-by-pixel image-segmentation sensor architecture for high-dynamic-range hyperspectral imaging. Applied Optics, 41(29):6093–6103.

  • Dudley, D., Duncan, W., and Slaughter, J. February 2003. Emerging digital micromirror device (DMD) applications. White paper, Texas Intruments.

  • Ginosar, R., Hilsenrath, O., and Zeevi, Y. September 1992. Wide dynamic range camera. U.S. Patent 5, 144,442.

  • Goodman, J. W. 1968. Introduction to Fourier Optics. McGraw-Hill, New York.

  • Grossberg, M.D. and Nayar, S.K. 2005. The raxel imaging model and ray-based calibration. IJCV, 61(2):119–137.

    Article  Google Scholar 

  • Hicks, R. A. 2003. Personal Communication.

  • Hornbeck, L.J. 1988. Bistable deformable mirror device. In Spat. Light Mod. and Apps., vol. 8. OSA.

  • Hornbeck, L.J. August 1989. Deformable-mirror spatial light modulators. In Projection Displays III, vol. 1150, pp. 86–102. SPIE.

  • Hornbeck, L.J. 1995. Projection displays and MEMS: Timely convergence for a bright future. In Micromachined Devices and Components, vol. 2642. SPIE.

  • Kang, S.B., Uyttendaele, M., Winder, S., and Szeliski, R. 2003. High dynamic range video. ACM Trans. on Graph. (Proc. of SIGGRAPH 2003), 22(3):319–325.

  • Kearney, K.J. and Ninkov, Z. 1998. Characterization of digital micromirror device for use as an optical mask in imaging and spectroscopy. Spatial Light Modulators, 3292:81–92. SPIE.

    Google Scholar 

  • Malbet, F., Yu, J., and Shao, M. 1995. High dynamic range imaging using a deformable mirror for space coronography. Public. of the Astro. Soc. of the Pacific, 107:386.

    Google Scholar 

  • Murase, H. and Nayar, S.K. 1995. Visual learning and recognition of 3d objects from appearance. IJCV, 14(1):5–24.

    Article  Google Scholar 

  • Nayar, S.K. and Branzoi, V. 2003. Adaptive Dynamic Range Imaging: Optical Control of Pixel Exposures over Space and Time. Proc. of Inter. Conf. on Computer Vision (ICCV), pp. 1168–1175.

  • Nayar, S.K. and Mitsunaga, T. 2000. High dynamic range imaging: Spatially varying pixel exposures. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1:472–479.

  • Nayar, S.K. and Narasimhan, S.G. 2002. Assorted pixels: Multisampled imaging with structural models. Proc. of Euro. Conf. on Comp. Vis. (ECCV), 4:636–152.

  • Smith, W.J. 1966. Modern Optical Engineering. McGraw-Hill.

  • Turk, M. and Pentland, A.P. 1991. Face recognition using eigenfaces. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 586–591.

  • Tyson, R.K. 1998. Principles of Adaptive Optics. Academic Press.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Columbia University, 450 Mudd Hall, New York, N.Y., 10027

    Shree K. Nayar & Vlad Branzoi

  2. Department of Computer Science, University of Colorado, Colorado Springs, Colorado, 80933

    Terry E. Boult

Authors
  1. Shree K. Nayar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vlad Branzoi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Terry E. Boult
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shree K. Nayar.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nayar, S.K., Branzoi, V. & Boult, T.E. Programmable Imaging: Towards a Flexible Camera. Int J Comput Vision 70, 7–22 (2006). https://doi.org/10.1007/s11263-005-3102-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-005-3102-6

Keywords

Search

Navigation