Emerging theories and technologies on computational imaging

  • Xue-mei Hu
  • Jia-min Wu
  • Jin-li Suo
  • Qiong-hai Dai


Computational imaging describes the whole imaging process from the perspective of light transport and information transmission, features traditional optical computing capabilities, and assists in breaking through the limitations of visual information recording. Progress in computational imaging promotes the development of diverse basic and applied disciplines. In this review, we provide an overview of the fundamental principles and methods in computational imaging, the history of this field, and the important roles that it plays in the development of science. We review the most recent and promising advances in computational imaging, from the perspective of different dimensions of visual signals, including spatial dimension, temporal dimension, angular dimension, spectral dimension, and phase. We also discuss some topics worth studying for future developments in computational imaging.

Key words

Computational imaging Multi-scale and multi-dimensional Super-resolution Femto-photography 3D reconstruction Hyperspectral imaging 

CLC number



  1. Assion, A., Baumert, T., Bergt, M., et al., 1998. Control of chemical reactions by feedback-optimized phase-shaped femtosecond laser pulses. Science, 282(5390):919–922. Scholar
  2. Backman, V., Wallace, M.B., Perelman, L., et al., 2000. Detection of preinvasive cancer cells. Nature, 406(6791): 35–36. Scholar
  3. Bao, J., Bawendi, M.G., 2015. A colloidal quantum dot spectrometer. Nature, 523(7558):67–70. Scholar
  4. Bifano, T., 2011. Adaptive imaging: MEMS deformable mirrors. Nat. Photon., 5(1):21–23. Scholar
  5. Bina, M., Magatti, D., Molteni, M., et al., 2013. Backscattering differential ghost imaging in turbid media. Phys. Rev. Lett., 110(8), Article 083901. Scholar
  6. Brady, D., Gehm, M., Stack, R., et al., 2012. Multiscale gigapixel photography. Nature, 486(7403):386–389. Scholar
  7. Brenner, D.J., Hall, E.J., 2007. Computed tomography—an increasing source of radiation exposure. New Engl. J. Med., 357:2277–2284. Scholar
  8. Candès, E.J., Romberg, J., Tao, T., 2006. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory, 52(2):489–509. Scholar
  9. Chaigne, T., Katz, O., Boccara, A.C., et al., 2014. Controlling light in scattering media non-invasively using the photoacoustic transmission matrix. Nat. Photon., 8(1):58–64. Scholar
  10. Chakrabarti, A., Zickler, T., 2011. Statistics of real-world hyperspectral images. IEEE Conf. on Computer Vision and Pattern Recognition, p.193–200. Scholar
  11. Chao, T.H., Zhou, H., Xia, X., et al., 2005. Near IR electrooptic imaging Fourier transform spectrometer. Proc. Optical Pattern Recognition, p.163–172. Scholar
  12. Charles, A.S., Olshausen, B.A., Rozell, C.J., 2011. Learning sparse codes for hyperspectral imagery. IEEE J. Sel. Topics Signal Process., 5(5):963–978. Scholar
  13. Choi, W., Fang-Yen, C., Badizadegan, K.R., et al., 2007. Tomographic phase microscopy. Nat. Meth., 4(9):717–719. Scholar
  14. Cotte, Y., Toy, F., Jourdain, P., et al., 2013. Marker-free phase nanoscopy. Nat. Photon., 7(2):113–117. Scholar
  15. Cuche, E., Bevilacqua, F., Depeursinge, C., 1999. Digital holography for quantitative phase-contrast imaging. Opt. Lett., 24(5):291–293. Scholar
  16. Delalieux, S., Auwerkerken, A., Verstraeten, W.W., et al., 2009. Hyperspectral reflectance and fluorescence imaging to detect scab induced stress in apple leaves. Remote Sens., 1(4):858–874. Scholar
  17. Descour, M., Dereniak, E., 1995. Computed-tomography imaging spectrometer: experimental calibration and reconstruction results. Appl. Opt., 34(22):4817–4826. Scholar
  18. Diaspro, A., Chirico, G., Collini, M., 2005. Two-photon fluorescence excitation and related techniques in biological microscopy. Q. Rev. Biophys., 38(02):97–166. Scholar
  19. Ding, W., Wang, Y., Chen, H., et al., 2014. Plasmonic nanocavity organic light-emitting diode with significantly enhanced light extraction, contrast, viewing angle, brightness, and low-glare. Adv. Funct. Mater., 24(40):6329–6339. Scholar
  20. Ferguson, R., Phillips, W., 1967. High-resolution nuclear magnetic resonance spectroscopy. Science, 157(3786): 257–267. Scholar
  21. Fienup, J.R., 1982. Phase retrieval algorithms: a comparison. Appl. Opt., 21(15):2758–2769. Scholar
  22. Fienup, J.R., 2013. Phase retrieval algorithms: a personal tour [invited]. Appl. Opt., 52(1):45–56. Scholar
  23. Frenkel, K.A., 2010. Panning for science. Science, 330(6005):748–749. Scholar
  24. Gatti, A., Brambilla, E., Bache, M., et al., 2004. Ghost imaging with thermal light: comparing entanglement and classical correlation. Phys. Rev. Lett., 93(9), Article 093602. Scholar
  25. Gebbie, H., 1961. Molecular emission spectroscopy from 2μ to 12μ by a michelson interferometer. Nature, 191:264–265. Scholar
  26. Goda, K., Tsia, K., Jalali, B., 2009. Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena. Nature, 458(7242):1145–1149. Scholar
  27. Greenbaum, A., Luo, W., Su, T.W., et al., 2012. Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy. Nat. Meth., 9(9):889–895. Scholar
  28. Gustafsson, M.G., 2005. Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution. PNAS, 102(37):13081–13086. Scholar
  29. Heide, F., Hullin, M.B., Gregson, J., et al., 2013. Low-budget transient imaging using photonic mixer devices. ACM Trans. Graph., 32(4), Article ai]45. Scholar
  30. Hein, B., Willig, K.I., Hell, S.W., 2008. Stimulated emission depletion (STED) nanoscopy of a fluorescent protein-labeled organelle inside a living cell. PNAS, 105(38):14271–14276. Scholar
  31. Hell, S.W., Wichmann, J., 1994. Breaking the diffraction resolution limit by stimulated emission: stimulatedemission- depletion fluorescence microscopy. Opt. Lett., 19(11):780–782. Scholar
  32. Helmchen, F., Denk, W., 2005. Deep tissue two-photon microscopy. Nat. Meth., 2(12):932–940. Scholar
  33. Hess, S.T., Girirajan, T.P., Mason, M.D., 2006. Ultrahigh resolution imaging by fluorescence photoactivation localization microscopy. Biophys. J., 91(11):4258–4272. Scholar
  34. Horton, N.G., Wang, K., Kobat, D., et al., 2013. In vivo three-photon microscopy of subcortical structures within an intact mouse brain. Nat. Photon., 7(3):205–209. Scholar
  35. Howard, S.S., Straub, A., Horton, N.G., et al., 2013. Frequency-multiplexed in vivo multiphoton phosphorescence lifetime microscopy. Nat. Photon., 7(1):33–37. Scholar
  36. Jahr, W., Schmid, B., Schmied, C., et al., 2015. Hyperspectral light sheet microscopy. Nat. Commun., 6, Article 7990. Scholar
  37. Ji, N., Milkie, D.E., Betzig, E., 2010. Adaptive optics via pupil segmentation for high-resolution imaging in biological tissues. Nat. Meth., 7(2):141–147. Scholar
  38. Kester, R.T., Bedard, N., Gao, L., et al., 2011. Real-time snapshot hyperspectral imaging endoscope. J. Biomed. Opt., 16(5), Article 056005. Scholar
  39. Kim, T., Zhou, R., Mir, M., et al., 2014. White-light diffraction tomography of unlabelled live cells. Nat. Photon., 8(3):256–263. Scholar
  40. Levoy, M., Hanrahan, P., 1996. Light field rendering. Proc. 23rd Annual Conf. on Computer Graphics and Interactive Techniques, p.31–42. Scholar
  41. Levoy, M., Ng, R., Adams, A., et al., 2006. Light field microscopy. ACM Trans. Graph., 25(3):924–934. Scholar
  42. Lin, X., Liu, Y., Wu, J., et al., 2014. Spatial-spectral encoded compressive hyperspectral imaging. ACM Trans. Graph., 33(6), Article 233. Scholar
  43. Lin, X., Wu, J., Zheng, G., et al., 2015. Camera array based light field microscopy. Biomed. Opt. Expr., 6(9):3179–3189. Scholar
  44. Ma, C., Cao, X., Tong, X., et al., 2014. Acquisition of high spatial and spectral resolution video with a hybrid camera system. Int. J. Comput. Vis., 110(2):141–155. Scholar
  45. Manley, S., Gillette, J.M., Patterson, G.H., et al., 2008. High-density mapping of single-molecule trajectories with photoactivated localization microscopy. Nat. Meth., 5(2):155–157. Scholar
  46. Marks, D.L., Son, H.S., Kim, J., et al., 2012. Engineering a gigapixel monocentric multiscale camera. Opt. Eng., 51(8), Article 083202. Scholar
  47. Morris, P.A., Aspden, R.S., Bell, J.E., et al., 2015. Imaging with a small number of photons. Nat. Commun., 6, Article 5913. Scholar
  48. Nakagawa, K., Iwasaki, A., Oishi, Y., et al., 2014. Sequentially timed all-optical mapping photography (STAMP). Nat. Photon., 8(9):695–700. Scholar
  49. Neifeld, M.A., Shankar, P., 2003. Feature-specific imaging. Appl. Opt., 42(17):3379–3389. Scholar
  50. Ng, R., Levoy, M., Brédif, M., et al., 2005. Light field photography with a hand-held plenoptic camera. Comput. Sci. Techn. Rep., 2(11):1–11.Google Scholar
  51. Orth, A., Tomaszewski, M.J., Ghosh, R.N., et al., 2015. Gigapixel multispectral microscopy. Optica, 2(7):654–662. Scholar
  52. Pal, H., Neifeld, M., 2003. Multispectral principal component imaging. Opt. Expr., 11(18):2118–2125. Scholar
  53. Popescu, G., Deflores, L.P., Vaughan, J.C., et al., 2004. Fourier phase microscopy for investigation of biological structures and dynamics. Opt. Lett., 29(21):2503–2505. Scholar
  54. Prevedel, R., Yoon, Y.G., Hoffmann, M., et al., 2014. Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy. Nat. Meth., 11(7):727–730. Scholar
  55. Rust, M.J., Bates, M., Zhuang, X., 2006. Sub-diffractionlimit imaging by stochastic optical reconstruction microscopy (STORM). Nat. Meth., 3(10):793–796. Scholar
  56. Ryle, M., 1972. The 5-km radio telescope at Cambridge. Nature, 239:435–438. Scholar
  57. Schermelleh, L., Heintzmann, R., Leonhardt, H., 2010. A guide to super-resolution fluorescence microscopy. J. Cell Biol., 190(2):165–175. Scholar
  58. Stoklasa, B., Motka, L., Rehacek, J., et al., 2014. Wavefront sensing reveals optical coherence. Nat. Commun., 5, Article 3275. Scholar
  59. Strack, R., 2016. Highly multiplexed imaging. Nat. Meth., 13(1), Article 35. Scholar
  60. Suo, J., Bian, L., Chen, F., et al., 2014. Bispectral coding: compressive and high-quality acquisition of fluorescence and reflectance. Opt. Expr., 22(2):1697–1712. Scholar
  61. Teague, M.R., 1983. Deterministic phase retrieval: a green’s function solution. JOSA, 73(11):1434–1441. Scholar
  62. van Tilbeurgh, H., Egloff, M., Martinez, C., et al., 1993. Interfacial activation of the lipase-procolipase complex by mixed micelles revealed by X-ray crystallography. Nature, 362(6423):814–820. Scholar
  63. Vellekoop, I., Lagendijk, A., Mosk, A., 2010. Exploiting disorder for perfect focusing. Nat. Photon., 4(5):320–322. Scholar
  64. Velten, A., Willwacher, T., Gupta, O., et al., 2012. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun., 3, Article ai]745. Scholar
  65. Velten, A., Wu, D., Jarabo, A., et al., 2013. Femtophotography: capturing and visualizing the propagation of light. ACM Trans. Graph., 32(4), Article ai]44. Scholar
  66. Waller, L., Kou, S.S., Sheppard, C.J., et al., 2010a. Phase from chromatic aberrations. Opt. Expr., 18(22):22817–22825. Scholar
  67. Waller, L., Tian, L., Barbastathis, G., 2010b. Transport of intensity phase-amplitude imaging with higher order intensity derivatives. Opt. Expr., 18(12):12552–12561. Scholar
  68. Waller, L., Situ, G., Fleischer, J.W., 2012. Phase-space measurement and coherence synthesis of optical beams. Nat. Photon., 6(7):474–479. Scholar
  69. Wang, L.V., Hu, S., 2012. Photoacoustic tomography: in vivo imaging from organelles to organs. Science, 335(6075):1458–1462. Scholar
  70. Wilburn, B., Joshi, N., Vaish, V., et al., 2004. Highspeed videography using a dense camera array. Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, p.294–301. Scholar
  71. Willett, R., Gehm, M.E., Brady, D.J., 2007. Multiscale reconstruction for computational spectral imaging. Proc. Electronic Imaging, Article 64980L. Scholar
  72. Wong, G., 2009. Snapshot hyperspectral imaging and practical applications. J. Phys., 178(1), Article 012048. Scholar
  73. Zernike, F., 1955. How I discovered phase contrast. Science, 121(3141):345–349.CrossRefGoogle Scholar
  74. Zheng, G., Horstmeyer, R., Yang, C., 2013. Wide-field, high-resolution Fourier ptychographic microscopy. Nat. Photon., 7(9):739–745. Scholar

Copyright information

© Zhejiang University and Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Department of AutomationTsinghua UniversityBeijingChina
  2. 2.Zhejiang Future Technology InstituteJiaxingChina

Personalised recommendations