High-resolution spectral video acquisition

  • Lin-sen Chen
  • Tao Yue
  • Xun Cao
  • Zhan Ma
  • David J. Brady


Compared with conventional cameras, spectral imagers provide many more features in the spectral domain. They have been used in various fields such as material identification, remote sensing, precision agriculture, and surveillance. Traditional imaging spectrometers use generally scanning systems. They cannot meet the demands of dynamic scenarios. This limits the practical applications for spectral imaging. Recently, with the rapid development in computational photography theory and semiconductor techniques, spectral video acquisition has become feasible. This paper aims to offer a review of the state-of-the-art spectral imaging technologies, especially those capable of capturing spectral videos. Finally, we evaluate the performances of the existing spectral acquisition systems and discuss the trends for future work.

Key words

Multispectral/hyperspectral video acquisition Snapshot Under-sampling and reconstruction 

CLC number



Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abed, F.M., Amirshahi, S.H., Abed, M.R.M., 2009. Reconstruction of reflectance data using an interpolation technique. J. Opt. Soc. Am. A, 26(3):613–624. Scholar
  2. Adelson, E.H., Bergen, J.R., 1991. The plenoptic function and the elements of early vision. In: Landy, M.S., Movshon, J.A. (Eds.), Computational Models of Visual Processing. MIT Press, Cambridge, p.3–20.Google Scholar
  3. Arce, G.R., Brady, D.J., Carin, L., et al., 2014. Compressive coded aperture spectral imaging: an introduction. IEEE Signal Process. Mag., 31(1):105–115. Scholar
  4. Bao, J., Bawendi, M.G., 2015. A colloidal quantum dot spectrometer. Nature, 523(7558):67–70. Scholar
  5. Bioucas-Dias, J.M., Figueiredo, M.A., 2007. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Imag. Process., 16(12):2992–3004. Scholar
  6. Bodkin, A., Sheinis, A., Norton, A., et al., 2009. Snapshot hyperspectral imaging: the hyperpixel array camera. SPIE, 7334:73340H. Scholar
  7. Boyd, S., Parikh, N., Chu, E., et al., 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn., 3(1):1–122. Scholar
  8. Candès, E.J., Wakin, M.B., 2008. An introduction to compressive sampling. IEEE Signal Process. Mag., 25(2): 21–30. Scholar
  9. Candès, E.J., Romberg, J., Tao, T., 2006. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory, 52(2):489–509. Scholar
  10. Cao, X., Du, H., Tong, X., et al., 2011a. A prism-mask system for multispectral video acquisition. IEEE Trans. Patt. Anal. Mach. Intell., 33(12):2423–2435. Scholar
  11. Cao, X., Tong, X., Dai, Q., et al., 2011b. High resolution multispectral video capture with a hybrid camera system. IEEE Conf. on Computer Vision and Pattern Recognition, p.297–304. Scholar
  12. Cao, X., Yue, T., Lin, X., et al., 2016. Computational snapshot multispectral cameras. IEEE Signal Process. Mag., 33(5):95–108. Scholar
  13. Chakrabarti, A., Zickler, T., 2011. Statistics of real-world hyperspectral images. IEEE Conf. on Computer Vision and Pattern Recognition, p.193–200. Scholar
  14. Descour, M., Dereniak, E., 1995. Computed-tomography imaging spectrometer: experimental calibration and reconstruction results. Appl. Opt., 34(22):4817–4826. Scholar
  15. Descour, M., Volin, C.E., Ford, B.K., et al., 2001. Snapshot hyperspectral imaging. In: Integrated Computational Imaging Systems. OSA Publishing, Washington, D.C., paper IWB4.Google Scholar
  16. Donoho, D.L., 2006. Compressed sensing. IEEE Trans. Inform. Theory, 52(4):1289–1306. Scholar
  17. Du, H., Tong, X., Cao, X., et al., 2009. A prism-based system for multispectral video acquisition. IEEE 12th Int. Conf. on Computer Vision, p.175–182. Scholar
  18. Gao, L., Kester, R.T., Hagen, N., et al., 2010. Snapshot image mapping spectrometer (IMS) with high sampling density for hyperspectral microscopy. Opt. Expr., 18(14):14330–14344. Scholar
  19. Gat, N., 2000. Imaging spectroscopy using tunable filters: a review. SPIE, 4056:50–64. Scholar
  20. Golbabaee, M., Vandergheynst, P., 2012. Compressed sensing of simultaneous low-rank and joint-sparse matrices. arXiv:1211.5058. Scholar
  21. Green, R.O., Eastwood, M.L., Sarture, C.M., et al., 1998. Imaging spectroscopy and the airborne visible/infrared imaging spectrometer (AVIRIS). Remote Sens. Environ., 65(3):227–248. Scholar
  22. Harvey, A.R., Beale, J.E., Greenaway, A.H., et al., 2000. Technology options for imaging spectrometry. Int. Symp. on Optical Science and Technology, p.13–24. Scholar
  23. Herrala, E., Okkonen, J.T., Hyvarinen, T.S., et al., 1994. Imaging spectrometer for process industry applications. SPIE, 2248:33–40. Scholar
  24. Hunicz, J., Piernikarski, D., 2001. Investigation of combustion in a gasoline engine using spectrophotometric methods. SPIE, 4516:307–314. Scholar
  25. Kindzelskii, A.L., Yang, Z.Y., Nabel, G.J., et al., 2000. Ebola virus secretory glycoprotein (sGP) diminishes FcγRIIIB-to-CR3 proximity on neutrophils. J. Immun., 164(2):953–958. Scholar
  26. Kittle, D., Choi, K., Wagadarikar, A., et al., 2010. Multiframe image estimation for coded aperture snapshot spectral imagers. Appl. Opt., 49(36):6824–6833.CrossRefGoogle Scholar
  27. Lawlor, J., Fletcher-Holmes, D., Harvey, A., et al., 2002. In vivo hyperspectral imaging of human retina and optic disc. Invest. Ophthalmol. Vis. Sci., 43(13):4350–4350. Scholar
  28. Liao, X., Li, H., Carin, L., 2014. Generalized alternating projection for weighted-葧2,1 minimization with applications to model-based compressive sensing. SIAM J. Imag. Sci., 7(2):797–823. Scholar
  29. Lin, X., Liu, Y., Wu, J., et al., 2014a. Spatial-spectral encoded compressive hyperspectral imaging. ACM Trans. Graph., 33(6), Article 233. Scholar
  30. Lin, X., Wetzstein, G., Liu, Y., et al., 2014b. Dualcoded compressive hyperspectral imaging. Opt. Lett., 39(7):2044–2047. Scholar
  31. Ma, C., Cao, X., Wu, R., et al., 2014. Content-adaptive high-resolution hyperspectral video acquisition with a hybrid camera system. Opt. Lett., 39(4):937–940. Scholar
  32. Mansfield, C.L., 2005. Seeing into the Past. http://www. Scholar
  33. MitchellP.A.1995. Hyperspectral digital imagery collection experiment (HYDICE). SPIE, 2587:70–95. Scholar
  34. Mooney, J.M., Vickers, V.E., An, M., et al., 1997. Highthroughput hyperspectral infrared camera. J. Opt. Soc. Am. A, 14(11):2951–2961. Scholar
  35. Morovic, P., Finlayson, G.D., 2006. Metamer-set-based approach to estimating surface reflectance from camera RGB. J. Opt. Soc. Am. A, 23(8):1814–1822. Scholar
  36. Morris, H.R., Hoyt, C.C., Treado, P.J., 1994. Imaging spectrometers for fluorescence and Raman microscopy: acousto-optic and liquid crystal tunable filters. Appl. Spectr., 48(7):857–866.CrossRefGoogle Scholar
  37. Nguyen, R.M., Prasad, D.K., Brown, M.S., 2014. Trainingbased spectral reconstruction from a single RGB image. European Conf. on Computer Vision, p.186–201. Scholar
  38. Oh, W.S., Brown, M.S., Pollefeys, M., et al., 2016. Do it yourself hyperspectral imaging with everyday digital cameras. IEEE Conf. on Computer Vision and Pattern Recognition, p.2461–2469. Scholar
  39. Radon, J., 1917. Über die Bestimmung von Funktionen durch ihre Integralwerte längs gewisser Mannigfaltigkeiten. Akad. Wiss., 69:262–277 (in German).zbMATHGoogle Scholar
  40. Rørslett, B., 2004. All you ever wanted to know about digital UV and IR photography, but could not afford to ask. Scholar
  41. Schechner, Y.Y., Nayar, S.K., 2002. Generalized mosaicing: wide field of view multispectral imaging. IEEE Trans. Patt. Anal. Mach. Intell., 24(10):1334–1348. Scholar
  42. Shepp, L.A., Vardi, Y., 1982. Maximum likelihood reconstruction for emission tomography. IEEE Trans. Med. Imag., 1(2):113–122. Scholar
  43. Su, L., Zhou, Z., Yuan, Y., et al., 2015. A snapshot light field imaging spectrometer. Opt.-Int. J. Light Electr. Opt., 126(9):877–881. Scholar
  44. Wagadarikar, A.A., Pitsianis, N.P., Sun, X., et al., 2009. Video rate spectral imaging using a coded aperture snapshot spectral imager. Opt. Expr., 17(8):6368–6388. Scholar
  45. Willett, R.M., Duarte, M.F., Davenport, M.A., et al., 2014. Sparsity and structure in hyperspectral imaging: sensing, reconstruction, and target detection. IEEE Signal Process. Mag., 31(1):116–126. Scholar
  46. Wu, Y., Mirza, I.O., Arce, G.R., et al., 2011. Development of a digital-micromirror-device-based multishot snapshot spectral imaging system. Opt. Lett., 36(14):2692–2694. Scholar
  47. Yamaguchi, M., Haneishi, H., Fukuda, H., et al., 2006. Highfidelity video and still-image communication based on spectral information: natural vision system and its applications. SPIE, 6062:60620G. Scholar
  48. Yasuma, F., Mitsunaga, T., Iso, D., et al., 2010. Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE Trans. Imag. Process., 19(9):2241–2253. Scholar
  49. Zhou, Z., Yuan, Y., Bin, X.L., 2010. Light field imaging spectrometer: conceptual design and simulated performance. Frontiers in Optics/Laser Science XXVI, paper FThM3. Scholar

Copyright information

© Zhejiang University and Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.School of Electronic Science and EngineeringNanjing UniversityNanjingChina
  2. 2.Department of Electrical & Computer EngineeringDuke UniversityDurhamUSA

Personalised recommendations