High-resolution spectral video acquisition

  • Lin-sen Chen
  • Tao Yue
  • Xun Cao
  • Zhan Ma
  • David J. Brady


Compared with conventional cameras, spectral imagers provide many more features in the spectral domain. They have been used in various fields such as material identification, remote sensing, precision agriculture, and surveillance. Traditional imaging spectrometers use generally scanning systems. They cannot meet the demands of dynamic scenarios. This limits the practical applications for spectral imaging. Recently, with the rapid development in computational photography theory and semiconductor techniques, spectral video acquisition has become feasible. This paper aims to offer a review of the state-of-the-art spectral imaging technologies, especially those capable of capturing spectral videos. Finally, we evaluate the performances of the existing spectral acquisition systems and discuss the trends for future work.

Key words

Multispectral/hyperspectral video acquisition Snapshot Under-sampling and reconstruction 

CLC number



Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abed, F.M., Amirshahi, S.H., Abed, M.R.M., 2009. Reconstruction of reflectance data using an interpolation technique. J. Opt. Soc. Am. A, 26(3):613–624. https://doi.org/10.1364/JOSAA.26.000613CrossRefGoogle Scholar
  2. Adelson, E.H., Bergen, J.R., 1991. The plenoptic function and the elements of early vision. In: Landy, M.S., Movshon, J.A. (Eds.), Computational Models of Visual Processing. MIT Press, Cambridge, p.3–20.Google Scholar
  3. Arce, G.R., Brady, D.J., Carin, L., et al., 2014. Compressive coded aperture spectral imaging: an introduction. IEEE Signal Process. Mag., 31(1):105–115. https://doi.org/10.1109/MSP.2013.2278763CrossRefGoogle Scholar
  4. Bao, J., Bawendi, M.G., 2015. A colloidal quantum dot spectrometer. Nature, 523(7558):67–70. https://doi.org/10.1038/nature14576CrossRefGoogle Scholar
  5. Bioucas-Dias, J.M., Figueiredo, M.A., 2007. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Imag. Process., 16(12):2992–3004. https://doi.org/10.1109/TIP.2007.909319MathSciNetCrossRefGoogle Scholar
  6. Bodkin, A., Sheinis, A., Norton, A., et al., 2009. Snapshot hyperspectral imaging: the hyperpixel array camera. SPIE, 7334:73340H. https://doi.org/10.1117/12.818929Google Scholar
  7. Boyd, S., Parikh, N., Chu, E., et al., 2011. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn., 3(1):1–122. https://doi.org/10.1561/2200000016CrossRefMATHGoogle Scholar
  8. Candès, E.J., Wakin, M.B., 2008. An introduction to compressive sampling. IEEE Signal Process. Mag., 25(2): 21–30. https://doi.org/10.1109/MSP.2007.914731CrossRefGoogle Scholar
  9. Candès, E.J., Romberg, J., Tao, T., 2006. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory, 52(2):489–509. https://doi.org/10.1109/TIT.2005.862083MathSciNetCrossRefMATHGoogle Scholar
  10. Cao, X., Du, H., Tong, X., et al., 2011a. A prism-mask system for multispectral video acquisition. IEEE Trans. Patt. Anal. Mach. Intell., 33(12):2423–2435. https://doi.org/10.1109/TPAMI.2011.80CrossRefGoogle Scholar
  11. Cao, X., Tong, X., Dai, Q., et al., 2011b. High resolution multispectral video capture with a hybrid camera system. IEEE Conf. on Computer Vision and Pattern Recognition, p.297–304. https://doi.org/10.1109/CVPR.2011.5995418Google Scholar
  12. Cao, X., Yue, T., Lin, X., et al., 2016. Computational snapshot multispectral cameras. IEEE Signal Process. Mag., 33(5):95–108. https://doi.org/10.1109/MSP.2016.2582378CrossRefGoogle Scholar
  13. Chakrabarti, A., Zickler, T., 2011. Statistics of real-world hyperspectral images. IEEE Conf. on Computer Vision and Pattern Recognition, p.193–200. https://doi.org/10.1109/CVPR.2011.5995660Google Scholar
  14. Descour, M., Dereniak, E., 1995. Computed-tomography imaging spectrometer: experimental calibration and reconstruction results. Appl. Opt., 34(22):4817–4826. https://doi.org/10.1364/AO.34.004817CrossRefGoogle Scholar
  15. Descour, M., Volin, C.E., Ford, B.K., et al., 2001. Snapshot hyperspectral imaging. In: Integrated Computational Imaging Systems. OSA Publishing, Washington, D.C., paper IWB4.Google Scholar
  16. Donoho, D.L., 2006. Compressed sensing. IEEE Trans. Inform. Theory, 52(4):1289–1306. https://doi.org/10.1109/TIT.2006.871582MathSciNetCrossRefMATHGoogle Scholar
  17. Du, H., Tong, X., Cao, X., et al., 2009. A prism-based system for multispectral video acquisition. IEEE 12th Int. Conf. on Computer Vision, p.175–182. https://doi.org/10.1109/ICCV.2009.5459162Google Scholar
  18. Gao, L., Kester, R.T., Hagen, N., et al., 2010. Snapshot image mapping spectrometer (IMS) with high sampling density for hyperspectral microscopy. Opt. Expr., 18(14):14330–14344. https://doi.org/10.1364OE.18.014330CrossRefGoogle Scholar
  19. Gat, N., 2000. Imaging spectroscopy using tunable filters: a review. SPIE, 4056:50–64. https://doi.org/10.1117/12.381686Google Scholar
  20. Golbabaee, M., Vandergheynst, P., 2012. Compressed sensing of simultaneous low-rank and joint-sparse matrices. arXiv:1211.5058. http://arxiv.org/abs/1211.5058Google Scholar
  21. Green, R.O., Eastwood, M.L., Sarture, C.M., et al., 1998. Imaging spectroscopy and the airborne visible/infrared imaging spectrometer (AVIRIS). Remote Sens. Environ., 65(3):227–248. https://doi.org/10.1016/S0034-4257(98)00064-9CrossRefGoogle Scholar
  22. Harvey, A.R., Beale, J.E., Greenaway, A.H., et al., 2000. Technology options for imaging spectrometry. Int. Symp. on Optical Science and Technology, p.13–24. https://doi.org/10.1117/12.406592Google Scholar
  23. Herrala, E., Okkonen, J.T., Hyvarinen, T.S., et al., 1994. Imaging spectrometer for process industry applications. SPIE, 2248:33–40. https://doi.org/10.1117/12.194344Google Scholar
  24. Hunicz, J., Piernikarski, D., 2001. Investigation of combustion in a gasoline engine using spectrophotometric methods. SPIE, 4516:307–314. https://doi.org/10.1117/12.435940Google Scholar
  25. Kindzelskii, A.L., Yang, Z.Y., Nabel, G.J., et al., 2000. Ebola virus secretory glycoprotein (sGP) diminishes FcγRIIIB-to-CR3 proximity on neutrophils. J. Immun., 164(2):953–958. https://doi.org/10.4049/jimmunol.164.2.953CrossRefGoogle Scholar
  26. Kittle, D., Choi, K., Wagadarikar, A., et al., 2010. Multiframe image estimation for coded aperture snapshot spectral imagers. Appl. Opt., 49(36):6824–6833.CrossRefGoogle Scholar
  27. Lawlor, J., Fletcher-Holmes, D., Harvey, A., et al., 2002. In vivo hyperspectral imaging of human retina and optic disc. Invest. Ophthalmol. Vis. Sci., 43(13):4350–4350. https://doi.org/10.1364/AO.49.006824Google Scholar
  28. Liao, X., Li, H., Carin, L., 2014. Generalized alternating projection for weighted-葧2,1 minimization with applications to model-based compressive sensing. SIAM J. Imag. Sci., 7(2):797–823. https://doi.org/10.1137/130936658MathSciNetCrossRefMATHGoogle Scholar
  29. Lin, X., Liu, Y., Wu, J., et al., 2014a. Spatial-spectral encoded compressive hyperspectral imaging. ACM Trans. Graph., 33(6), Article 233. https://doi.org/10.1145/2661229.2661262CrossRefMATHGoogle Scholar
  30. Lin, X., Wetzstein, G., Liu, Y., et al., 2014b. Dualcoded compressive hyperspectral imaging. Opt. Lett., 39(7):2044–2047. https://doi.org/10.1364/OL.39.002044CrossRefGoogle Scholar
  31. Ma, C., Cao, X., Wu, R., et al., 2014. Content-adaptive high-resolution hyperspectral video acquisition with a hybrid camera system. Opt. Lett., 39(4):937–940. https://doi.org/10.1364/OL.39.000937CrossRefGoogle Scholar
  32. Mansfield, C.L., 2005. Seeing into the Past. http://www. nasa.gov/vision/earth/technologies/scrolls.htmlGoogle Scholar
  33. MitchellP.A.1995. Hyperspectral digital imagery collection experiment (HYDICE). SPIE, 2587:70–95. https://doi.org/10.1117/12.22680Google Scholar
  34. Mooney, J.M., Vickers, V.E., An, M., et al., 1997. Highthroughput hyperspectral infrared camera. J. Opt. Soc. Am. A, 14(11):2951–2961. https://doi.org/10.1364/JOSAA.14.002951CrossRefGoogle Scholar
  35. Morovic, P., Finlayson, G.D., 2006. Metamer-set-based approach to estimating surface reflectance from camera RGB. J. Opt. Soc. Am. A, 23(8):1814–1822. https://doi.org/10.1364/JOSAA.23.001814CrossRefGoogle Scholar
  36. Morris, H.R., Hoyt, C.C., Treado, P.J., 1994. Imaging spectrometers for fluorescence and Raman microscopy: acousto-optic and liquid crystal tunable filters. Appl. Spectr., 48(7):857–866.CrossRefGoogle Scholar
  37. Nguyen, R.M., Prasad, D.K., Brown, M.S., 2014. Trainingbased spectral reconstruction from a single RGB image. European Conf. on Computer Vision, p.186–201. https://doi.org/10.1007/978-3-319-10584-0_13Google Scholar
  38. Oh, W.S., Brown, M.S., Pollefeys, M., et al., 2016. Do it yourself hyperspectral imaging with everyday digital cameras. IEEE Conf. on Computer Vision and Pattern Recognition, p.2461–2469. https://doi.org/10.1109/CVPR.2016.270Google Scholar
  39. Radon, J., 1917. Über die Bestimmung von Funktionen durch ihre Integralwerte längs gewisser Mannigfaltigkeiten. Akad. Wiss., 69:262–277 (in German).MATHGoogle Scholar
  40. Rørslett, B., 2004. All you ever wanted to know about digital UV and IR photography, but could not afford to ask. http://www.naturfotograf.com/UV_IR_rev00.htmlGoogle Scholar
  41. Schechner, Y.Y., Nayar, S.K., 2002. Generalized mosaicing: wide field of view multispectral imaging. IEEE Trans. Patt. Anal. Mach. Intell., 24(10):1334–1348. https://doi.org/10.1109/TPAMI.2002.1039205CrossRefGoogle Scholar
  42. Shepp, L.A., Vardi, Y., 1982. Maximum likelihood reconstruction for emission tomography. IEEE Trans. Med. Imag., 1(2):113–122. https://doi.org/10.1109/TMI.1982.4307558CrossRefGoogle Scholar
  43. Su, L., Zhou, Z., Yuan, Y., et al., 2015. A snapshot light field imaging spectrometer. Opt.-Int. J. Light Electr. Opt., 126(9):877–881. https://doi.org/10.1016/j.ijleo.2015.01.034CrossRefGoogle Scholar
  44. Wagadarikar, A.A., Pitsianis, N.P., Sun, X., et al., 2009. Video rate spectral imaging using a coded aperture snapshot spectral imager. Opt. Expr., 17(8):6368–6388. https://doi.org/10.1364/OE.17.006368CrossRefGoogle Scholar
  45. Willett, R.M., Duarte, M.F., Davenport, M.A., et al., 2014. Sparsity and structure in hyperspectral imaging: sensing, reconstruction, and target detection. IEEE Signal Process. Mag., 31(1):116–126. https://doi.org/10.1109/MSP.2013.2279507CrossRefGoogle Scholar
  46. Wu, Y., Mirza, I.O., Arce, G.R., et al., 2011. Development of a digital-micromirror-device-based multishot snapshot spectral imaging system. Opt. Lett., 36(14):2692–2694. https://doi.org/10.1364/OL.36.002692CrossRefGoogle Scholar
  47. Yamaguchi, M., Haneishi, H., Fukuda, H., et al., 2006. Highfidelity video and still-image communication based on spectral information: natural vision system and its applications. SPIE, 6062:60620G. https://doi.org/10.1117/12.649454Google Scholar
  48. Yasuma, F., Mitsunaga, T., Iso, D., et al., 2010. Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE Trans. Imag. Process., 19(9):2241–2253. https://doi.org/10.1109/TIP.2010.2046811MathSciNetCrossRefMATHGoogle Scholar
  49. Zhou, Z., Yuan, Y., Bin, X.L., 2010. Light field imaging spectrometer: conceptual design and simulated performance. Frontiers in Optics/Laser Science XXVI, paper FThM3. https://doi.org/10.1364/FIO.2010.FThM3Google Scholar

Copyright information

© Zhejiang University and Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.School of Electronic Science and EngineeringNanjing UniversityNanjingChina
  2. 2.Department of Electrical & Computer EngineeringDuke UniversityDurhamUSA

Personalised recommendations