Sparse Recovery of Hyperspectral Signal from Natural RGB Images

  • Boaz Arad
  • Ohad Ben-ShaharEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9911)


Hyperspectral imaging is an important visual modality with growing interest and range of applications. The latter, however, is hindered by the fact that existing devices are limited in either spatial, spectral, and/or temporal resolution, while yet being both complicated and expensive. We present a low cost and fast method to recover high quality hyperspectral images directly from RGB. Our approach first leverages hyperspectral prior in order to create a sparse dictionary of hyperspectral signatures and their corresponding RGB projections. Describing novel RGB images via the latter then facilitates reconstruction of the hyperspectral image via the former. A novel, larger-than-ever database of hyperspectral images serves as a hyperspectral prior. This database further allows for evaluation of our methodology at an unprecedented scale, and is provided for the benefit of the research community. Our approach is fast, accurate, and provides high resolution hyperspectral cubes despite using RGB-only input.


Hyperspectral Image Natural Image Hyperspectral Data Overcomplete Dictionary Sparse Dictionary 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This research was supported in part by the by the Israel Science Foundation (ISF FIRST/BIKURA Grant 281/15) and the European Commission (Horizon 2020 grant SWEEPER GA no. 644313). We also thank the Frankel Fund and the Helmsley Charitable Trust through the ABC Robotics Initiative, both at Ben-Gurion University of the Negev.


  1. 1.
    Kerekes, J., Schott, J.: Hyperspectral imaging systems. Hyperspectral data exploitation: theory and applications (2007)Google Scholar
  2. 2.
    Lillesand, T., Kiefer, R., Chipman, J., et al.: Remote Sensing and Image Interpretation. Wiley, New York (2004)Google Scholar
  3. 3.
    Haboudane, D., Miller, J., Pattey, E., Zarco-Tejada, P., Strachan, I.: Hyperspectral vegetation indices and novel algorithms for predicting green lai of crop canopies: modeling and validation in the context of precision agriculture. In: Remote Sensing of Environment (2004)Google Scholar
  4. 4.
    Cloutis, E.: Review article hyperspectral geological remote sensing: evaluation of analytical techniques. Int. J. Remote Sens. 17, 2215–2242 (1996)CrossRefGoogle Scholar
  5. 5.
    Hege, E., O’Connell, D., Johnson, W., Basty, S., Dereniak, E.: Hyperspectral imaging for astronomy and space surviellance. In: SPIE (2004)Google Scholar
  6. 6.
    Mustard, J., Sunshine, J.: Spectral analysis for earth science: investigations using remote sensing data. In: Manual of Remote Sensing, Remote Sensing for the Earth Sciences (1999)Google Scholar
  7. 7.
    Green, R., Eastwood, M., Sarture, C., Chrien, T., Aronsson, M., Chippendale, B., Faust, J., Pavri, B., Chovit, C., Solis, M., Olah, M., Williams, O.: Imaging spectroscopy and the airborne visible/infrared imaging spectrometer (AVIRIS). In: Remote Sensing of Environment (1998)Google Scholar
  8. 8.
    James, J.: Spectrograph Design Fundamentals. Cambridge University Press, New York (2007)CrossRefGoogle Scholar
  9. 9.
    Descour, M., Dereniak, E.: Computed-tomography imaging spectrometer: experimental calibration and reconstruction results. Appl. Opt. 34, 4817–4826 (1995)CrossRefGoogle Scholar
  10. 10.
    Okamoto, T., Yamaguchi, I.: Simultaneous acquisition of spectral image information. Opt. Lett. 16, 1277–1279 (1991)CrossRefGoogle Scholar
  11. 11.
    Johnson, W., Wilson, D., Bearman, G.: Spatial-spectral modulating snapshot hyperspectral imager. Appl. Opt. 45, 1898–1908 (2006)CrossRefGoogle Scholar
  12. 12.
    Brady, D., Gehm, M.: Compressive imaging spectrometers using coded apertures. In: Defense and Security Symposium (2006)Google Scholar
  13. 13.
    Gehm, M., John, R., Brady, D., Willett, R., Schulz, T.: Single-shot compressive spectral imaging with a dual-disperser architecture. Opt. Express 15, 14013–14027 (2007)CrossRefGoogle Scholar
  14. 14.
    Lin, X., Wetzstein, G., Liu, Y., Dai, Q.: Dual-coded compressive hyperspectral imaging. Opt. Lett. 39, 2044–2047 (2014)CrossRefGoogle Scholar
  15. 15.
    Fletcher-Holmes, D., Harvey, A.: Real-time imaging with a hyperspectral fovea. J. Opt. A Pure Appl. Opt. 7, S298–S302 (2005)CrossRefGoogle Scholar
  16. 16.
    Wang, T., Zhu, Z., Rhody, H.: A smart sensor with hyperspectral/range fovea and panoramic peripheral view. In: CVPR (2009)Google Scholar
  17. 17.
    Du, H., Tong, X., Cao, X., Lin, S.: A prism-based system for multispectral video acquisition. In: ICCV (2009)Google Scholar
  18. 18.
    Kawakami, R., Wright, J., Yu-Wing, T., Matsushita, Y., Ben-Ezra, M., Ikeuchi, K.: High-resolution hyperspectral imaging via matrix factorization. In: CVPR (2011)Google Scholar
  19. 19.
    Cao, X., Tong, X., Dai, Q., Lin, S.: High resolution multispectral video capture with a hybrid camera system. In: CVPR (2011)Google Scholar
  20. 20.
    Goel, M., Whitmire, E., Mariakakis, A., Saponas, T.S., Joshi, N., Morris, D., Guenter, B., Gavriliu, M., Borriello, G., Patel, S.N.: Hypercam: hyperspectral imaging for ubiquitous computing applications (2015)Google Scholar
  21. 21.
    Parmar, M., Lansel, S., Wandell, B.A.: Spatio-spectral reconstruction of the multispectral datacube using sparse recovery. In: ICIP (2008)Google Scholar
  22. 22.
    Kohonen, O., Parkkinen, J., Jääskeläinen, T.: Databases for spectral color science. Color Res. Appl. 31, 381–390 (2006)CrossRefGoogle Scholar
  23. 23.
    NASA: Airborne Visual Infrared Imaging Spectrometer website.
  24. 24.
    Brelstaff, G., Párraga, A., Troscianko, T., Carr, D.: Hyperspectral camera system: acquisition and analysis. In: SPIE (1995)Google Scholar
  25. 25.
    Foster, D., Amano, K., Nascimento, S., Foster, M.: Frequency of metamerism in natural scenes. JOSA A 23, 2359–2372 (2006)CrossRefGoogle Scholar
  26. 26.
    Yasuma, F., Mitsunaga, T., Iso, D., Nayar, S.: Generalized assorted pixel camera: post-capture control of resolution, dynamic range and spectrum. Technical report (2008)Google Scholar
  27. 27.
    Chakrabarti, A., Zickler, T.: Statistics of real-world hyperspectral images. In: CVPR (2011)Google Scholar
  28. 28.
    BGU Interdisciplinary Computational Vision Laboratory (iCVL): Hyperspectral Image Database website.
  29. 29.
    Palmer, S.: Vision Science: Photons to Phenomenology. The MIT Press, Cambrdige (1999)Google Scholar
  30. 30.
    Cohen, J.: Dependency of the spectral reflectance curves of the Munsell color chips. Psychonomic Sci. 1, 369–370 (1964)CrossRefGoogle Scholar
  31. 31.
    Maloney, L.: Evaluation of linear models of surface spectral reflectance with small numbers of parameters. JOSA A 3, 1673–1683 (1986)CrossRefGoogle Scholar
  32. 32.
    Parkkinen, J.P., Hallikainen, J., Jaaskelainen, T.: Characteristic spectra of Munsell colors. JOSA A 6, 318–322 (1989)CrossRefGoogle Scholar
  33. 33.
    Hardeberg, J.Y.: On the spectral dimensionality of object colors. In: Proceedings of CGIV 2002, First European Conference on Colour in Graphics (2002)Google Scholar
  34. 34.
    Adams, J., Smith, M., Gillespie, A.: Simple models for complex natural surfaces: a strategy for the hyperspectral era of remote sensing. In: IGARSS (1989)Google Scholar
  35. 35.
    Heikkinen, V., Lenz, R., Jetsu, T., Parkkinen, J., Hauta-Kasari, M., Jääskeläinen, T.: Evaluation and unification of some methods for estimating reflectance spectra from RGB images. JOSA A 25, 2444–2458 (2008)CrossRefGoogle Scholar
  36. 36.
    López-Álvarez, M.A., Hernández-Andrés, J., Romero, J., Olmo, F., Cazorla, A., Alados-Arboledas, L.: Using a trichromatic CCD camera for spectral skylight estimation. Appl. Opt. 47, 31–38 (2008)CrossRefGoogle Scholar
  37. 37.
    Ayala, F., Echávarri, J.F., Renet, P., Negueruela, A.I.: Use of three tristimulus values from surface reflectance spectra to calculate the principal components for reconstructing these spectra by using only three eigenvectors. JOSA A (2006)Google Scholar
  38. 38.
    Xing, Z., Zhou, M., Castrodad, A., Sapiro, G., Carin, L.: Dictionary learning for noisy and incomplete hyperspectral images. SIAM J. Imaging Sci. 5, 33–56 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  39. 39.
    Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54, 4311–4322 (2006)CrossRefGoogle Scholar
  40. 40.
    Pati, Y., Rezaiifar, R., Krishnaprasad, P.: Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: The Twenty-Seventh Asilomar Conference on Signals, Systems and Computers (1993)Google Scholar
  41. 41.
    Jiang, J., Liu, D., Gu, J., Susstrunk, S.: What is the space of spectral sensitivity functions for digital color cameras? In: WACV, IEEE (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceBen-Gurion University of the NegevBeershebaIsrael

Personalised recommendations