The Visual Computer

, Volume 34, Issue 12, pp 1773–1783 | Cite as

Efficient spectral reconstruction using a trichromatic camera via sample optimization

  • Yuqi LiEmail author
  • Chong Wang
  • Jieyu Zhao
  • Qingshu Yuan
Original Article


Training-based multispectral reconstruction can effectively recover spectral reflectance of captured objects using a trichromatic camera. However, existing methods are based on synthesized data, and the sizes of training sample set (e.g., multispectral images, reflectance targets) are usually large. In this paper, we present a spectral reconstruction approach using real measured data. To improve the efficiency and accuracy of spectral reconstruction, we propose a volume maximization method for sample optimization without any prior knowledge of light and cameras. We use heuristic global search algorithms to optimize samples and give an efficient spectral reconstruction method which is suitable for sparse sampling. Experimental results show that the proposed sample selection method outperforms other existing methods in terms of both spectral and colorimetric reconstruction errors. Moreover, the proposed reconstruction method achieves higher efficiency and accuracy due to lower sample redundancy.


Spectral reconstruction Sample optimization Multispectral images 



The work in this paper has been supported by the National Natural Science Foundation of China (Grant Nos. 61602268, 61603202, 61571247), the National Natural Science Foundation of Zhejiang Province (Nos. LY13F020050, LZ16F030001), and the K.C.Wong Magna Fund in Ningbo University.


  1. 1.
    Agahian, F., Amirshahi, S.A., Amirshahi, S.H.: Reconstruction of reflectance spectra using weighted principal component analysis. Color Res. Appl. 33(5), 360–371 (2008)CrossRefGoogle Scholar
  2. 2.
    Aharon, M., Elad, M., Bruckstein, A.: K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)CrossRefGoogle Scholar
  3. 3.
    Alvarez-Cortes, S., Kunkel, T., Masia, B.: Practical low-cost recovery of spectral power distributions. In: Computer Graphics Forum, vol. 35, pp. 166–178. Wiley (2016)Google Scholar
  4. 4.
    Arad, B., Ben-Shahar, O.: Sparse recovery of hyperspectral signal from natural rgb images. In: European Conference on Computer Vision, pp. 19–34. Springer (2016)Google Scholar
  5. 5.
    Brill, M.H.: Acquisition and reproduction of color images: colorimetric and multispectral approaches. Color Res. Appl. 27(4), 304–305 (2002)CrossRefGoogle Scholar
  6. 6.
    Brown, M., Süsstrunk, S.: Multi-spectral sift for scene category recognition. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 177–184. IEEE (2011)Google Scholar
  7. 7.
    Carroll, R., Ramamoorthi, R., Agrawala, M.: Illumination decomposition for material recoloring with consistent interreflections. In: ACM Transactions on Graphics (TOG), vol. 30, p. 43. ACM (2011)CrossRefGoogle Scholar
  8. 8.
    Cheung, V., Westland, S.: Methods for optimal color selection. J. Imaging Sci. Technol. 50(5), 481–488 (2006)CrossRefGoogle Scholar
  9. 9.
    Çivril, A., Magdon-Ismail, M.: On selecting a maximum volume sub-matrix of a matrix and related problems. Theor. Comput. Sci. 410(47), 4801–4811 (2009)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Fu, Y., Zheng, Y., Sato, I., Sato, Y.: Exploiting spectral-spatial correlation for coded hyperspectral image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3727–3736 (2016)Google Scholar
  11. 11.
    Han, S., Sato, I., Okabe, T., Sato, Y.: Fast spectral reflectance recovery using dlp projector. Int. J. Comput. Vis. 110(2), 172–184 (2014)CrossRefGoogle Scholar
  12. 12.
    Heikkinen, V., Cámara, C., Hirvonen, T., Penttinen, N.: Spectral imaging using consumer-level devices and kernel-based regression. JOSA A 33(6), 1095–1110 (2016)CrossRefGoogle Scholar
  13. 13.
    Heikkinen, V., Jetsu, T., Parkkinen, J., Hauta-Kasari, M., Jaaskelainen, T., Lee, S.D.: Regularized learning framework in the estimation of reflectance spectra from camera responses. JOSA A 24(9), 2673–2683 (2007)CrossRefGoogle Scholar
  14. 14.
    Iizuka, S., Simo-Serra, E., Ishikawa, H.: Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH 2016), vol. 35, No. 4, pp. 110:1–110:11 (2016)CrossRefGoogle Scholar
  15. 15.
    Jiang, J., Liu, D., Gu, J., Süsstrunk, S.: Camera spectral sensitivity.
  16. 16.
    Kalantari, N.K., Ramamoorthi, R.: Deep high dynamic range imaging of dynamic scenes. In: ACM Transactions on Graphics (Proceedings of SIGGRAPH 2017), vol. 36, No. 4 (2017)CrossRefGoogle Scholar
  17. 17.
    Kim, S., Min, D., Ham, B., Do, M., Sohn, K.: Dasc: Robust dense descriptor for multi-modal and multi-spectral correspondence estimation. IEEE Trans. Pattern Anal. Mach. Intell. 39(9), 1712–1729 (2016)CrossRefGoogle Scholar
  18. 18.
    Lan, Y., Wang, J., Lin, S., Gong, M., Tong, X., Guo, B.: Interactive chromaticity mapping for multispectral images. Visual Comput. 29(6–8), 773–783 (2013)CrossRefGoogle Scholar
  19. 19.
    Lee, M.H., Park, H., Ryu, I., Park, J.I.: Fast model-based multispectral imaging using nonnegative principal component analysis. Opt. Lett. 37(11), 1937–1939 (2012)CrossRefGoogle Scholar
  20. 20.
    Li, Y., Majumder, A., Lu, D., Gopi, M.: Content-independent multi-spectral display using superimposed projections. Comput. Graph. Forum 34(2), 337–348 (2015)CrossRefGoogle Scholar
  21. 21.
    Liu, C., Yuen, J., Torralba, A.: Sift flow: dense correspondence across scenes and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 978–994 (2011)CrossRefGoogle Scholar
  22. 22.
    Mansouri, A., Sliwa, T., Hardeberg, J.Y., Voisin, Y.: Representation and estimation of spectral reflectances using projection on pca and wavelet bases. Color Res. Appl. 33(6), 485–493 (2008)CrossRefGoogle Scholar
  23. 23.
    Melanie, M.: An introduction to genetic algorithms. Cambridge, Massachusetts London, England, Fifth printing vol. 3, pp. 62–75 (1999)Google Scholar
  24. 24.
    Mohammadi, M., Nezamabadi, M., Berns, R.S., Taplin, L.A.: Spectral imaging target development based on hierarchical cluster analysis. In: Color and Imaging Conference, vol. 2004, pp. 59–64. Society for Imaging Science and Technology (2004)Google Scholar
  25. 25.
    Nalbach, O., Seidel, H.P., Ritschel, T.: Practical capture and reproduction of phosphorescent appearance. In: Computer Graphics Forum, vol. 36, pp. 409–420. Wiley (2017)Google Scholar
  26. 26.
    Nguyen, R.M., Prasad, D.K., Brown, M.S.: Training-based spectral reconstruction from a single rgb image. In: European Conference on Computer Vision, pp. 186–201. Springer (2014)Google Scholar
  27. 27.
    Park, J.I., Lee, M.H., Grossberg, M.D., Nayar, S.K.: Multispectral imaging using multiplexed illumination. In: IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, pp. 1–8. IEEE (2007)Google Scholar
  28. 28.
    Parkkinen, J.P., Hallikainen, J., Jaaskelainen, T.: Characteristic spectra of munsell colors. JOSA A 6(2), 318–322 (1989)CrossRefGoogle Scholar
  29. 29.
    Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In: 1993 Conference Record of The Twenty-Seventh Asilomar Conference on Signals, Systems and Computers, 1993, pp. 40–44. IEEE (1993)Google Scholar
  30. 30.
    Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  31. 31.
    Shen, H.L., Yao, J.F., Li, C., Du, X., Shao, S.J., Xin, J.H.: Channel selection for multispectral color imaging using binary differential evolution. Appl. Opt. 53(4), 634–642 (2014)CrossRefGoogle Scholar
  32. 32.
    Shen, H.L., Zhang, H.G., Xin, J.H., Shao, S.J.: Optimal selection of representative colors for spectral reflectance reconstruction in a multispectral imaging system. Appl. Opt. 47(13), 2494–2502 (2008)CrossRefGoogle Scholar
  33. 33.
    University of Joensuu Color Group: Spectral Database.
  34. 34.
    Wagadarikar, A., John, R., Willett, R., Brady, D.: Single disperser design for coded aperture snapshot spectral imaging. Appl. Opt. 47(10), B44–B51 (2008)CrossRefGoogle Scholar
  35. 35.
    Wug O.S., Brown, M.S., Pollefeys, M., Joo K.S.: Do it yourself hyperspectral imaging with everyday digital cameras. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2461–2469 (2016)Google Scholar
  36. 36.
    Yasuma, F., Mitsunaga, T., Iso, D., Nayar, S.K.: Generalized assorted pixel camera: postcapture control of resolution, dynamic range, and spectrum. IEEE Trans. Image Process. 19(9), 2241–2253 (2010)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Zhang, L., Li, B., Pan, Z., Liang, D., Kang, Y., Zhang, D., Ma, X.: A method for selecting training samples based on camera response. Laser Phys. Lett. 13(9), 095201 (2016)CrossRefGoogle Scholar
  38. 38.
    Zhang, Q., Zheng, G., Zhou, D.: Comparison study of gauss, mq and tps for interpolation application. Int. J. Ind. Syst. Eng. 18(2), 185–198 (2014)Google Scholar
  39. 39.
    Zhang, W.F., Tang, G., Dai, D.Q., Nehorai, A.: Estimation of reflectance from camera responses by the regularized local linear model. Opt. Lett. 36(19), 3933–3935 (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Yuqi Li
    • 1
    • 2
    Email author
  • Chong Wang
    • 1
  • Jieyu Zhao
    • 1
  • Qingshu Yuan
    • 3
  1. 1.College of Information Science and EngineeringNingbo UniversityNingboChina
  2. 2.College of Computer Science and TechnologyZhejiang UniversityNingboChina
  3. 3.Hangzhou Institute of Service EngineeringHangzhou Normal UniversityHangzhouChina

Personalised recommendations