Abstract
The capture of scene spectral reflectance (SR) provides a wealth of information about the material properties of objects, and has proven useful for applications including classification, synthetic relighting, medical imaging, and more. Thus many methods for SR capture have been proposed. While effective, past methods do not consider the effects of indirectly bounced light from within the scene, and the estimated SR from traditional techniques is largely affected by interreflection. For example, different lighting directions can cause different SR estimates. On the other hand, past work has shown that accurate interreflection separation in hyperspectral images is possible but the SR of all surface points needs to be known a priori. Thus we see that the estimation of SR and interreflection in its current form constitutes a chicken and egg dilemma. In this work, we propose the challenging and novel problem of simultaneously performing SR recovery and interreflection removal from a single hyperspectral image, and develop the first strategy to address it. Specifically, we model this problem using a compact sparsity regularized nonnegative matrix factorization (NMF) formulation, and introduce a scalable optimization algorithm on the basis of the alternating direction method of multipliers (ADMM). Our experiments have demonstrated its effectiveness on scenes with a single or two reflectance colors, containing possibly concave surfaces that lead to interreflection.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Balas, C., Papadakis, V., Papadakis, N., Papadakis, A., Vazgiouraki, E., Themelis, G.: A novel hyper-spectral imaging apparatus for the non-destructive analysis of objects of artistic and historic value. J. Cult. Herit. 4, 330–337 (2003)
Maloney, L.T., Wandell, B.A.: Color constancy: a method for recovering surface spectral reflectance. JOSA A 3, 29–33 (1986)
Gao, L., Kester, R.T., Tkaczyk, T.S.: Compact image slicing spectrometer (ISS) for hyperspectral fluorescence microscopy. OpEx 17, 12293–12308 (2009)
Gao, L., Kester, R.T., Hagen, N., Tkaczyk, T.S.: Snapshot image mapping spectrometer (IMS) with high sampling density for hyperspectral microscopy. OpEx 18, 14330–14344 (2010)
Gorman, A., Fletcher-Holmes, D.W., Harvey, A.R.: Generalization of the lyot filter and its application to snapshot spectral imaging. OpEx 18, 5602–5608 (2010)
Gorman, A., Muyo, G., Harvey, A.R.: Snapshot spectral imaging using birefringent interferometry and image replication. The Optical Society (2010)
Tominaga, S.: Multichannel vision system for estimating surface and illumination functions. JOSA A 13, 2163–2173 (1996)
Chi, C., Yoo, H., Ben-Ezra, M.: Multi-spectral imaging by optimized wide band illumination. IJCV 86, 140–151 (2010)
DiCarlo, J.M., Xiao, F., Wandell, B.A.: Illuminating illumination. In: CIC, IS&T/SID, pp. 27–34 (2001)
Han, S., Sato, I., Okabe, T., Sato, Y.: Fast spectral reflectance recovery using DLP projector. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010. LNCS, vol. 6492, pp. 323–335. Springer, Heidelberg (2011). doi:10.1007/978-3-642-19315-6_25
Lam, A., Subpa-Asa, A., Sato, I., Okabe, T., Sato, Y.: Spectral imaging using basis lights. In: BMVC. BMVA Press (2013)
Park, J.I., Lee, M.H., Grossberg, M.D., Nayar, S.K.: Multispectral imaging using multiplexed illumination. In: ICCV, pp. 1–8 (2007)
Forsyth, D., Zisserman, A.: Mutual illumination. In: CVPR, pp. 466–473 (1989)
Forsyth, D., Zisserman, A.: Shape from shading in the light of mutual illumination. Image Vis. Comput. 8, 42–49 (1990)
Nayar, S.K., Krishnan, G., Grossberg, M.D., Raskar, R.: Fast separation of direct and global components of a scene using high frequency illumination. In: ACM SIGGRAPH, pp. 935–944 (2006)
Liao, M., Huang, X., Yang, R.: Interreflection removal for photometric stereo by using spectrum-dependent albedo. In: CVPR, pp. 689–696 (2011)
Fu, Y., Lam, A., Matsushita, Y., Sato, I., Sato, Y.: Interreflection removal using fluorescence. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 203–217. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10602-1_14
Nam, G., Kim, M.: Multispectral photometric stereo for acquiring high-fidelity surface normals. Comput. Graph. Appl. 34, 57–68 (2014)
Imai, F.H., Rosen, M.R., Berns, R.S.: Comparison of spectrally narrow-band capture versus wide-band with a priori sample analysis for spectral reflectance estimation. In: Proceedings of Eighth Color Imaging Conference: Color Science and Engineering, Systems, Technologies and Applications, IS&T, pp. 234–241 (2000)
Jiang, J., Gu, J.: Recovering spectral reflectance under commonly available lighting conditions. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1–8 (2012)
Koenderink, J.J., van Doorn, A.J.: Geometrical modes as a general method to treat diffuse interreflections in radiometry. JOSA 73, 843–850 (1983)
Nayar, S.K., Ikeuchi, K., Kanade, T.: Shape from interreflections. IJCV 6, 173–195 (1991)
Nayar, S.K., Gao, Y.: Colored interreflections and shape recovery. In: Proceedings of the Image Understanding Workshop (1992)
Funt, B.V., Drew, M.S.: Color space analysis of mutual illumination. PAMI 15, 1319–1326 (1993)
Funt, B.V., Drew, M.S., Ho, J.: Color constancy from mutual reflection. IJCV 6, 5–24 (1991)
Seitz, S.M., Matsushita, Y., Kutulakos, K.N.: A theory of inverse light transport. In: ICCV, pp. 1440–1447 (2005)
Lee, D., Seung, H.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)
Eggert, J., Korner, E.: Sparse coding and NMF. In: IJCNN, pp. 2529–2533 (2004)
Lin, Z., Chen, M., Wu, L., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. ArXiv e-prints (2010)
Zheng, Y., Liu, G., Sugimoto, S., Yan, S., Okutomi, M.: Practical low-rank matrix approximation under robust l1-norm. In: CVPR, pp. 1410–1417 (2012)
Xu, Y., Yin, W., Wen, Z., Zhang, Y.: An alternating direction algorithm for matrix completion with nonnegative factors. Front. Math. China 7, 365–384 (2012)
Acknowledgements
This work was supported in part by Grant-in-Aid for Scientific Research on Innovative Areas (No.15H05918) from MEXT, Japan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Okawa, H., Zheng, Y., Lam, A., Sato, I. (2017). Spectral Reflectance Recovery with Interreflection Using a Hyperspectral Image. In: Lai, SH., Lepetit, V., Nishino, K., Sato, Y. (eds) Computer Vision – ACCV 2016. ACCV 2016. Lecture Notes in Computer Science(), vol 10114. Springer, Cham. https://doi.org/10.1007/978-3-319-54190-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-54190-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-54189-1
Online ISBN: 978-3-319-54190-7
eBook Packages: Computer ScienceComputer Science (R0)