Abstract
The existing high-quality environment matting methods usually require the capturing of a few thousand sample images and spend a few hours in data acquisition. In this paper, a novel environment matting algorithm is proposed to capture and extract the environment matte data effectively and efficiently. First, the recently developed compressive sensing theory is incorporated to reformulate the environment matting problem and simplify the data acquisition process. Next, taking into account the special properties of light refraction and reflection effects of transparent objects, two advanced priors, group clustering and Gaussian priors, as well as other basic constraints are introduced during the matte data recovery process to combat with the limited image samples, suppress the effects of the measurement noise resulted from data acquisition, and faithfully recover the sparse environment matte data. Compared with most of the existing environment matting methods, our algorithm significantly simplifies and accelerates the environment matting extraction process while still achieving high-accurate composition results.
Similar content being viewed by others
References
Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28(3), 253–263 (2008)
Candès, E.J., Romberg, J.K., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)
Candès, E.J., Tao, T.: Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006)
Chuang, Y.-Y., Zongker, D.E., Hindorff, J., Curless, B., Salesin, D., Szeliski, R.: Environment matting extensions: towards higher accuracy and real-time capture. In: SIGGRAPH ’00, pp. 121–130. ACM Press, New York (2000)
Duan, Q., Cai, J., Zheng, J., Lin, W.: Fast environment matting extraction using compressive sensing. In: 2011 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2011)
Duarte, M., Davenport, M., Takhar, D., Laska, J., Sun, T., Kelly, K., Baraniuk, R.: Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25(2), 83–91 (2008)
Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., Sagar, M.: Acquiring the reflectance field of a human face. In: SIGGRAPH ’00, pp. 145–156. ACM Press, New York (2000)
Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
Fuchs, M., Blanz, V., Lensch, H.P., Seidel, H.-P.: Adaptive sampling of reflectance fields. ACM Trans. Gr. 26(2), 10 (2007)
Goyal, V., Fletcher, A., Rangan, S.: Compressive sampling and lossy compression. IEEE Signal Process. Mag. 25(2), 48–56 (2008)
Gu, J., Nayar, S., Grinspun, E., Belhumeur, P., Ramamoorthi, R.: Compressive structured light for recovering inhomogeneous participating media. In: Proceedings of the 10th European Conference on Computer Vision, ECCV 2008, pp. 845–858. Springer, Berlin (2008)
Garg, G.G., Talvala, E.-V., Levoy, M., Lensch, H.P.A.: Symmetric photography: Exploiting data-sparseness in reflectance fields. In: Anenine-Möller, T., Heidrich, W., (eds.) Rendering Techniques 2006: Eurographics Symposium on Rendering, pp. 251–262. Eurographics Association, Nicosia (2006)
Haldar, J., Hernando, D., Liang, Z.P.: Compressed-sensing MRI with random encoding. IEEE Trans. Med. Imaging 30(4), 893–903 (2011)
Huang, J., Huang, X., Metaxas, D.N.: Learning with dynamic group sparsity. In: IEEE 12th International Conference on Computer Vision, ICCV 2009, pp. 64–71. Kyoto (2009)
Ihrke, I., Ziegler, G., Tevs, A., Theobalt, C., Magnor, M.A., Seidel, H.-P.: Eikonal rendering: efficient light transport in refractive objects. ACM Trans. Gr. 26(3), 59 (2007)
Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn. Reson. Med 58(6), 1182–1195 (2007)
Levoy, M., Hanrahan, P.: Light field rendering. In: SIGGRAPH, pp. 31–42 (1996)
Matusik, W., Loper, M., Pfister, H.: Progressively-refined reflectance functions from natural illumination. In: Proceedings of the 15th Eurographics workshop on rendering, Eurographics Association, pp. 299–308 (2004)
Masselus, V., Peers, P., Dutré, P., Willems, Y.D.: Relighting with 4D incident light fields. In: SIGGRAPH ’03, pp. 613–620. ACM, New York (2003)
Matusik, W., Pfister, H., Ziegler, R., Ngan, A., McMillan, L.: Acquisition and rendering of transparent and refractive objects. In: Proceedings of the 13th Eurographics workshop on rendering, Eurographics Association, pp. 267–278 (2002)
Peers, P., Dutré, P.: Wavelet environment matting. In: Proceedings of the 14th Eurographics workshop on rendering, Eurographics Association, pp. 157–166 (2003)
Peers, P., Dutré, P.: Inferring reflectance functions from wavelet noise. In: Proceedings of the 16th Eurographics workshop on rendering, Eurographics Association, pp. 173–182 (2005)
Peyre, G.: Best basis compressed sensing. IEEE Trans. Signal Process. 58(5), 2613–2622 (2010)
Peers, P., Mahajan, D., Lamond, B., Ghosh, A., Matusik, W., Ramamoorthi, R., Debevec, P.E.: Compressive light transport sensing. ACM Trans. Gr. 28(1), 1355–1363 (2009)
Sen, P., Chen, B., Garg, G., Marschner, S.R., Horowitz, M., Levoy, M., Lensch, H.P.A.: Dual photography. ACM Trans. Gr. 24(3), 745–755 (2005)
Sen, P., Darabi, S.: Compressive dual photography. Comput. Gr. Forum 28(2), 609–618 (2009)
Sen, P., Darabi, S.: Compressive estimation for signal integration in rendering. Comput. Gr. Forum 29(4), 1355–1363 (2010)
Sen, P., Darabi, S.: Compressive rendering: a rendering application of compressed sensing. IEEE Trans. Vis. Comput. Gr. 17(4), 487–499 (2011)
Wang, J., Dong, Y., Tong, X., Lin, Z., Guo, B.: Kernel nyström method for light transport. In: SIGGRAPH ’09: ACM SIGGRAPH 2009 papers, , pp. 1–10. ACM, New York (2009)
Wexler, Y., Fitzgibbon, A.W., Zisserman, A.: Image-based environment matting. In: Rendering Techniques, pp. 279–290 (2002)
Wakin, M., Laska, J., Duarte, M., Baron, D., Sarvotham, S., Takhar, D., Kelly, K., Baraniuk, R.: An architecture for compressive imaging. In: 2006 IEEE International Conference on Image Processing, pp. 1273–1276 (2006)
Zongker, D.E., Werner, D.M., Curless, B., Salesin, D.: Environment matting and compositing. In: SIGGRAPH, pp. 205–214 (1999)
Zhu, J., Yang, Y.-H.: Frequency-based environment matting. In: Pacific Conference on Computer Graphics and Applications, pp. 402–410. IEEE Computer Society (2004)
Zhang, S., Zhan, Y., Dewan, M., Huang, J., Metaxas, D.N., Zhou, X.S.: Towards robust and effective shape modeling: Sparse shape composition. Med Image Anal 16(1), 265–277 (2012)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Duan, Q., Cai, J. & Zheng, J. Compressive environment matting. Vis Comput 31, 1587–1600 (2015). https://doi.org/10.1007/s00371-014-1035-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-014-1035-1