Skip to main content
SpringerLink
Log in

Robust PCA-based solution to image composition using augmented Lagrange multiplier (ALM)

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Computational photography relies on specialized image-processing techniques to combine multiple images captured by a camera to generate a desired image of the scene. We first consider the high dynamic range (HDR) imaging problem. We can change either the exposure time or the aperture while capturing multiple images of the scene to generate an HDR image. This paper addresses the HDR imaging problem for static and dynamic scenes captured using a stationary camera under various aperture and exposure settings, when we do not have any knowledge of the camera settings. We have proposed a novel framework based on sparse representation which enables us to process images while getting rid of artifacts due to moving objects and defocus blur. We show that the proposed approach is able to produce significantly good results through dynamic object rejection and deblurring capabilities. We compare the results with other competitive approaches and discuss the relative advantages of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

References

  1. Peterson, B.: Understanding Exposure: How to Shoot Great Photographs with Any Camera. Random House Digital Inc, New York (2010)

    Google Scholar 

  2. Reinhard, E., Heidrich, W., Debevec, P., Pattanaik, S., Ward, G., Myszkowski, K.: High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting, vol. 2. Morgan Kaufmann, Burlington (2010)

    Google Scholar 

  3. Levin, A., Weiss, Y., Durand, F., Freeman, W.T.: Understanding blind deconvolution algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(12), 2354–2367 (2011)

    Article  Google Scholar 

  4. Debevec, P.E., Malik, J.: Recovering high dynamic range radiance maps from photographs. In: ACM SIGGRAPH. ACM, New York (1997)

  5. Mann, S., Picard, R.: Being Undigital with Digital Cameras, pp. 442–448. MIT Media Lab Perceptual, Cambridge (1994)

  6. Goshtasby, A.A.: Fusion of multi-exposure images. Image Vis. Comput. 236, 611–618 (2005)

  7. Mertens, T., Kautz, J., Van Reeth, F.: Exposure Fusion. Pacific Graphics, Frank (2007)

    Book  Google Scholar 

  8. Burt, P.J., Adelson, E.H.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31(4), 532–540 (1983)

  9. Raman, S., Chaudhuri, S., Matte-less, A.: IEEE International Conference on Variational Approach to Automatic Scene Compositing, Computer Vision. IEEE, New York, pp. 1–6 (2007)

  10. Granados, M., Seidel, H.P., Lensch, H.: Background estimation from non-time sequence images. In: Proceedings of Graphics Interface. Canadian Information Processing Society, Canada, pp. 33–40 (2008)

  11. Zhang, W., Cham, W.K.: Gradient-directed multiexposure composition. In: IEEE Transactions on Image Processing, vol. 21, no. 4. IEEE, New York, pp. 2318–2323 (2012)

  12. Raman, S., Chaudhuri, S.: Reconstruction of high contrast images for dynamic scenes. Vis. Comput. 27(12), 1099–1114 (2011)

  13. Hasinoff, S.W., Durand, F., Freeman, W.T.: Noise-optimal capture for high dynamic range photography. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, New York, pp. 553–560 (2010)

  14. Granados, M., Kim, K.I., Tompkin, J., Theobalt, C.: Automatic noise modeling for ghost-free HDR reconstruction. ACM Trans. Graph. (TOG) 32(6), 201 (2013)

  15. Hu, J., Gallo, O., Pulli, K., Sun, X.: HDR Deghosting: How to Deal with Saturation? IEEE CVPR, New York (2013)

  16. Wu, S., Xie, S., Rahardja, S., Li, Z.: A robust and fast anti-ghosting algorithm for high dynamic range imaging. In: 2010 17th IEEE International Conference on Image Processing (ICIP). IEEE, New York, pp. 397–400 (2010)

  17. Heo, Y.S., Lee, K.M., Lee, S.U.K., Moon, Y., Cha, J.: Ghost-free high dynamic range imaging. In: Computer Vision-ACCV 2010. Springer, Berlin pp. 486–500 (2011)

  18. Khan, E.A., Akyiiz, A.O., Reinhard, E.: Ghost removal in high dynamic range images. In: 2006 IEEE International Conference on Image Processing, 2005–2008. IEEE, New York (2006)

  19. Gallo, O., Gelfand, N.: Artifact-free high dynamic range imaging. In: Chen, W.-C. Tico, M., Pulli, K (eds.) IEEE ICCP, N. (2009)

  20. Ward, G.: Fast, robust image registration for compositing high dynamic range photographs from hand-held exposures. J. Graph. Tools 8(2), 17–30 (2003)

  21. Jacobs, K., Loscos, C., Ward, G.: Automatic high-dynamic range image generation for dynamic scenes. IEEE Comput. Graph. Appl. 28(2), 84–93 (2008)

    Article  Google Scholar 

  22. Zimmer, H., Bruhn, A., Weickert, J.: Freehand HDR imaging of moving scenes with simultaneous resolution enhancement. Computer Graphics Forum, Wiley Online Library 30(2), 405–414 (2011)

    Article  Google Scholar 

  23. Sen, P., Kalantari, N.K., Yaesoubi, M., Darabi, S., Goldman, D.B., Shechtman, E.: Robust patch-based HDR reconstruction of dynamic scenes. ACM Trans. Graph. 31(6), 203 (2012)

  24. Kalantari, N.K., Shechtman, E., Barnes, C., Darabi, S., Goldman, D.B., Sen, P.: Patch-based high dynamic range video. ACM Trans. Graph. (TOG) 32(6), 202 (2013)

  25. Srikantha, A., Sidibé, D.: Ghost detection and removal for high dynamic range images: recent advances. Signal Process. Image Commun. 27(6), 650–662 (2012)

  26. De la Torre, F., Black, M.J.: Robust principal component analysis for computer vision. In: Eighth IEEE International Conference on Computer Vision, 2001. ICCV 2001. Proceedings, vol. 1. IEEE, New York, pp. 362–369 (2001)

  27. De La Torre, F., Black, M.J.: A framework for robust subspace learning. Int. J. Comput. Vis. 54(1–3), 117–142 (2003)

  28. Ke, Q., Kanade, T.: Robust L 1 norm factorization in the presence of outliers and missing data by alternative convex programming. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol. 1. IEEE, New York, pp. 739–746 (2005)

  29. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

  30. Lee, C., Li, Y., Monga, V.: Ghost-Free High Dynamic Range Imaging via Rank Minimization. IEEE, New York (2014)

    Google Scholar 

  31. Wright, J., Ganesh, A., Rao, S., Peng, Y., Ma, Y.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. Adv. Neural Inf. Process. Syst. 2080–2088 (2009)

  32. Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM (JACM) 58(3), 11 (2011)

  33. Lin, Z., Chen, M., Ma, Y.: The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices (2010). arXiv:1009.5055

  34. Oh, T.H., Lee, J.Y., Kweon, I.S.: High Dynamic Range Imaging by a Rank-1 Constraint. IEEE ICIP, New York (2013)

  35. Oh, T.H., Lee, J.Y., Jean-Charles, B., Kweon, I.S.: Partial Sum Minimization of Singular Values in RPCA for Low-Level Vision. IEEE ICCV, New York (2013)

  36. Oh, T., Lee, J., Tai, Y., Kweon, I.: Robust high dynamic range imaging by rank minimization. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 99. IEEE, New York, pp. 1–1 (2014)

  37. Bhardwaj, A., Raman, S.: HDR Imaging Using Augmented Lagrange Multipliers (ALM), Eurographics 2014-Posters, pp. 3–4. The Eurographics Association, New York (2014)

    Google Scholar 

  38. Hasinoff, S.W., Kutulakos, K.N.: Confocal stereo. Int. J. Comput. Vis. 81(1), 82–104 (2009)

  39. Rajagopalan, A.N., Chaudhuri, S.: An MRF model-based approach to simultaneous recovery of depth and restoration from defocused images. In: IEEE Transactions on Pattern Analysis and Machine Intelligence 21(7), 577–589 (1999)

  40. Jin, H., Favaro, P.: A variational approach to shape from defocus. Comput. Vis. ECCV 2002, 18–30 (2002)

  41. Hasinoff, S., Kutulakos, W.: A Layer-based Restoration Framework for Variable-Aperture Photography. IEEE ICCV, Kiriakos N (2007)

  42. Bhardwaj, A., Raman, S.: PCA-HDR: A Robust PCA based Solution to HDR Imaging. SPCOM 2014. IEEE, New York (2014)

  43. Candès, E.J., Recht, B.: Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)

  44. Candès, E.J., Tao, T.: The power of convex relaxation: near-optimal matrix completion. In: IEEE Transactions on Information Theory, vol. 56, no. 5. IEEE, New York, pp. 2053–2080 (2010)

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Technology Gandhinagar, Ahmedabad, India

    Adit Bhardwaj & Shanmuganathan Raman

Authors
  1. Adit Bhardwaj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shanmuganathan Raman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adit Bhardwaj.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bhardwaj, A., Raman, S. Robust PCA-based solution to image composition using augmented Lagrange multiplier (ALM). Vis Comput 32, 591–600 (2016). https://doi.org/10.1007/s00371-015-1075-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-015-1075-1

Keywords

Search

Navigation