Skip to main content

All-in-focus and multi-focus color image reconstruction from a database of color and depth image pairs

Multimedia Tools and Applications volume 75, pages 15493–15507 (2016)Cite this article

Abstract

Owing to recent advances in depth sensors and computer vision algorithms, depth images are often available with co-registered color images. In this paper, we propose a simple but effective method for obtaining an all-in-focus (AIF) color image from a database of color and depth image pairs. Since the defocus blur is inherently depth-dependent, the color pixels are first grouped according to their depth values. The defocus blur parameters are then estimated using the amount of the defocus blur of the grouped pixels. Given a defocused color image and its estimated blur parameters, the AIF image is produced by adopting the conventional pixel-wise mapping technique. In addition, the availability of the depth image disambiguates the objects located far or near from the in-focus object and thus facilitates image refocusing. We demonstrate the effectiveness of the proposed algorithm using both synthetic and real color and depth images.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. Ayatollahi S M, Moghadam A M E, Hosseini M S (2014) A taxonomy of depth map creation methods used in multiview video compression. Multimedia Tools Appl 72(2):1887–1909

    Article  Google Scholar 

  2. Cao Y, Fang S, Wang Z (2013) Digital multi-focusing from a single photograph taken by an uncalibrated conventional camera. IEEE Trans Image Process 22(9):3703–3714

    Article  Google Scholar 

  3. Fletcher R (1982) Second order corrections for non-differentiable optimization. Numer Anal 912:85–144

    Article  MathSciNet  MATH  Google Scholar 

  4. He L, Bleyer M, Gelautz M (2011) Object removal by depth-guided inpainting. In: Proceedings of AAPR Workshop, pp 1–8

  5. Jung S-W, Ko S-J (2012) Depth map based image enhancement using color stereopsis. IEEE Signal Process Lett 19(5):303–306

    Article  Google Scholar 

  6. Jung S-W, Choi O (2013) Color image enhancement using depth and intensity measurements of a time-of-flight depth camera. Opt Eng 52(10):1–11

    Google Scholar 

  7. Kodama K, Kubota A (2013) Efficient reconstruction of all-in-focus images through shifted pinholes from multi-focus images for dense light field synthesis and rendering. IEEE Trans Image Process 22(11):4407–4421

    Article  MathSciNet  Google Scholar 

  8. Lai K, Bo L, Ren X, Fox D (2011) A large-scale hierarchical multi-view RGB-D object dataset, pp 1817–1824

  9. Pentland A P (1987) A new sense for depth of field. IEEE Trans Pattern Anal Mach Intell 9(4):523–531

    Article  Google Scholar 

  10. Pertuz S, Puig D, Garcia M A, Fusiello A (2013) Generation of all-in-focus images by noise-robust selective fusion of limited depth-of-field images. IEEE Trans Image Process 22(3):1242–1251

    Article  MathSciNet  Google Scholar 

  11. Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1):7–42

    Article  MATH  Google Scholar 

  12. Zhang W, Cham W-K (2012) Single-image refocusing and defocusing. IEEE Trans Image Process 21(2):873–882

    Article  MathSciNet  Google Scholar 

  13. Zhang X, Constable M, Chan KL (2011) Aesthetic enhancement of landscape photographs as informed by paintings across depth layers. In: Proceedings IEEE International Conference on Image Processing, pp 1113–1116

  14. Zhou S, Sim T (2011) Defocus map estimation from a single image. Pattern Recognition 44(9):1852–1858

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Science, ICT & Future Planning(NRF- 2014R1A1A2057970) and by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the ITRC(Information Technology Research Center) support program (NIPA-2015-H0301-15-1021) supervised by the NIPA (National IT Industry Promotion Agency).

Author information

Authors and Affiliations

  1. Department of Multimedia Engineering, Dongguk University, Pildong-ro 1gil, Jung-gu, Seoul, 100-715, South Korea

    Seung-Won Jung & Young-Sik Jeong

  2. Department of Computer Science and Engineering, Seoul National University of Science and Technology, Gongneung-ro, Nowon-gu, Seoul, 139-743, South Korea

    Jong Hyuk Park

Authors
  1. Seung-Won Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jong Hyuk Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Young-Sik Jeong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jong Hyuk Park or Young-Sik Jeong.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jung, SW., Park, J.H. & Jeong, YS. All-in-focus and multi-focus color image reconstruction from a database of color and depth image pairs. Multimed Tools Appl 75, 15493–15507 (2016). https://doi.org/10.1007/s11042-015-2535-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2535-3

Keywords

  • All-in-focus
  • Defocus blur
  • Depth image
  • Image refocus
  • Multi-focus