Skip to main content

All-in-focus and multi-focus color image reconstruction from a database of color and depth image pairs


Owing to recent advances in depth sensors and computer vision algorithms, depth images are often available with co-registered color images. In this paper, we propose a simple but effective method for obtaining an all-in-focus (AIF) color image from a database of color and depth image pairs. Since the defocus blur is inherently depth-dependent, the color pixels are first grouped according to their depth values. The defocus blur parameters are then estimated using the amount of the defocus blur of the grouped pixels. Given a defocused color image and its estimated blur parameters, the AIF image is produced by adopting the conventional pixel-wise mapping technique. In addition, the availability of the depth image disambiguates the objects located far or near from the in-focus object and thus facilitates image refocusing. We demonstrate the effectiveness of the proposed algorithm using both synthetic and real color and depth images.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9


  1. Ayatollahi S M, Moghadam A M E, Hosseini M S (2014) A taxonomy of depth map creation methods used in multiview video compression. Multimedia Tools Appl 72(2):1887–1909

    Article  Google Scholar 

  2. Cao Y, Fang S, Wang Z (2013) Digital multi-focusing from a single photograph taken by an uncalibrated conventional camera. IEEE Trans Image Process 22(9):3703–3714

    Article  Google Scholar 

  3. Fletcher R (1982) Second order corrections for non-differentiable optimization. Numer Anal 912:85–144

    Article  MathSciNet  MATH  Google Scholar 

  4. He L, Bleyer M, Gelautz M (2011) Object removal by depth-guided inpainting. In: Proceedings of AAPR Workshop, pp 1–8

  5. Jung S-W, Ko S-J (2012) Depth map based image enhancement using color stereopsis. IEEE Signal Process Lett 19(5):303–306

    Article  Google Scholar 

  6. Jung S-W, Choi O (2013) Color image enhancement using depth and intensity measurements of a time-of-flight depth camera. Opt Eng 52(10):1–11

    Google Scholar 

  7. Kodama K, Kubota A (2013) Efficient reconstruction of all-in-focus images through shifted pinholes from multi-focus images for dense light field synthesis and rendering. IEEE Trans Image Process 22(11):4407–4421

    Article  MathSciNet  Google Scholar 

  8. Lai K, Bo L, Ren X, Fox D (2011) A large-scale hierarchical multi-view RGB-D object dataset, pp 1817–1824

  9. Pentland A P (1987) A new sense for depth of field. IEEE Trans Pattern Anal Mach Intell 9(4):523–531

    Article  Google Scholar 

  10. Pertuz S, Puig D, Garcia M A, Fusiello A (2013) Generation of all-in-focus images by noise-robust selective fusion of limited depth-of-field images. IEEE Trans Image Process 22(3):1242–1251

    Article  MathSciNet  Google Scholar 

  11. Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1):7–42

    Article  MATH  Google Scholar 

  12. Zhang W, Cham W-K (2012) Single-image refocusing and defocusing. IEEE Trans Image Process 21(2):873–882

    Article  MathSciNet  Google Scholar 

  13. Zhang X, Constable M, Chan KL (2011) Aesthetic enhancement of landscape photographs as informed by paintings across depth layers. In: Proceedings IEEE International Conference on Image Processing, pp 1113–1116

  14. Zhou S, Sim T (2011) Defocus map estimation from a single image. Pattern Recognition 44(9):1852–1858

    Article  Google Scholar 

Download references


This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Science, ICT & Future Planning(NRF- 2014R1A1A2057970) and by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the ITRC(Information Technology Research Center) support program (NIPA-2015-H0301-15-1021) supervised by the NIPA (National IT Industry Promotion Agency).

Author information

Authors and Affiliations


Corresponding authors

Correspondence to Jong Hyuk Park or Young-Sik Jeong.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jung, SW., Park, J.H. & Jeong, YS. All-in-focus and multi-focus color image reconstruction from a database of color and depth image pairs. Multimed Tools Appl 75, 15493–15507 (2016).

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • All-in-focus
  • Defocus blur
  • Depth image
  • Image refocus
  • Multi-focus