Abstract
Depth from defocus is a ranging method that provides depth estimation at every pixel in an image. It uses a pair of defocused images from a conventional monocular camera. To enable video-rate processing, which is important for industrial applications, an established algorithm was implemented on a field programmable gate array (FPGA). A bifurcating pipeline of 2-D filters provided the depth calculation. Multiplier and SRAM facilities on the FPGA were utilised efficiently by exploiting the filter symmetry. The coefficient and data bit-widths were limited to improve efficiency. The results compared favourably with the full width calculation. Range images were processed within 14 ms.
Similar content being viewed by others
References
Starck, J., Maki, A., Nobuhara, S., Hilton, A., Matsuyama, T.: The multiple-camera 3-D production studio. IEEE Trans. Circuits Syst. Video Technol. 19, 856–859 (2009)
De Silva, D.V.S.X., Fernando, W.A.C., Yasakethu, S.L.P.: Object based coding of the depth maps for 3D video coding. IEEE Trans. Consum. Electron. 55, 1699–1706 (2009)
Andreopoulos, A., Hasler, S., Wersing, H., Janssen, H., Tsotsos, J.K., Korner, E.: Active 3D object localization using a humanoid robot. IEEE Trans. Robot. 27, 47–64 (2011)
De Cristóforis, P., Nitsche, M.A., Krajník, T., Mejail, M.: Real-time monocular image-based path detection: a GPU-based embedded solution for on-board execution on mobile robots. J. Real-time Image Process. 1–14 (2013). doi:10.1007/s11554-013-0356-z
Taylor, R.H.: A perspective on medical robotics. Proc. IEEE 94, 1652–1664 (2006)
Dandekar, O., Castro-Pareja, C., Shekhar, R.: FPGA-based real-time 3-D image preprocessing for image-guided medical interventions. J. Real-Time Image Process. 1, 285–301 (2007)
Broggi, A., Cappalunga, A., Caraffi, C., Cattani, S., Ghidoni, S., Grisleri, P., Porta, P.P., Posterli, M., Zani, P.: TerraMax vision at the urban challenge 2007. IEEE Trans. Intell. Transp. Syst. 11, 194–205 (2010)
Chang, K.I., Bowyer, K.W., Flynn, P.J.: An evaluation of multimodal 2D+3D face biometrics. IEEE Trans. Pattern Anal. Mach. Intell. 27, 619–624 (2005)
Pentland, A.P.: A new sense for depth of field. IEEE Trans. Pattern Anal. Mach. Intell. 9, 523–531 (1987)
Pentland, A., Darrell, T., Scherock, S., Girod, B.: Simple range cameras based on focal error. J Opt. Soc. Am. A 11(11), 2925–2934 (1994)
Subbarao, M., Wei, T.C.: Depth from defocus and rapid auto focussing: a practical approach, pp. 773–776. Proc. IEEE Comput. Soc. Conf. Compu. Vis. Pattern Recognit., Champaign (1992)
Subbarao, M., Surya, G.: Depth from defocus: spatial domain approach. Int. J. Compu. Vis. 13(3), 271–294 (1994)
Ens, J., Lawrence, P.: An investigation of methods for determining depth from defocus. IEEE Trans. Pattern Anal. Mach. Intell. 15, 97–108 (1993)
Watanabe, M., Nayar, S.K.: Rational filters for passive depth from defocus. Int. J. Compu. Vis. 27(3), 203–225 (1998)
Joseph-Raj, A.N., Staunton, R.C.: Rational filter design for depth from defocus. Pattern Recognit. 45, 198–207 (2012)
Joseph-Raj, A.N.: Accurate depth from defocus estimation with video-rate implementation. Ph.D. dissertation, School of Eng., Univ. Warwick, Coventry (2009)
Rajagopalan, A.N., Chaudhuri, S.: Depth from defocus: a real aperture imaging approach. Springer, New York (1999)
Rajagopalan, A.N., Chaudhuri, S.: An MRF model-based approach to simultaneous recovery of depth and restoration from defocused images. IEEE Trans. Pattern Anal. Mach. Intell. 21, 577–589 (1999)
Favaro, P., Soatto, S.: A geometric approach to shape from defocus. IEEE Trans. Pattern Anal. Mach. Intell. 27, 406–417 (2005)
Favaro, P., Soatto, S.: Learning shape from defocus. In: Proc. 7th European Conf. Computer Vision, Copenhagen, Part 2 pp. 735–745 (2002)
Deschenes, F., Ziou, D., Fuchs, P.: Improved estimation of defocus blur and spatial shifts in spatial domain: a homotopy-based approach. Pattern Recognit. 36(9), 2105–2125 (2003)
Deschenes, F., Ziou, D., Fuchs, P.: A homotopy-based approach for computing defocus blur and affine transform simultaneously. Pattern Recognit. 41(7), 2263–2282 (2008)
Watanabe, M., Nayar, S.K.: Minimal operator set for passive depth from defocus, pp. 431–438. IEEE Conference on CVPR, San Francisco (1996)
Nayar, S.K., Watanabe, M., Noguchi, M.: Real-time focus range sensor. IEEE Trans. Pattern Anal. Mach. Intell. 18(12), 1186–1198 (1996)
Ghita, O., Whelan, P.F.: A video-rate sensor based on depth from defocus. Opt. Laser Technol. 33(3), 167–176 (2001)
Leroy, J.V., Simon, T., Deschenes, F.: Real time monocular depth from defocus. In: 3rd International Conference on Image and Signal Processing, pp. 103–111 (2008)
Favaro P.: Recovering thin structures via nonlocal-means regularization with application to depth from defocus. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1133–1140 (2010)
Ben-Ari R., Raveh G.: Variational, depth from defocus in real-time. IEEE Int. Conf. Comput. Vis. 522–529 (2011)
Zhou, C., Lin, S., Nayar, S.K.: Coded aperture pairs for depth from defocus and defocus deblurring. Int. J. Comput. Vis. 93(1), 53–72 (2011)
Jin, S., Kim, D., Nguyen, T.T., Kim, D., Kim, M., Jeon, J.W.: Design and implementation of a pipelined datapath for high-speed face detection using FPGA. IEEE Trans. Ind. Informat. 8(1), 158–167 (2012)
Monmasson, E., Idkhajine, L., Cirstea, M.N., Bahri, I., Tisan, A., Naouar, M.W.: FPGAs in industrial control applications. IEEE Trans. Ind. Informat. 7, 224–243 (2011)
Xilinx, Virtex-II Pro/Virtex-II Pro X, Complete Data Sheet [Online]. Available: http://www.xilinx.com/support/documentation/user_guides/ug012.pdf. Accessed 14 Dec 2014
Reddy, H.C., Khoo, I.H., Rajan, P.K.: 2-D symmetry: theory and filter design applications. IEEE Circuits Syst. Mag. 3(3), 4–33 (2003)
Cheng, Chao, Parhi, K.K.: Hardware efficient fast parallel FIR filter structures based on iterated short convolution. IEEE Trans. Circuits Syst. I Regul. Pap. 51(8), 1492–1500 (2004)
Bougas, P., Kalivas, P., Tsirikos, A., Pekmestzi, K.Z.: Pipelined array-based FIR filter folding. IEEE Trans. Circuits Syst. I Regul. Pap. 52(1), 108–118 (2005)
Lim, Jae S.: Two dimensional signal and image processing, pp. 196–199. Prentice Hall Inc, New Jersey (1990)
Kung, H.T.: Why systolic architectures? IEEE Trans. Compu. 15, 37–46 (1982)
Kung, H.T., Leiserson, C.E., Mead, C.A., Conway, L.A.: Systolic arrays (for VLSI). Sparse Matrix Proc. 1978, 256 (1979)
Hsia, S.-C.: Parallel VLSI design for a real-time video-impulse noise-reduction processor. IEEE Trans. Very Large Scale Integr. Syst. 11, 651–658 (2003)
Claxton, C.D., Staunton, R.C.: Measurement of the point-spread function of a noisy imaging system. J. Optical Soc. Am. A 25, 159–170 (2008)
Bonyadi, Y.: 3D TV from miniature camera systems. MSc. dissertation, School of Eng., Univ. Warwick, Coventry (2011)
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1
See Table 4.
Appendix 2: General triangular convolution
The following algorithm can be used to generate the partial equations used by the triangular method. For an m × m convolution where m is odd, there is eightfold symmetry, and the central pixel is I 0,0, then the pixels in the window will be indexed I −n,−n to I +n,+n , where n = (m −1)/2. The number of partial equations, coefficients and multipliers required \(a = \sum\nolimits_{i = 1}^{n + 1} i\). Let the partial results be P b and the coefficients C b where b is an integer from 1 to a. Then if n > 1:
As algorithm:
Rights and permissions
About this article
Cite this article
Joseph Raj, A.N., Staunton, R.C. Video-rate calculation of depth from defocus on a FPGA. J Real-Time Image Proc 14, 469–480 (2018). https://doi.org/10.1007/s11554-014-0480-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-014-0480-4