Signal, Image and Video Processing

, Volume 9, Issue 4, pp 809–818 | Cite as

VLSI architecture for 4-D depth filtering

  • Arjuna Madanayake
  • Randeel Wimalagunarathne
  • Donald G. Dansereau
  • Renato J. Cintra
  • Len T. Bruton
Original Paper


We propose the application of light field cameras and depth-selective 4-D IIR filtering to enable video surveillance, leveraging the post-capture depth-selective filtering enabled by computational photography. Novel ultralow-complexity differential-form depth-selective 4-D IIR filter algorithms and their corresponding architectures are proposed for processing 4-D light fields. Practical results are presented for real-world video sequences, and a CMOS VLSI implementation of the arithmetic processing elements is synthesized. The architecture shows \(86.66\), \(78.94\,\%\) reduction in multipliers and adders compared to direct-form structure and delivers 26 frames/s for light fields of size \(16\times 16\times 128\times 128\).


4-D filtering Light fields Video imaging and surveillance Plenoptic Digital architecture 


  1. 1.
    Adelson, E.H., Bergen, J.R.: The plenoptic function and the elements of early vision. Comput. Models Vis. Process. 1, 3–20 (1991)Google Scholar
  2. 2.
    Agathoklis, P., Bruton, L.T.: Practical-BIBO stability of N-dimensional discrete systems. Electron. Circuits Sys. IEEE Proc. G 130(6), 236–242 (1983)CrossRefGoogle Scholar
  3. 3.
    Bertschmann, R.K., Bartley, N.R., Bruton, L.T.: A 3-D integrator-differentiator double-loop (IDD) filter for raster-scan video processing. In: IEEE international symposium on circuits and systems, ISCAS’95, vol. 1, pp. 471–473 (1995)Google Scholar
  4. 4.
    Bruton, L.T., Bartley, N.R.: Three-dimensional image processing using the concept of network resonance. IEEE Trans. Circuits Sys. 32(7), 664–672 (1985)Google Scholar
  5. 5.
    Dansereau, D.G., Bruton, L.T.: A 4D frequency-planar IIR filter and its application to light field processing. In: Circuits and systems, 2003. ISCAS ’03, vol. 4, pp. 476–479 (2003)Google Scholar
  6. 6.
    Dansereau, D.G., Bruton, L.T.: A 4-D dual-fan filter bank for depth filtering in light fields. IEEE Trans. Signal Process. 55(2), 542–549 (2007)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Dansereau, D.G., Mahon, I., Pizarro, O., Williams, S.B.: Plenoptic flow: Closed-form visual odometry for light field cameras. In: Proceedings, 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 4455–4462. IEEE (2011) Google Scholar
  8. 8.
    Harris, M.: Focusing on everything: light field cameras promise an imaging revolution. IEEE Spectr. 49, 44–50 (2012)CrossRefGoogle Scholar
  9. 9.
    Jarvis, R.A.: A perspective on range finding techniques for computer vision. Pattern Anal. Mach. Intell. IEEE Trans. PAMI 5, 122–139 (1983)CrossRefGoogle Scholar
  10. 10.
    Joshi, N., Avidan, S., Matusik, W., Kriegman, D.J.: Synthetic aperture tracking: tracking through occlusions. In: IEEE international conference on computer vision (ICCV), pp. 1–8. IEEE (2007)Google Scholar
  11. 11.
    Lai, S.H., Fu, C.W., Chang, S.: A generalized depth estimation algorithm with a single image. Pattern Anal. Mach. Intell. IEEE Trans. 14, 405–411 (1992)CrossRefGoogle Scholar
  12. 12.
    Levoy, M., Hanrahan, P.: Light field rendering. In: Proceedings of the 23rd annual conference on computer graphics and interactive techniques, pp. 31–42. ACM, New York (1996)Google Scholar
  13. 13.
    Lumsdaine, A., Georgiev, T.: Full resolution lightfield rendering. Technical report, Adobe Systems (2008)Google Scholar
  14. 14.
    Madanayake, A., Bruton, L.T.: A fully-multiplexed first-order frequency-planar module for fan, beam, and cone plane-wave filters. IEEE Trans. Circuits Sys. II Express Briefs 53(8), 697–701 (2006)CrossRefGoogle Scholar
  15. 15.
    Madanayake, A., Wimalagunaratne, R., Dansereau, D.G., Bruton, L.T.: Design and FPGA-implementation of 1st-order 4D IIR frequency-hyperplanar digital filters. In: 2011 IEEE 54th international midwest symposium on circuits and systems (MWSCAS), pp. 1–4 (2011)Google Scholar
  16. 16.
    Ng, R.: Fourier slice photography. ACM Trans. Graph. (TOG) 24, 735–744 (2005)CrossRefGoogle Scholar
  17. 17.
    Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., Hanrahan, P.: Light field photography with a hand-held plenoptic camera. Computer Science Technical Report CSTR 2 (2005)Google Scholar
  18. 18.
    Ramamoorthy, P.A., Bruton, L.T.: Design of stable 2-dimensional discrete recursive filters. Electron. Lett. 12(25), 659–660 (1976)CrossRefGoogle Scholar
  19. 19.
    Smith, B.M., Zhang, L., Jin, H., Agarwala, A.: Light field video stabilization. In: 2009 IEEE 12th international conference on computer vision, pp. 341–348. IEEE (2010)Google Scholar
  20. 20.
    The (New) Stanford Light Field Archive (2012)
  21. 21.
    Vaish, V., Levoy, M., Szeliski, R., Zitnick, C., Kang, S.: Reconstructing occluded surfaces using synthetic apertures: Stereo, focus and robust measures. In: 2006 IEEE computer society conference on computer vision and pattern recognition, vol. 2, pp. 2331–2338 (2006)Google Scholar
  22. 22.
    Wilburn, B., Joshi, N., Vaish, V., Talvala, E.V., Antunez, E., Barth, A., Adams, A., Horowitz, M., Levoy, M.: High performance imaging using large camera arrays. ACM Trans. Graph. (TOG) 24, 765–776 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Arjuna Madanayake
    • 1
  • Randeel Wimalagunarathne
    • 1
  • Donald G. Dansereau
    • 2
  • Renato J. Cintra
    • 3
  • Len T. Bruton
    • 4
  1. 1.Department of Electrical and Computer EngineeringUniversity of AkronAkronUSA
  2. 2.Australian Centre for Field Robotics, School of Aerospace, Mechanical and Mechatronic EngineeringUniversity of SydneySydneyAustralia
  3. 3.Signal Processing Group, Departamento of EstatísticaUniversidade Federal de PernambucoRecifeBrazil
  4. 4.Department of Electrical and Computer EngineeringUniversity of CalgaryCalgaryCanada

Personalised recommendations