Skip to main content
SpringerLink
Log in

A High-Performance Dense Optical Flow Architecture Based on Red-Black SOR Solver

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

Optical flow (OF) is an integral part of many vision systems, especially in the embedded and mobile application with ever-increasing challenges in achieving higher speed, minimal resource and lower power consumption. The work introduces a Dense High Throughput Optical Flow (DHTOF) architecture based on a novel fast converging Red-Black Successive Over Relaxation (RBSOR) solver architecture for computing dense and accurate OF using Horn and Schunck Optical Flow (HSOF) algorithm from Full High Definition (FHD) frames in real-time. The DHTOF architecture can capture dense OF from Ultra High Definition (UHD) frames at 48 Frames Per Second (FPS) with a throughput of 406 Megapixels/sec achieving a Throughput Per Watt (TPW) of 43 Giga Operation Per Second Per Watt (GOPS/Watt). The superscalar and deeply pipelined DHTOF architecture achieve same or lower Average Angular Error (AAE) with ≈ 4 × lesser number of RBSOR solver iterations as compared to the prior HSOF implementations based on Jacobi solver. It consumes 12.5 × lesser resources and 29.3% lower power for FHD resolution when compared to prior architectures. The proposed DHTOF architecture achieves highest area delay normalized speedup (at least by 28.2 ×) among the state of the art HSOF architectures. The successful evaluation of the proposed architecture for real-time OF sensor is demonstrated in Xilinx Virtex-VC707 Field Programmable Gate Array (FPGA) evaluation board.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16

Similar content being viewed by others

References

  1. Horn, B.K., & Schunck, B.G. (1981). Determining optical flow. Artificial intelligence, 17(1-3), 185–203.

    Article  Google Scholar 

  2. Russell, D.L., & Young, L.C. (1976). Calculus of variations and control theory: Proceedings of a symposium conducted by the mathematics research center, University of Wisconsin-Madison, September 22-24, 1975. Academic Press, no. 36.

  3. Tu, Z., Xie, W., Zhang, D., Poppe, R., Veltkamp, R.C., Li, B., Yuan, J. (2019). A survey of variational and cnn-based optical flow techniques. Signal Processing: Image Communication, 72, 9–24.

    Google Scholar 

  4. Zach, C., Pock, T., Bischof, H. (2007). A duality based approach for realtime tv-l1 optical flow. In: DAGM-symposium.

  5. Lazcano, V., & Rivera, F. (2019). Gpu based horn-schunck method to estimate optical flow and occlusion. In Gopal, T., & Watada, J. (Eds.) Theory and applications of models of computation (pp. 424–437). Cham: Springer International Publishing.

  6. Durkovic, M., Zwick, M., Obermeier, F., Diepold, K. (2006). Performance of optical flow techniques on graphics hardware. In: 2006 IEEE International Conference on Multimedia and Expo. IEEE, pp. 241–244.

  7. Martín, J.L., Zuloaga, A., Cuadrado, C., Lázaro, J., Bidarte, U. (2005). Hardware implementation of optical flow constraint equation using fpgas. Computer Vision and Image Understanding, 98(3), 462–490.

    Article  Google Scholar 

  8. Bahar, M.R.B., & Karimian, G. (2012). High performance implementation of the horn and schunck optical flow algorithm on fpga, 20th Iranian Conference on Electrical Engineering (ICEE2012), pp. 736–741.

  9. Rustam, R., Hamid, N.H., Hussin, F.A. (2012). Fpga-based hardware implementation of optical flow constraint equation of horn and schunck, 2012 4th International Conference on Intelligent and Advanced Systems (ICIAS2012), vol. 2, pp. 790–794.

  10. Gultekin, G.K., & Saranli, A. (2013). An fpga based high performance optical flow hardware design for computer vision applications. Microprocessors and Microsystems, 37(3), 270–286.

    Article  Google Scholar 

  11. Kunz, M., Ostrowski, A., Zipf, P. (2014). An fpga-optimized architecture of horn and schunck optical flow algorithm for real-time applications. In: 2014 24th international conference on field programmable logic and applications (FPL), pp. 1–4.

  12. Komorkiewicz, M., Kryjak, T., Gorgon, M. (2014). Efficient hardware implementation of the horn-schunck algorithm for high-resolution real-time dense optical flow sensor. Sensors, 14(2), 2860–2891.

    Article  Google Scholar 

  13. Karniadakis, G.E., & Kirby, R.M. II. (2003). Parallel scientific computing in C++ and MPI: a seamless approach to parallel algorithms and their implementation. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  14. Evans, D.J. (1984). Parallel sor iterative methods. Parallel computing, 1(1), 3–18.

    Article  Google Scholar 

  15. Chen, W., Wang, Z., Wu, Q., Liang, J., Chai, Z. (2016). Implementing dense optical flow computation on a heterogeneous fpga soc in c. ACM Transactions on Architecture and Code Optimization (TACO), 13(3), 25.

    Google Scholar 

  16. Forsell, M.J. (2002). Architectural differences of efficient sequential and parallel computers. Journal of Systems Architecture, 47(13), 1017–1041.

    Article  Google Scholar 

  17. Johnson, B., & et al. (2016). A high throughput fully parallel-pipelined fpga accelerator for dense cloud motion analysis. In: 2016 IEEE Region 10 Conference (TENCON). IEEE, pp. 2589–2592.

  18. Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R. (2007). A database and evaluation methodology for optical flow, 2007 IEEE 11th international conference on computer vision, pp. 1–8.

  19. Isa, M., Benkrid, K., Clayton, T. (2012). Efficient architecture and scheduling technique for pairwise sequence alignment. ACM SIGARCH Computer Architecture News, 40(4), 26–31.

    Article  Google Scholar 

  20. Kutty, J S.S., Boussaid, F., Amira, A. (2014). A high speed configurable fpga architecture for bilateral filtering. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 1248–1252.

Download references

Author information

Authors and Affiliations

  1. Department of Avionics, Indian Institute of Space Science and Technology, Trivandrum, India

    Bibin Johnson, Sachin Thomas & Rani J. Sheeba

Authors
  1. Bibin Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sachin Thomas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rani J. Sheeba
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bibin Johnson.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Johnson, B., Thomas, S. & Sheeba, R.J. A High-Performance Dense Optical Flow Architecture Based on Red-Black SOR Solver. J Sign Process Syst 92, 357–373 (2020). https://doi.org/10.1007/s11265-019-01490-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-019-01490-5

Keywords

Search

Navigation