A High-Performance Dense Optical Flow Architecture Based on Red-Black SOR Solver

  • Bibin JohnsonEmail author
  • Sachin Thomas
  • Rani J. Sheeba


Optical flow (OF) is an integral part of many vision systems, especially in the embedded and mobile application with ever-increasing challenges in achieving higher speed, minimal resource and lower power consumption. The work introduces a Dense High Throughput Optical Flow (DHTOF) architecture based on a novel fast converging Red-Black Successive Over Relaxation (RBSOR) solver architecture for computing dense and accurate OF using Horn and Schunck Optical Flow (HSOF) algorithm from Full High Definition (FHD) frames in real-time. The DHTOF architecture can capture dense OF from Ultra High Definition (UHD) frames at 48 Frames Per Second (FPS) with a throughput of 406 Megapixels/sec achieving a Throughput Per Watt (TPW) of 43 Giga Operation Per Second Per Watt (GOPS/Watt). The superscalar and deeply pipelined DHTOF architecture achieve same or lower Average Angular Error (AAE) with ≈ 4 × lesser number of RBSOR solver iterations as compared to the prior HSOF implementations based on Jacobi solver. It consumes 12.5 × lesser resources and 29.3% lower power for FHD resolution when compared to prior architectures. The proposed DHTOF architecture achieves highest area delay normalized speedup (at least by 28.2 ×) among the state of the art HSOF architectures. The successful evaluation of the proposed architecture for real-time OF sensor is demonstrated in Xilinx Virtex-VC707 Field Programmable Gate Array (FPGA) evaluation board.


Optical flow Horn and Schunck Red Black SOR FPGA Real-time 



  1. 1.
    Horn, B.K., & Schunck, B.G. (1981). Determining optical flow. Artificial intelligence, 17(1-3), 185–203.CrossRefGoogle Scholar
  2. 2.
    Russell, D.L., & Young, L.C. (1976). Calculus of variations and control theory: Proceedings of a symposium conducted by the mathematics research center, University of Wisconsin-Madison, September 22-24, 1975. Academic Press, no. 36.Google Scholar
  3. 3.
    Tu, Z., Xie, W., Zhang, D., Poppe, R., Veltkamp, R.C., Li, B., Yuan, J. (2019). A survey of variational and cnn-based optical flow techniques. Signal Processing: Image Communication, 72, 9–24.Google Scholar
  4. 4.
    Zach, C., Pock, T., Bischof, H. (2007). A duality based approach for realtime tv-l1 optical flow. In: DAGM-symposium.Google Scholar
  5. 5.
    Lazcano, V., & Rivera, F. (2019). Gpu based horn-schunck method to estimate optical flow and occlusion. In Gopal, T., & Watada, J. (Eds.) Theory and applications of models of computation (pp. 424–437). Cham: Springer International Publishing.Google Scholar
  6. 6.
    Durkovic, M., Zwick, M., Obermeier, F., Diepold, K. (2006). Performance of optical flow techniques on graphics hardware. In: 2006 IEEE International Conference on Multimedia and Expo. IEEE, pp. 241–244.Google Scholar
  7. 7.
    Martín, J.L., Zuloaga, A., Cuadrado, C., Lázaro, J., Bidarte, U. (2005). Hardware implementation of optical flow constraint equation using fpgas. Computer Vision and Image Understanding, 98(3), 462–490.CrossRefGoogle Scholar
  8. 8.
    Bahar, M.R.B., & Karimian, G. (2012). High performance implementation of the horn and schunck optical flow algorithm on fpga, 20th Iranian Conference on Electrical Engineering (ICEE2012), pp. 736–741.Google Scholar
  9. 9.
    Rustam, R., Hamid, N.H., Hussin, F.A. (2012). Fpga-based hardware implementation of optical flow constraint equation of horn and schunck, 2012 4th International Conference on Intelligent and Advanced Systems (ICIAS2012), vol. 2, pp. 790–794.Google Scholar
  10. 10.
    Gultekin, G.K., & Saranli, A. (2013). An fpga based high performance optical flow hardware design for computer vision applications. Microprocessors and Microsystems, 37(3), 270–286.CrossRefGoogle Scholar
  11. 11.
    Kunz, M., Ostrowski, A., Zipf, P. (2014). An fpga-optimized architecture of horn and schunck optical flow algorithm for real-time applications. In: 2014 24th international conference on field programmable logic and applications (FPL), pp. 1–4.Google Scholar
  12. 12.
    Komorkiewicz, M., Kryjak, T., Gorgon, M. (2014). Efficient hardware implementation of the horn-schunck algorithm for high-resolution real-time dense optical flow sensor. Sensors, 14(2), 2860–2891.CrossRefGoogle Scholar
  13. 13.
    Karniadakis, G.E., & Kirby, R.M. II. (2003). Parallel scientific computing in C++ and MPI: a seamless approach to parallel algorithms and their implementation. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  14. 14.
    Evans, D.J. (1984). Parallel sor iterative methods. Parallel computing, 1(1), 3–18.CrossRefGoogle Scholar
  15. 15.
    Chen, W., Wang, Z., Wu, Q., Liang, J., Chai, Z. (2016). Implementing dense optical flow computation on a heterogeneous fpga soc in c. ACM Transactions on Architecture and Code Optimization (TACO), 13(3), 25.Google Scholar
  16. 16.
    Forsell, M.J. (2002). Architectural differences of efficient sequential and parallel computers. Journal of Systems Architecture, 47(13), 1017–1041.CrossRefGoogle Scholar
  17. 17.
    Johnson, B., & et al. (2016). A high throughput fully parallel-pipelined fpga accelerator for dense cloud motion analysis. In: 2016 IEEE Region 10 Conference (TENCON). IEEE, pp. 2589–2592.Google Scholar
  18. 18.
    Baker, S., Scharstein, D., Lewis, J.P., Roth, S., Black, M.J., Szeliski, R. (2007). A database and evaluation methodology for optical flow, 2007 IEEE 11th international conference on computer vision, pp. 1–8.Google Scholar
  19. 19.
    Isa, M., Benkrid, K., Clayton, T. (2012). Efficient architecture and scheduling technique for pairwise sequence alignment. ACM SIGARCH Computer Architecture News, 40(4), 26–31.CrossRefGoogle Scholar
  20. 20.
    Kutty, J S.S., Boussaid, F., Amira, A. (2014). A high speed configurable fpga architecture for bilateral filtering. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 1248–1252.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of AvionicsIndian Institute of Space Science and TechnologyTrivandrumIndia

Personalised recommendations