Neural Computing and Applications

, Volume 30, Issue 9, pp 2697–2708 | Cite as

A neuro-inspired visual tracking method based on programmable system-on-chip platform

  • Shufan YangEmail author
  • KongFatt Wong-Lin
  • James Andrew
  • Terrence Mak
  • T. Martin McGinnity
Original Article


Using programmable system-on-chip to implement computer vision functions poses many challenges due to highly constrained resources in cost, size and power consumption. In this work, we propose a new neuro-inspired image processing model and implemented it on a system-on-chip Xilinx Z702c board. With the attractor neural network model to store the object’s contour information, we eliminate the computationally expensive steps in the curve evolution re-initialisation at every new iteration or frame. Our experimental results demonstrate that this integrated approach achieves accurate and robust object tracking, when they are partially or completely occluded in the scenes. Importantly, the system is able to process 640 by 480 videos in real-time stream with 30 frames per second using only one low-power Xilinx Zynq-7000 system-on-chip platform. This proof-of-concept work has demonstrated the advantage of incorporating neuro-inspired features in solving image processing problems during occlusion.


Visual object tracking Mean-shift Level set Attractor neural network model Occlusion System-on-chip 



SY was supported by the Early Research Scheme Reward from University of Wolverhampton and National High Technology Research and Development Program from China. KFW-L and SY were supported by ASUR (1014-C4-Ph1-071).


  1. 1.
    Malamas EN, Petrakis EG, Zervakis M, Petit L, Legat JD (2003) A survey on industrial vision systems, applications and tools. Image Vis Comput 21(2):171–188CrossRefGoogle Scholar
  2. 2.
    Nikitakis A, Papaioannou S, Papaefstathiou I (2013) A novel low-power embedded object recognition system working at multi-frames per second. ACM Trans Embed Comput Syst (TECS) 12(33):39–58Google Scholar
  3. 3.
    Díaz J, Ros E, Pelayo F, Ortigosa EM, Mota S (2006) FPGA-based real-time optical-flow system. Circuits Syst Video Technol IEEE Trans 16(2):274–279CrossRefGoogle Scholar
  4. 4.
    Jin J, Lee S, Jeon B, Nguyen TT, Jeon JW (2013) Real-time multiple object centroid tracking for gesture recognition based on FPGA. In: Proceedings of the 7th international conference on ubiquitous information management and communication, article 80Google Scholar
  5. 5.
    Nguyen HT, Smeulders A (2004) Tracking aspects of the foreground against the background. In Computer Vision-ECCV 2004. Springer, Berlin, pp 446–456Google Scholar
  6. 6.
    Lee BY, Liew LH, Cheah WS, Wang YC (2014) Occlusion handling in videos object tracking: a survey. In IOP conference series: earth and environmental science, vol 18(1). IOP Publishing, p 012020Google Scholar
  7. 7.
    Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv (CSUR) 38(4):13CrossRefGoogle Scholar
  8. 8.
    Yantis S, Johnson DN (1990) Mechanisms of attentional priority. J Exp Psychol Hum Percept Perform 16(4):812–825Google Scholar
  9. 9.
    Frintrop S, Rome E, Christensen HI (2010) Computational visual attention systems and their cognitive foundations: a survey. ACM Transact Appl Percept (TAP) 7(1):1–39CrossRefGoogle Scholar
  10. 10.
    García GM, Frintrop S, Cremers AB (2013) Attention-based detection of unknown objects in a situated vision framework. KI-Künstliche Intelligenz 27(3):267–272CrossRefGoogle Scholar
  11. 11.
    Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. Pattern Anal Mach Intell IEEE Trans 24(5):603–619CrossRefGoogle Scholar
  12. 12.
    Perez P, Vermaak J, Blake A (2004) Data fusion for visual tracking with particles. Proc IEEE 92(3):495–513CrossRefGoogle Scholar
  13. 13.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Computer vision and pattern recognition, 2005. CVPR 2005. IEEE computer society conference on IEEE, vol 1, pp 886–893Google Scholar
  14. 14.
    Epanechnikov VA (1969) Non-parametric estimation of a multivariate probability density. Theory Probab Appl 14:153–158. doi: 10.1137/1114019 MathSciNetCrossRefGoogle Scholar
  15. 15.
    Han M, Xu W, Gong Y (2004) An algorithm for multiple object trajectory tracking. Proc IEEE Comput Soc Conf 1:864–871Google Scholar
  16. 16.
    Chan TE, Vese LA (2001) A level set algorithm for minimizing the Mumford–Shah functional in image processing. In: Variational and level set methods in computer vision, 2001. Proceedings of IEEE Workshop on IEEE, pp 161–168Google Scholar
  17. 17.
    Osher S, Sethian JA (1988) Fronts propagating with curvature-dependent speed: algorithms based on Hamilton–Jacobi formulations. J Comput Phys 79(1):12–49MathSciNetCrossRefGoogle Scholar
  18. 18.
    Mendi E, Milanova M (2010) Contour-based image segmentation using selective visual attention. J Softw Eng Appl 3(08):796CrossRefGoogle Scholar
  19. 19.
    Cremers D (2006) Dynamical statistical shape priors for level set-based tracking. IEEE Trans Pattern Anal Mach Intell 28(8):1262–1273CrossRefGoogle Scholar
  20. 20.
    Yang S, McGinnity TM (2011) A biologically plausible real-time spiking neuron simulation environment based on a multiple-FPGA platform. ACM SIGARCH Comput Archit News 39(4):78–81CrossRefGoogle Scholar
  21. 21.
    Johnson JS, Spencer JP, Luck SJ, Schöner G (2009) A dynamic neural field model of visual working memory and change detection. Psychol Sci 20(5):568–577CrossRefGoogle Scholar
  22. 22.
    Xilinx (2014) Xilinx Xpower analyzer.
  23. 23.
    Cehovin L, Kristan M, Leonardis A (2013) Robust visual tracking using an adaptive coupled-layer visual model. IEEE Trans Pattern Anal Mach Intell 35(4):941–953CrossRefGoogle Scholar
  24. 24.
    Dollár P, Wojek C, Schiele B, Perona P (2009) Pedestrian detection: a benchmark. In: Computer vision and pattern recognition, 2009. CVPR 2009. IEEE conference on IEEE, pp 304–311Google Scholar
  25. 25.
    Kruger N, Janssen P, Kalkan S, Lappe M, Leonardis A, Piater J, Wiskott L (2013) Deep hierarchies in the primate visual cortex: What can we learn for computer vision? Pattern Anal Mach Intell IEEE Trans 35(8):1847–1871CrossRefGoogle Scholar
  26. 26.
    Fung CC, Wong KY, Wang H, Wu S (2012) Dynamical synapses enhance neural information processing: gracefulness, accuracy, and mobility. Neural Comput 24:1147–1185MathSciNetCrossRefGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2017

Authors and Affiliations

  1. 1.School of Mathematics and Computer Science, Faculty of Science and Engineering, City CampusUniversity of WolverhamptonWolverhamptonEngland, UK
  2. 2.Intelligent Systems Research CentreUniversity of UlsterLondonderryUK
  3. 3.Department of Electronics and Computer ScienceUniversity of SouthamptonSouthamptonUK
  4. 4.School of Science and TechnologyNottingham Trent UniversityNottinghamUK

Personalised recommendations