Abstract
Using programmable system-on-chip to implement computer vision functions poses many challenges due to highly constrained resources in cost, size and power consumption. In this work, we propose a new neuro-inspired image processing model and implemented it on a system-on-chip Xilinx Z702c board. With the attractor neural network model to store the object’s contour information, we eliminate the computationally expensive steps in the curve evolution re-initialisation at every new iteration or frame. Our experimental results demonstrate that this integrated approach achieves accurate and robust object tracking, when they are partially or completely occluded in the scenes. Importantly, the system is able to process 640 by 480 videos in real-time stream with 30 frames per second using only one low-power Xilinx Zynq-7000 system-on-chip platform. This proof-of-concept work has demonstrated the advantage of incorporating neuro-inspired features in solving image processing problems during occlusion.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Malamas EN, Petrakis EG, Zervakis M, Petit L, Legat JD (2003) A survey on industrial vision systems, applications and tools. Image Vis Comput 21(2):171–188
Nikitakis A, Papaioannou S, Papaefstathiou I (2013) A novel low-power embedded object recognition system working at multi-frames per second. ACM Trans Embed Comput Syst (TECS) 12(33):39–58
Díaz J, Ros E, Pelayo F, Ortigosa EM, Mota S (2006) FPGA-based real-time optical-flow system. Circuits Syst Video Technol IEEE Trans 16(2):274–279
Jin J, Lee S, Jeon B, Nguyen TT, Jeon JW (2013) Real-time multiple object centroid tracking for gesture recognition based on FPGA. In: Proceedings of the 7th international conference on ubiquitous information management and communication, article 80
Nguyen HT, Smeulders A (2004) Tracking aspects of the foreground against the background. In Computer Vision-ECCV 2004. Springer, Berlin, pp 446–456
Lee BY, Liew LH, Cheah WS, Wang YC (2014) Occlusion handling in videos object tracking: a survey. In IOP conference series: earth and environmental science, vol 18(1). IOP Publishing, p 012020
Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv (CSUR) 38(4):13
Yantis S, Johnson DN (1990) Mechanisms of attentional priority. J Exp Psychol Hum Percept Perform 16(4):812–825
Frintrop S, Rome E, Christensen HI (2010) Computational visual attention systems and their cognitive foundations: a survey. ACM Transact Appl Percept (TAP) 7(1):1–39
García GM, Frintrop S, Cremers AB (2013) Attention-based detection of unknown objects in a situated vision framework. KI-Künstliche Intelligenz 27(3):267–272
Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. Pattern Anal Mach Intell IEEE Trans 24(5):603–619
Perez P, Vermaak J, Blake A (2004) Data fusion for visual tracking with particles. Proc IEEE 92(3):495–513
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Computer vision and pattern recognition, 2005. CVPR 2005. IEEE computer society conference on IEEE, vol 1, pp 886–893
Epanechnikov VA (1969) Non-parametric estimation of a multivariate probability density. Theory Probab Appl 14:153–158. doi:10.1137/1114019
Han M, Xu W, Gong Y (2004) An algorithm for multiple object trajectory tracking. Proc IEEE Comput Soc Conf 1:864–871
Chan TE, Vese LA (2001) A level set algorithm for minimizing the Mumford–Shah functional in image processing. In: Variational and level set methods in computer vision, 2001. Proceedings of IEEE Workshop on IEEE, pp 161–168
Osher S, Sethian JA (1988) Fronts propagating with curvature-dependent speed: algorithms based on Hamilton–Jacobi formulations. J Comput Phys 79(1):12–49
Mendi E, Milanova M (2010) Contour-based image segmentation using selective visual attention. J Softw Eng Appl 3(08):796
Cremers D (2006) Dynamical statistical shape priors for level set-based tracking. IEEE Trans Pattern Anal Mach Intell 28(8):1262–1273
Yang S, McGinnity TM (2011) A biologically plausible real-time spiking neuron simulation environment based on a multiple-FPGA platform. ACM SIGARCH Comput Archit News 39(4):78–81
Johnson JS, Spencer JP, Luck SJ, Schöner G (2009) A dynamic neural field model of visual working memory and change detection. Psychol Sci 20(5):568–577
Xilinx (2014) Xilinx Xpower analyzer. www.xilinx.com/products
Cehovin L, Kristan M, Leonardis A (2013) Robust visual tracking using an adaptive coupled-layer visual model. IEEE Trans Pattern Anal Mach Intell 35(4):941–953
Dollár P, Wojek C, Schiele B, Perona P (2009) Pedestrian detection: a benchmark. In: Computer vision and pattern recognition, 2009. CVPR 2009. IEEE conference on IEEE, pp 304–311
Kruger N, Janssen P, Kalkan S, Lappe M, Leonardis A, Piater J, Wiskott L (2013) Deep hierarchies in the primate visual cortex: What can we learn for computer vision? Pattern Anal Mach Intell IEEE Trans 35(8):1847–1871
Fung CC, Wong KY, Wang H, Wu S (2012) Dynamical synapses enhance neural information processing: gracefulness, accuracy, and mobility. Neural Comput 24:1147–1185
Acknowledgements
SY was supported by the Early Research Scheme Reward from University of Wolverhampton and National High Technology Research and Development Program from China. KFW-L and SY were supported by ASUR (1014-C4-Ph1-071).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yang, S., Wong-Lin, K., Andrew, J. et al. A neuro-inspired visual tracking method based on programmable system-on-chip platform. Neural Comput & Applic 30, 2697–2708 (2018). https://doi.org/10.1007/s00521-017-2847-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-017-2847-5