Abstract
In this paper the research on optimisation of visual object tracking using a Siamese neural network for embedded vision systems is presented. It was assumed that the solution shall operate in real-time, preferably for a high resolution video stream, with the lowest possible energy consumption. To meet these requirements, techniques such as the reduction of computational precision and pruning were considered. Brevitas, a tool dedicated for optimisation and quantisation of neural networks for FPGA implementation, was used. A number of training scenarios were tested with varying levels of optimisations – from integer uniform quantisation with 16 bits to ternary and binary networks. Next, the influence of these optimisations on the tracking performance was evaluated. It was possible to reduce the size of the convolutional filters up to 10 times in relation to the original network. The obtained results indicate that using quantisation can significantly reduce the memory and computational complexity of the proposed network while still enabling precise tracking, thus allow to use it in embedded vision systems. Moreover, quantisation of weights positively affects the network training by decreasing overfitting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Held, D., Thrun, S., Savarese, S.: Learning to track at 100 FPS with deep regression networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 749–765. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_45
Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: Fully-convolutional Siamese networks for object tracking. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 850–865. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_56
Guo, Q., Feng, W., Zhou, C., Huang, R., Wan, L., Wang, S.: Learning dynamic Siamese network for visual object tracking. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, pp. 1781–1789 (2017)
Kristan, M., et al.: The seventh visual object tracking VOT2019 challenge results. In: The IEEE International Conference on Computer Vision (ICCV) Workshops (2019)
Valmadre, J., Bertinetto, L., Henriques, J., Vedaldi, A., Torr, P.H.S.: End-to-end representation learning for correlation filter based tracking. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)
Chen, Z., Zhong, B., Li, G., Zhang, S., Ji, R.: Siamese box adaptive network for visual tracking. Accepted to CVPR 2020
Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., Bengio, Y.: Binarized neural networks: training deep neural networks with weights and activations constrained to +1 or \(-1\) (2016)
Deng, L., Jiao, P., Pei, J., Wua, Z., Li, G.: GXNOR-Net: training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization framework. Neural Netw. 100, 48–58 (2018)
Wu, J., Leng, C., Wang, Y., Hu, Q., Cheng, J.: Quantized convolutional neural networks for mobile devices. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Han, S., Mao, H., Dally, W.J.: GXNOR-Net: deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. In: ICLR (2016)
Blalock, D., Ortiz, J.J.G., Frankle, J., Guttag, J.: What is the state of neural network pruning? In: Proceedings of Machine Learning and Systems 2020 (MLSys) (2020)
Ma, N., Zhang, X., Zheng, H.-T., Sun, J.: ShuffleNet V2: practical guidelines for efficient CNN architecture design. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 122–138. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_8
Cao, Y., Ji, H., Zhang, W., Shirani, S.: Extremely tiny Siamese networks with multi-level fusions for visual object tracking. In: 22th International Conference on Information Fusion (FUSION) (2019)
Zhang, B., Li, X., Han, J., Zeng, Z.: MiniTracker: a lightweight CNN-based system for visual object tracking on embedded device. In: 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP) (2018)
Baldi, P., Chauvin, Y.: Neural networks for fingerprint recognition. Neural Comput. 5(3), 402–418 (1993)
Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., Shah, R.: Signature verification using a Siamese time delay neural network. In: Advances in Neural Information Processing Systems 6, [7th NIPS Conference, Denver, Colorado, USA, 1993], pp. 737–744 (1993)
Pflugfelder, R.P.: Siamese learning visual tracking: a survey. CoRR, abs/1707.00569 (2017)
FINN. https://xilinx.github.io/finn/. Accessed 27 May 2020
Yann, L., Denker, J.S., Solla, S.A.: Optimal brain damage. In: Advances in Neural Information Processing Systems, pp. 598–605 (1990)
Han, S., Pool, J., Tran, J., Dally, W.J.: Learning both weights and connections for efficient neural networks. In: Proceedings of the 28th International Conference on Neural Information Processing Systems, NIPS 2015, pp. 1135–1143 (2015)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: The ACM, vol. 60, pp. 84–90 (2017)
Acknowledgements
The work presented in this paper was supported by the Faculty of Electrical Engineering, Automatics, Computer Science and Biomedical Engineering Dean grant – project number 16.16.120.773 (first author) and the National Science Centre project no. 2016/23/D/ST6/01389 entitled “The development of computing resources organization in latest generation of heterogeneous reconfigurable devices enabling real-time processing of UHD/4K video stream”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Przewlocka, D., Wasala, M., Szolc, H., Blachut, K., Kryjak, T. (2020). Optimisation of a Siamese Neural Network for Real-Time Energy Efficient Object Tracking. In: Chmielewski, L.J., Kozera, R., Orłowski, A. (eds) Computer Vision and Graphics. ICCVG 2020. Lecture Notes in Computer Science(), vol 12334. Springer, Cham. https://doi.org/10.1007/978-3-030-59006-2_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-59006-2_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59005-5
Online ISBN: 978-3-030-59006-2
eBook Packages: Computer ScienceComputer Science (R0)