Abstract
A new algorithm of 6-coordinate tracking of a moving object on a sequence of RGB-images that is based on the convolutional neural network is proposed. Training the neural network is carried out by using the synthesized data of the object with a dynamic model of motion. A Kalman filter is included into the feedback from the network output to its input to obtain a smoothed estimate of the object coordinates. Preliminary results of object tracking on synthesized images demonstrates the efficiency of the proposed approach.
Similar content being viewed by others
REFERENCES
M. Siam, A. Singh, C. Perez, and M. Jägersand, ‘‘4-dof tracking for robot fine manipulation tasks,’’ in 14th Conf. on Computer and Robot Vision, 2017, Edmonton, Canada, 2017, pp. 329–336. https://doi.org/10.1109/CRV.2017.41
G. Gallego, J. E. A. Lund, E. Mueggler, H. Rebecq, T. Delbruck, and D. Scaramuzza, ‘‘Event-based, 6-DOF camera tracking from photometric depth maps,’’ IEEE Trans. Pattern Anal. Machine Intell. 40, 2402–2412 (2017). https://doi.org/10.1109/TPAMI.2017.2769655
M. Garon and J.-F. Lalonde, ‘‘Deep 6-DOF tracking,’’ IEEE Trans. Vis. Computer Graph. 23, 2410–2418 (2017). https://doi.org/10.1109/TVCG.2017.2734599
A. Doering, U. Iqbal, and J. Gall, ‘‘Joint flow: temporal flow fields for multi person tracking,’’ in 29th British Machine Vision Conf., Newcastle, 2018 http://arxiv.org/abs/1805.04596. Cited March 3, 2020.
W. Zheng, F. Zhou, and Z. Wang, ‘‘External vision based robust pose estimation system for a quadrotor in outdoor environments,’’ in Proc. 3rd Int. Conf. on Pattern Recognition Applications and Methods, Angers, France, 2014, pp. 718–723. https://doi.org/10.5220/0004906007180723
M. Faessler, E. Mueggler, K. Schwabe, and D. Scaramuzza, ‘‘A monocular pose estimation system based on infrared LEDs,’’ in IEEE Int. Conf. on Robotics and Automation, Hong Kong, China, 2014, pp. 907–913. https://doi.org/10.1109/ICRA.2014.6906962
A. Censi, J. Strubel, Ch. Brandli, T. Delbruck, and D. Scaramuzza, ‘‘Low-latency localization by active led markers tracking using a dynamic vision sensor,’’ in IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Tokyo, Japan, 2013, pp. 891–898. https://doi.org/10.1109/IROS.2013.6696456
E. Mueggler, M. Faessler, F. Fontana, and D. Scaramuzza, ‘‘Aerial-guided navigation of a ground robot among movable obstacles,’’ in Proc. 12th IEEE Intern. Symposium on Safety, Security and Rescue Robotics, Hokkaido, Japan, 2014, pp. 1–8. https://doi.org/10.1109/SSRR.2014.7017662
A. Millard, J. Hilder, J. Timmis, and A. Winfield, ‘‘A low-cost real-time tracking infrastructure for ground-based robot swarms,’’ in Swarm Intelligence, Ed. by M. Dorigo, M. Birattari, S. Garnier, H. Hamann, M. Montes de Oca, C. Solnon, and T. Stützle (Springer, Cham, 2014), vol. 8667, pp. 278–279.
D. Held, S. Thrun, and S. Savarese, ‘‘Learning to track at 100 FPS with deep regression networks,’’ in Computer Vision — ECCV 2016, Ed. by B. Leibe, J. Matas, N. Sebe, M. Welling (Springer, Cham, 2016). doi 10.1007/978-3-319-46448-0_45. http://arxiv.org/abs/1604.01802. Cited March 3, 2020.
D. J. Tan, N. Navab, and F. Tombari, ‘‘6d object pose estimation with depth images: a seamless approach for robotic interaction and augmented reality,’’ in Computer Vision and Pattern Recognition (Cornell Univ., 2017). http://arxiv.org/abs/1709.01459. Cited March 3, 2020.
K. Yu. Kotov, A. S. Mal’tsev, A. A. Nesterov, and A. P. Yan, ‘‘Algorithms and architecture of the multirotor aircraft trajectory motion control system,’’ Optoelectron., Instrum. Data Process. 56 (2011). https://doi.org/10.3103/S8756699020030085
Yu. N. Zolotukhin, K. Yu. Kotov, A. S. Maltsev, A. A. Nesterov, M. N. Filippov, and A. P. Yan, ‘‘Correction of transportation lag in the mobile robot control system,’’ Optoelectron., Instrum. Data Process. 47, 141–150 (2011). https://doi.org/10.3103/S8756699011020051
Funding
This work was partly supported by the Russian Foundation for Basic Research (project no. 18-58-76003 ERA_a) and the Ministry of Science and Higher Education of the Russian Federation (project no. AAAA-A17-117060610006-6).
Author information
Authors and Affiliations
Corresponding author
Additional information
Translated by E. Smirnova
About this article
Cite this article
Zolotukhin, Y.N., Kotov, K.Y., Nesterov, A.A. et al. Object Tracking in the Video Stream by Means of a Convolutional Neural Network. Optoelectron.Instrument.Proc. 56, 642–648 (2020). https://doi.org/10.3103/S8756699020060163
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3103/S8756699020060163