Skip to main content
Log in

Object Tracking in the Video Stream by Means of a Convolutional Neural Network

  • Published:
Optoelectronics, Instrumentation and Data Processing Aims and scope

Abstract

A new algorithm of 6-coordinate tracking of a moving object on a sequence of RGB-images that is based on the convolutional neural network is proposed. Training the neural network is carried out by using the synthesized data of the object with a dynamic model of motion. A Kalman filter is included into the feedback from the network output to its input to obtain a smoothed estimate of the object coordinates. Preliminary results of object tracking on synthesized images demonstrates the efficiency of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

REFERENCES

  1. M. Siam, A. Singh, C. Perez, and M. Jägersand, ‘‘4-dof tracking for robot fine manipulation tasks,’’ in 14th Conf. on Computer and Robot Vision, 2017, Edmonton, Canada, 2017, pp. 329–336. https://doi.org/10.1109/CRV.2017.41

  2. G. Gallego, J. E. A. Lund, E. Mueggler, H. Rebecq, T. Delbruck, and D. Scaramuzza, ‘‘Event-based, 6-DOF camera tracking from photometric depth maps,’’ IEEE Trans. Pattern Anal. Machine Intell. 40, 2402–2412 (2017). https://doi.org/10.1109/TPAMI.2017.2769655

    Article  Google Scholar 

  3. M. Garon and J.-F. Lalonde, ‘‘Deep 6-DOF tracking,’’ IEEE Trans. Vis. Computer Graph. 23, 2410–2418 (2017). https://doi.org/10.1109/TVCG.2017.2734599

    Article  Google Scholar 

  4. A. Doering, U. Iqbal, and J. Gall, ‘‘Joint flow: temporal flow fields for multi person tracking,’’ in 29th British Machine Vision Conf., Newcastle, 2018 http://arxiv.org/abs/1805.04596. Cited March 3, 2020.

  5. W. Zheng, F. Zhou, and Z. Wang, ‘‘External vision based robust pose estimation system for a quadrotor in outdoor environments,’’ in Proc. 3rd Int. Conf. on Pattern Recognition Applications and Methods, Angers, France, 2014, pp. 718–723. https://doi.org/10.5220/0004906007180723

  6. M. Faessler, E. Mueggler, K. Schwabe, and D. Scaramuzza, ‘‘A monocular pose estimation system based on infrared LEDs,’’ in IEEE Int. Conf. on Robotics and Automation, Hong Kong, China, 2014, pp. 907–913. https://doi.org/10.1109/ICRA.2014.6906962

  7. A. Censi, J. Strubel, Ch. Brandli, T. Delbruck, and D. Scaramuzza, ‘‘Low-latency localization by active led markers tracking using a dynamic vision sensor,’’ in IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Tokyo, Japan, 2013, pp. 891–898. https://doi.org/10.1109/IROS.2013.6696456

  8. E. Mueggler, M. Faessler, F. Fontana, and D. Scaramuzza, ‘‘Aerial-guided navigation of a ground robot among movable obstacles,’’ in Proc. 12th IEEE Intern. Symposium on Safety, Security and Rescue Robotics, Hokkaido, Japan, 2014, pp. 1–8. https://doi.org/10.1109/SSRR.2014.7017662

  9. A. Millard, J. Hilder, J. Timmis, and A. Winfield, ‘‘A low-cost real-time tracking infrastructure for ground-based robot swarms,’’ in Swarm Intelligence, Ed. by M. Dorigo, M. Birattari, S. Garnier, H. Hamann, M. Montes de Oca, C. Solnon, and T. Stützle (Springer, Cham, 2014), vol. 8667, pp. 278–279.

    Google Scholar 

  10. D. Held, S. Thrun, and S. Savarese, ‘‘Learning to track at 100 FPS with deep regression networks,’’ in Computer Vision — ECCV 2016, Ed. by B. Leibe, J. Matas, N. Sebe, M. Welling (Springer, Cham, 2016). doi 10.1007/978-3-319-46448-0_45. http://arxiv.org/abs/1604.01802. Cited March 3, 2020.

  11. D. J. Tan, N. Navab, and F. Tombari, ‘‘6d object pose estimation with depth images: a seamless approach for robotic interaction and augmented reality,’’ in Computer Vision and Pattern Recognition (Cornell Univ., 2017). http://arxiv.org/abs/1709.01459. Cited March 3, 2020.

  12. K. Yu. Kotov, A. S. Mal’tsev, A. A. Nesterov, and A. P. Yan, ‘‘Algorithms and architecture of the multirotor aircraft trajectory motion control system,’’ Optoelectron., Instrum. Data Process. 56 (2011). https://doi.org/10.3103/S8756699020030085

  13. Yu. N. Zolotukhin, K. Yu. Kotov, A. S. Maltsev, A. A. Nesterov, M. N. Filippov, and A. P. Yan, ‘‘Correction of transportation lag in the mobile robot control system,’’ Optoelectron., Instrum. Data Process. 47, 141–150 (2011). https://doi.org/10.3103/S8756699011020051

    Article  Google Scholar 

Download references

Funding

This work was partly supported by the Russian Foundation for Basic Research (project no. 18-58-76003 ERA_a) and the Ministry of Science and Higher Education of the Russian Federation (project no. AAAA-A17-117060610006-6).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to K. Yu. Kotov.

Additional information

Translated by E. Smirnova

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zolotukhin, Y.N., Kotov, K.Y., Nesterov, A.A. et al. Object Tracking in the Video Stream by Means of a Convolutional Neural Network. Optoelectron.Instrument.Proc. 56, 642–648 (2020). https://doi.org/10.3103/S8756699020060163

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.3103/S8756699020060163

Keywords:

Navigation