Skip to main content
SpringerLink
Log in

Improved Deep Neural Network Object Tracking System for Applications in Home Robotics

  • Chapter
  • First Online:
Computational Intelligence for Pattern Recognition

Part of the book series: Studies in Computational Intelligence ((SCI,volume 777))

Abstract

Robotic navigation in GPS-denied environments requires case specific approaches for controlling a mobile robot to any desired destinations. In general, a nominal path is created in an environment described by a set of distinct objects, in other words such obstacles and landmarks. Intelligent voice assistants or digital assistance devices are increasing their importance in today’s smart home. Especially, by the help of fast-growing Internet of Things (IoT) applications. These devices are amassing an ever-growing list of features such as controlling states of connected smart devices, recording tasks, and responding to queries. Assistive robots are the perfect complement to smart voice assistants for providing physical manipulation. A request made by a person can be assigned to the assistive robot by the voice assistant. In this chapter, a new approach for autonomous navigation is presented using pattern recognition and machine learning techniques such as Convolutional Neural Networks to identify markers or objects from images and videos. Computational intelligence techniques are implemented along with Robot Operating System and object positioning to navigate towards these objects and markers by using RGB-depth camera . Multiple potential matching objects detected by the robot with deep neural network object detectors will be displayed on a screen installed on the assistive robot to improve and evaluate Human-Robot Interaction (HRI).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. S.B. Kotsiantis, I. Zaharakis, P. Pintelas, Supervised machine learning: A review of classification techniques, 2007, 3–24

    Google Scholar 

  2. A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097–1105

    Google Scholar 

  3. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–9

    Google Scholar 

  4. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2818–2826

    Google Scholar 

  5. A. Karpathy, What I learned from competing against a convnet on imagenet, 2014, http://karpathy.github.io/2014/09/02/whati-learned-from-competing-against-a-convnet-on-imagenet

  6. J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: Unified, real-time object detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 779–788

    Google Scholar 

  7. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, A.C. Berg, Ssd: Single shot multibox detector, in European Conference on Computer Vision (Springer, 2016), pp. 21–37

    Chapter  Google Scholar 

  8. B.A. Erol, S. Vaishnav, J.D. Labrado, P. Benavidez, M. Jamshidi, Cloud-based control and vslam through cooperative mapping and localization, in World Automation Congress (WAC) (IEEE, 2016), pp. 1–6

    Google Scholar 

  9. L. Fan, Z. Wang, B. Cail, C. Tao, Z. Zhang, Y. Wang, S. Li, F. Huang, S. Fu, F. Zhang, A survey on multiple object tracking algorithm, in 2016 IEEE International Conference on Information and Automation (ICIA) (IEEE, 2016), pp. 1855–1862

    Google Scholar 

  10. J. Redmon, A. Farhadi, Yolo9000: better, faster, stronger, 2016, arXiv:1612.08242

  11. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S.E. Reed, C. Fu, A.C. Berg, SSD: single shot multibox detector. CoRR, abs/1512.02325 (2015). http://arxiv.org/abs/1512.02325

  12. S. Ren, K. He, R.B. Girshick, J. Sun, Faster R-CNN: towards real-time object detection with region proposal networks. CoRR abs/1506.01497 (2015). http://arxiv.org/abs/1506.01497

  13. J. Dai, Y. Li, K. He, J. Sun, R-FCN: object detection via region-based fully convolutional networks CoRR. abs/1605.06409 (2016). http://arxiv.org/abs/1605.06409

  14. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, Y. LeCun, Overfeat: Integrated recognition, localization and detection using convolutional networks. CoRR abs/1312.6229 (2013). http://arxiv.org/abs/1312.6229

  15. J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama, K. Murphy, Speed/accuracy trade-offs for modern convolutional object detectors. CoRR abs/1611.10012 (2016). http://arxiv.org/abs/1611.10012

  16. C. Tomasi, T. Kanade, Detection and tracking of point features. Int. J. Comput. Vision (Tech. Rep.) (1991)

    Google Scholar 

  17. S. Oron, A. Bar-Hille, S. Avidan, Extended lucas-kanade tracking, in European Conference on Computer Vision (Springer, Cham, 2014), pp. 142–156

    Google Scholar 

  18. H. Hu, B. Ma, Y. Wu, W. Ma, K. Xie, Kernel regression based online boosting tracking. J. Inf. Sci. Eng. 31(1), 267–282 (2015)

    MathSciNet  Google Scholar 

  19. K. Zhang, L. Zhang, Q. Liu, D. Zhang, M.H. Yang, Fast visual tracking via dense spatio-temporal context learning, in European Conference on Computer Vision (Springer, Cham, 2014), pp. 127–141

    Google Scholar 

  20. S. He, Q. Yang, R.W. Lau, J. Wang, M.H. Yang, Visual tracking via locality sensitive histograms, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 2427–2434

    Google Scholar 

  21. Z. Kalal, K. Mikolajczyk, J. Matas, Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012). https://doi.org/10.1109/TPAMI.2011.239

    Article  Google Scholar 

  22. G. Nebehay, R. Pflugfelder, Clustering of static-adaptive correspondences for deformable object tracking (Comput. Vision Pattern Recognit., IEEE, 2015)

    Book  Google Scholar 

  23. J.F. Henriques, R. Caseiro, P. Martins, J. Batista, High-speed tracking with kernelized correlation filters. CoRR abs/1404.7584, 2014. http://arxiv.org/abs/1404.7584

  24. Y. Wu, J. Lim, M.-H. Yang, Online object tracking: A benchmark, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013

    Google Scholar 

  25. Open source computer vision library, https://github.com/itseez/opencv

  26. A. Singh, M. Jagersand, Modular tracking framework: a unified approach to registration based tracking. CoRR. abs/1602.09130 (2016). http://arxiv.org/abs/1602.09130

  27. P. Benavidez, M. Jamshidi, Mobile robot navigation and target tracking system,” in 2011 6th International Conference on System of Systems Engineering (SoSE) (IEEE, 2011), pp. 299–304

    Google Scholar 

  28. J.D. Labrado, B.A. Erol, J. Ortiz, P. Benavidez, M. Jamshidi, B. Champion, Proposed testbed for the modeling and control of a system of autonomous vehicles, in 2016 11th IEEE System of Systems Engineering Conference (SoSE), 2016, pp. 1–6

    Google Scholar 

  29. P. Benavidez, M. Muppidi, P. Rad, J.J. Prevost, M. Jamshidi, L. Brown, Cloud-based realtime robotic visual slam, in 2015 9th Annual IEEE International Systems Conference (SysCon) (IEEE, 2015), pp. 773–777

    Google Scholar 

  30. P. Benavidez, Low-cost Home Multi-robot Rehabilitation System for the Disabled Population (Google, 2015)

    Google Scholar 

  31. P. Benavidez, M. Kumar, S. Agaian, M. Jamshidi, Design of a home multi-robot system for the elderly and disabled, in 2015 10th System of Systems Engineering Conference (SoSE), 2015, pp. 392–397

    Google Scholar 

  32. P. Benavidez, M. Kumar, B. Erol, M. Jamshidi, S. Agaian, Software interface design for home-based assistive multi-robot system, in 2015 10th System of Systems Engineering Conference (SoSE) (IEEE, 2015), pp. 404–409

    Google Scholar 

  33. S.A. Miratabzadeh, N. Gallardo, N. Gamez, K. Haradi, A. R. Puthussery, P. Rad, M. Jamshidi, Cloud robotics: A software architecture: For heterogeneous large-scale autonomous robots, in World Automation Congress (WAC), (IEEE, 2016), pp. 1–6

    Google Scholar 

  34. M. Roopaei, P. Rad, M. Jamshidi, Deep learning control for complex and large scale cloud systems, in Intelligent Automation & Soft Computing, 2017, pp. 1–3

    Article  Google Scholar 

  35. A.R. Puthussery, K. Haradi, M. Jamshidi, A deep vision landmark framework for robot navigation, in 2017 12th IEEE System of Systems Engineering Conference (SoSE), 2017, pp. 1–6

    Google Scholar 

  36. S. Lloyd, Least squares quantization in pcm. IEEE Trans. Inf. Theory 28(2), 129–137 (1982)

    Article  MathSciNet  Google Scholar 

  37. J.C. Dunn, A fuzzy relative of the isodata process and its use in detecting compact well-separated clusters. J. Cybern. 3(3), 32–57 (1973)

    Article  MathSciNet  Google Scholar 

  38. M. Ester, H.-P. Kriegel, J. Sander, X. Xu, A density-based algorithm for discovering clusters a density-based algorithm for discovering clusters in large spatial databases with noise, in Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (AAAI Press, 1996), pp. 226–231

    Google Scholar 

  39. X. Jin, J. Han, K-Means Clustering (Springer, US, 2010), pp. 563–564

    Google Scholar 

  40. A. Stetco, X.-J. Zeng, J. Keane, Fuzzy c-means ++: Fuzzy c-means with effective seeding initialization. Expert Syst. Appl. 42(21), 7541–7548 (2015)

    Article  Google Scholar 

  41. J.F. Kolen, T. Hutcheson, Reducing the time complexity of the fuzzy c-means algorithm. IEEE Trans. Fuzzy Syst. 10(2), 263–267 (2002)

    Article  Google Scholar 

  42. D. Arthur, S. Vassilvitskii, K-means ++: The advantages of careful seeding, in Proceedings of the Eighteenth Annual ACM-SIAM on Discrete Algorithms, SODA, 2007, pp. 1027–1035

    Google Scholar 

  43. T. Lin, M. Maire, S.J. Belongie et al., Microsoft COCO: common objects in context. CoRR abs/1405.0312, 2014. http://arxiv.org/abs/1405.0312

  44. M. Everingham, L. Van Gool, C.K.I. Williams, J. Winn, A. Zisserman, The pascal visual object classes (voc) challenge. Int. J. Comput. Vision 88(2), 303–338 (2010)

    Article  Google Scholar 

  45. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, ImageNet: A Large-Scale Hierarchical Image Database, in CVPR09, 2009

    Google Scholar 

  46. J. Lwowski, L. Sun, R.M. Saavedra, R. Sharma, D. Pack, A reactive bearing angle only obstacle avoidance technique for unmanned ground vehicles. J. Autom. Control Res. 1, 31–37 (2014). http://dx.doi.org/10.11159/jacr.2014.004

Download references

Acknowledgements

The authors would like to acknowledge the support from Air Force Research Laboratory and OSD for sponsoring this research under agreement number FA8750-15-2-0116. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of Air Force Research Laboratory, OSD, or the U.S. Government. The work partially supported by the Open Cloud Institute at The University of Texas at San Antonio.

Author information

Authors and Affiliations

  1. Autonomous Control Engineering (ACE) Laboratories, Department of Electrical and Computer Engineering, The University of Texas at San Antonio, One UTSA Circle, San Antonio, TX, 78249, USA

    Berat A. Erol, Abhijit Majumdar, Jonathan Lwowski, Patrick Benavidez & Mo Jamshidi

  2. Open Cloud Institute, The University of Texas at San Antonio, San Antonio, TX, 78249, USA

    Berat A. Erol, Patrick Benavidez & Paul Rad

  3. Department of Information Systems & Cyber Security, The University of Texas at San Antonio, One UTSA Circle, San Antonio, TX, 78249, USA

    Paul Rad

Authors
  1. Berat A. Erol
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abhijit Majumdar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jonathan Lwowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Patrick Benavidez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paul Rad
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mo Jamshidi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Berat A. Erol or Patrick Benavidez .

Editor information

Editors and Affiliations

  1. University of Alberta, Edmonton, Alberta, Canada

    Witold Pedrycz

  2. National Taiwan University of Science and Technology, Taipei, Taiwan

    Shyi-Ming Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Erol, B.A., Majumdar, A., Lwowski, J., Benavidez, P., Rad, P., Jamshidi, M. (2018). Improved Deep Neural Network Object Tracking System for Applications in Home Robotics. In: Pedrycz, W., Chen, SM. (eds) Computational Intelligence for Pattern Recognition. Studies in Computational Intelligence, vol 777. Springer, Cham. https://doi.org/10.1007/978-3-319-89629-8_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-89629-8_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-89628-1

  • Online ISBN: 978-3-319-89629-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation