Skip to main content
SpringerLink
Log in

Efficient Machine Learning of Mobile Robotic Systems Based on Convolutional Neural Networks

  • Chapter
  • First Online:
Artificial Intelligence for Robotics and Autonomous Systems Applications

Part of the book series: Studies in Computational Intelligence ((SCI,volume 1093))

  • 893 Accesses

Abstract

During the last decade, Convolutional Neural Networks (CNNs) have been recognized as one of the most promising machine learning methods that are being utilized for deep learning of autonomous robotic systems. Faced with everlasting uncertainties while working in unstructured and dynamical real-world environments, robotic systems need to be able to recognize different environmental scenarios and make adequate decisions based on machine learning of the current environment’s state representation. One of the main challenges in the development of machine learning models based on CNNs is in the selection of appropriate model structure and parameters that can achieve adequate accuracy of environment representation. In order to address this challenge, the book chapter provides a comprehensive analysis of the accuracy and efficiency of CNN models for autonomous robotic applications. Particularly, different CNN models (i.e., structures and parameters) are trained, validated, and tested on real-world image data gathered by a mobile robot’s stereo vision system. The best performing CNN models based on two criteria—the number of frames per second and mean intersection over union are implemented on the real-world wheeled mobile robot RAICO (Robot with Artificial Intelligence based COgnition), which is developed in the Laboratory for robotics and artificial intelligence (ROBOTICS&AI) and tested for obstacle avoidance tasks. The achieved experimental results show that the proposed machine learning strategy based on CNNs provides high accuracy of mobile robot’s current environment state estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Krizhevsky, A., Sutskever, I., & Hinton, G.E. (2012) ImageNet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097–1105).

    Google Scholar 

  2. Guo, Y., Liu, Y., Oerlemans, A., Lao, S., Wu, S., & Lew, M. S. (2016). Deep learning for visual understanding: A review. Neurocomputing, 187, 27–48.

    Article  Google Scholar 

  3. LeCun, Y., & Bengio, Y. (1995) Convolutional networks for images, speech, and time series. Handbook brain theory neural networks (Vol. 3361, no. 10).

    Google Scholar 

  4. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778).

    Google Scholar 

  5. Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. In 3rd International Conference on Learning Representations (pp. 1–14).

    Google Scholar 

  6. Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (pp. 1251–1258).

    Google Scholar 

  7. Nguyen, H., & Cheah, C.C. (2022). Analytic deep neural network-based robot control. IEEE/ASME Transactions Mechatronics (pp. 1–9).

    Google Scholar 

  8. Jokić, A., Petrović, M., & Miljković, Z. (2022). Mobile robot decision-making system based on deep machine learning. In 9th International Conference on Electrical, Electronics and Computer Engineering (IcETRAN 2022) (pp. 653–656).

    Google Scholar 

  9. Miljković, Z., Mitić, M., Lazarević, M., & Babić, B. (2013). Neural network reinforcement learning for visual control of robot manipulators. Expert Systems with Applications, 40(5), 1721–1736.

    Article  Google Scholar 

  10. Miljković, Z., Vuković, N., Mitić, M., & Babić, B. (2013). New hybrid vision-based control approach for automated guided vehicles. International Journal of Advanced Manufacturing Technology, 66(1–4), 231–249.

    Article  Google Scholar 

  11. Petrović, M., Ciężkowski, M., Romaniuk, S., Wolniakowski, A., & Miljković, Z. (2021). A novel hybrid NN-ABPE-based calibration method for improving accuracy of lateration positioning system. Sensors, 21(24), 8204.

    Article  Google Scholar 

  12. Mitić, M., Vuković, N., Petrović, M., & Miljković, Z. (2018). Chaotic metaheuristic algorithms for learning and reproduction of robot motion trajectories. Neural Computing and Applications, 30(4), 1065–1083.

    Article  Google Scholar 

  13. Mitić, M., & Miljković, Z. (2015). Bio-inspired approach to learning robot motion trajectories and visual control commands. Expert Systems with Applications, 42(5), 2624–2637.

    Article  Google Scholar 

  14. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., & Amodei, D. (2020). Language models are few-shot learners. In Advances in Neural Information Processing Systems (Vol. 33, pp. 1877–1901).

    Google Scholar 

  15. Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L.-M., Rothchild, D., So, D., Texier, M., & Dean, J. (2021). Carbon emissions and large neural network training (pp. 1–22). arXiv:2104.10350.

    Google Scholar 

  16. Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. arXiv:1906.02243.

    Google Scholar 

  17. Paszke, A., Chaurasia, A., Kim, S., & Culurciello, E. (2016). ENet: a deep neural network architecture for real-time semantic segmentation. arXiv:1606.02147.

    Google Scholar 

  18. Romera, E., Alvarez, J. M., Bergasa, L. M., & Arroyo, R. (2017). Erfnet: Efficient residual factorized convnet for real-time semantic segmentation. IEEE Transactions on Intelligent Transportation Systems, 19(1), 263–272.

    Article  Google Scholar 

  19. Milioto, A., Lottes, P., & Stachniss, C. (2018) Real-time semantic segmentation of crop and weed for precision agriculture robots leveraging background knowledge in CNNs. In 2018 IEEE International Conference on Robotics and Automation (pp. 2229–2235).

    Google Scholar 

  20. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., & Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size (pp. 1–13). arXiv:1602.07360.

    Google Scholar 

  21. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017). MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861.

    Google Scholar 

  22. Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018). Shufflenet: An extremely efficient convolutional neural network for mobile devices. Proceedings of the IEEE Conference on Computer Vision Pattern Recognition (pp. 6848–6856).

    Google Scholar 

  23. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. C. (2018). MobileNetV2: Inverted residuals and linear bottlenecks. IEEE Conference on Computer Vision Pattern Recognition (pp. 4510–4520).

    Google Scholar 

  24. Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018). Shufflenet v2: Practical guidelines for efficient CNN architecture design. Proceedings of European Conference on Computer Vision (pp. 116–131).

    Google Scholar 

  25. Ren, S., He, K., Girshick, R., & Sun, J. (2016). Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149.

    Article  Google Scholar 

  26. Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. IEEE Conference on Computer Vision Pattern Recognition (pp. 779–788).

    Google Scholar 

  27. Redmon, J., & Farhadi, A. (2017) YOLO9000: Better, faster, stronger. IEEE Conference on Computer Vision Pattern Recognition (pp. 7263–7271).

    Google Scholar 

  28. Redmon, J., & Farhadi, A. (2018) YOLOv3: An incremental improvement. arXiv:1804.02767.

    Google Scholar 

  29. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C. (2016). SSD: Single shot multibox detector. In European Conference Computer Vision (pp. 21–37).

    Google Scholar 

  30. Chen, X., & Guhl, J. (2018). Industrial robot control with object recognition based on deep learning. Procedia CIRP, 76, 149–154.

    Article  Google Scholar 

  31. Gao, Q., Liu, J., & Ju, Z. (2020). Robust real-time hand detection and localization for space human–robot interaction based on deep learning. Neurocomputing, 390, 198–206.

    Article  Google Scholar 

  32. Xiao, L., Wang, J., Qiu, X., Rong, Z., & Zou, X. (2019). Dynamic-SLAM: Semantic monocular visual localization and mapping based on deep learning in dynamic environment. Robotics and Autonomous Systems, 117, 1–16.

    Article  Google Scholar 

  33. Hwang, C.-L., Wang, D.-S., Weng, F.-C., & Lai, S.-L. (2020). Interactions between specific human and omnidirectional mobile robot using deep learning approach: SSD-FN-KCF. IEEE Access, 8, 41186–41200.

    Article  Google Scholar 

  34. Zhao, K., Wang, Y., Zuo, Y., & Zhang, C. (2022). Palletizing robot positioning bolt detection based on improved YOLO-V3. Journal of Intelligent and Robotic Systems, 104(3), 1–12.

    Article  Google Scholar 

  35. Liu, C., Li, X., Li, Q., Xue, Y., Liu, H., & Gao, Y. (2021). Robot recognizing humans intention and interacting with humans based on a multi-task model combining ST-GCN-LSTM model and YOLO model. Neurocomputing, 430, 174–184.

    Article  Google Scholar 

  36. Jokić, A., Petrović, M., & Miljković, Z. (2022). Semantic segmentation based stereo visual servoing of nonholonomic mobile robot in intelligent manufacturing environment. Expert Systems with Applications, 190, 116203.

    Article  Google Scholar 

  37. Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Martinez-Gonzalez, P., & Garcia-Rodriguez, J. (2018). A survey on deep learning techniques for image and video semantic segmentation. Applied Soft Computing, 70, 41–65.

    Google Scholar 

  38. Hu, P., Perazzi, F., Heilbron, F. C., Wang, O., Lin, Z., Saenko, K., & Sclaroff, S. (2020). Real-time semantic segmentation with fast attention. IEEE Robotics and Automation Letters, 6(1), 263–270.

    Article  Google Scholar 

  39. Falaschetti, L., Manoni, L., & Turchetti, C. (2022). A low-rank CNN architecture for real-time semantic segmentation in visual SLAM applications. IEEE Open Journal of Circuits and Systems.

    Google Scholar 

  40. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 234–241).

    Google Scholar 

  41. Alonso, I., Riazuelo, L., & Murillo, A. C. (2020). Mininet: An efficient semantic segmentation convnet for real-time robotic applications. IEEE Transactions on Robotics, 36(4), 1340–1347.

    Article  Google Scholar 

  42. Seichter, D., Köhler, M., Lewandowski, B., Wengefeld, T., & Gross, H.-M. (2021) Efficient rgb-d semantic segmentation for indoor scene analysis. 2021 IEEE International Conference on Robotics and Automation (pp. 13525–13531).

    Google Scholar 

  43. Bokovoy, A., Muravyev, K., & Yakovlev, K. (2019). Real-time vision-based depth reconstruction with NVidia Jetson. In 2019 European Conference on Mobile Robots (pp. 1–6).

    Google Scholar 

  44. Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016). The cityscapes dataset for semantic urban scene understanding. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (pp. 3213–3223).

    Google Scholar 

  45. Dustin, F. (2022) Hello AI world NVidia Jetson. https://github.com/dusty-nv/jetson-inference.

  46. Petrović, M., Miljković, Z., & Jokić, A. (2019). A novel methodology for optimal single mobile robot scheduling using whale optimization algorithm. Applied Soft Computing, 81, 105520.

    Article  Google Scholar 

  47. Petrović, M., Jokić, A., Miljković, Z., & Kulesza, Z. (2022). Multi-objective scheduling of single mobile robot based on grey wolf optimization algorithm. SSRN.

    Google Scholar 

  48. Petrović, M., Miljković, Z., Babić, B., Vuković, N., & Čović, N. (2012). Towards a conceptual design of intelligent material transport using artificial intelligence. Strojarstvo, 54(3), 205–219.

    Google Scholar 

  49. Song, S., Lichtenberg, S.P., & Xiao, J. (2015). SUN RGB-D: A RGB-D scene understanding benchmark suite. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition (pp. 567–576).

    Google Scholar 

  50. Corke, P. (2017). Robotics, vision and control: Fundamental algorithms in MATLAB®. Springer.

    Book  MATH  Google Scholar 

Download references

Acknowledgements

This work has been financially supported by the Ministry of Education, Science and Technological Development of the Serbian Government, through the project “Integrated research in macro, micro, and nano mechanical engineering–Deep learning of intelligent manufacturing systems in production engineering”, under the contract number 451-03-47/2023-01/200105, and by the Science Fund of the Republic of Serbia, Grant No. 6523109, AI-MISSION4.0, 2020-2022.

Author information

Authors and Affiliations

  1. Faculty of Mechanical Engineering, University of Belgrade, Belgrade, Serbia

    Milica Petrović, Zoran Miljković & Aleksandar Jokić

Authors
  1. Milica Petrović
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zoran Miljković
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aleksandar Jokić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Milica Petrović .

Editor information

Editors and Affiliations

  1. College of Computer and Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia

    Ahmad Taher Azar

  2. College of Computer and Information Sciences, Prince Sultan University, Riyadh, Saudi Arabia

    Anis Koubaa

Appendix A

Appendix A

Abbreviation List

RAICO:

Robot with Artificial Intelligence based COgnition

AI:

Artificial Intelligence

ML:

Machine Learning

ANN:

Artificial Neural Networks

DL:

Deep Learning

CNN:

Convolutional Neural Network

FLOPs:

FLoating Point Operations

FPS:

Frames Per Second

VGG:

Visual Geometry Group

R-CNN:

Region–Convolutional Neural Network

SSD:

Single Shot Detector

YOLO:

You Only Look Ones

SLAM:

Simultaneous Localization And Mapping

LSTM:

Long-Short Term Memory

RGBD:

Red Green Blue Depth

BN:

Batch Normalization

ReLU:

Rectified Linear Unit

BB:

Basic Block

BRB:

Basic Reduction Block

DB:

1D Block

DRB:

1D Reduction Block

SB:

Separation Block

SRB:

Separation Reduction Block

RN:

ResNet

mIoU:

Mean Intersection over Union

ONNX:

Open Neural Network eXchange.

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Petrović, M., Miljković, Z., Jokić, A. (2023). Efficient Machine Learning of Mobile Robotic Systems Based on Convolutional Neural Networks. In: Azar, A.T., Koubaa, A. (eds) Artificial Intelligence for Robotics and Autonomous Systems Applications. Studies in Computational Intelligence, vol 1093. Springer, Cham. https://doi.org/10.1007/978-3-031-28715-2_1

Download citation

Publish with us

Policies and ethics

Search

Navigation