Skip to main content
SpringerLink
Log in

The Role of Object Detection, Tracking and Augmentation in Guiding the Remote Assistance Systems

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Establishing a remote assistance system which will guide the user in repair or maintenance works of any manufacturer machine is quite challenging. The present paper proposes a solution where the user will train a deep learning models to detect different devices and devices parts. The paper discusses different solutions used in training the models and compares these models in detecting the objects of devices with respect to accuracy and time. The tracking and real-time detection of the objects help the user to perform any remote task with the help of an expert. The solution is ported in all types of mobile devices where users can see the devices and devices parts with augmented information on top of that. This solution will help the user to take remote tasks with the help of expert where expert cannot reach the field for repair or maintenance of devices. The entire solution is deployed for dialysis machine use case for performing multiple repair or maintenance activities of the dialysis machine. The accuracy of these models and model performance in real-time also play a key role as part of our remote assistance tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Lin T-Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR); 2017. pp. 2117–25.

  2. Redmon J, Divvala S, Girshick R, Farhadi A. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR); 2016. pp. 779–88.

  3. Ren S, He K, Girshick R, Sun J. Faster r-cnn: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst. 2015;2015:91–9.

    Google Scholar 

  4. Zhang S, Wen L, Bian X, Lei Z, Li SZ. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR); 2018. pp. 4203–12.

  5. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H. Mobilenets: efficient convolutional neural networks for mobile vision applications. 2017. arXiv: http://arxiv.org/abs/1704.04861.

  6. Kirillov A, Girshick R, He K, Dollár P. Panoptic feature pyramid networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2019. pp. 6399–408.

  7. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. pp. 770–8.

  8. Lapointe J-F, Molyneaux H, Allili MS. A literature review of AR-based remote guidance tasks with user studies. In: International conference on human-computer interaction. Cham: Springer; 2020. pp. 111–20.

  9. Huang W, Alem L, Tecchia F. HandsIn3D: supporting remote guidance with immersive virtual environments. In: IFIP conference on human-computer interaction. Berlin, Heidelberg: Springer; 2013. pp. 70–77.

  10. Chen AI, Balter ML, Maguire TJ, Yarmush ML. Deep learning robotic guidance for autonomous vascular access. Nature Mach Intell. 2020;2:104–15.

    Article  Google Scholar 

  11. Zillner J, Mendez E, Wagner D. Augmented reality remote collaboration with dense reconstruction. In: 2018 IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct), IEEE; 2018. pp. 38–9.

  12. Chen P-HC, Gadepalli K, MacDonald R, Liu Y, Kadowaki S, Nagpal K, Kohlberger T, et al. An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis. Nature Med. 2019;25(9):1453–7.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Wipro Technology Limited, Wipro CTO Office, Wipro, Bangalore, 560100, India

    Vivek Kumar Varma Nadimpalli, Gopichand Agnihotram & Pandurang Naik

Authors
  1. Vivek Kumar Varma Nadimpalli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gopichand Agnihotram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pandurang Naik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vivek Kumar Varma Nadimpalli.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Data Science and Communication” guest-edited by Kamesh Namudri, Naveen Chilamkurti, Sushma S J and S. Padmashree.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nadimpalli, V.K.V., Agnihotram, G. & Naik, P. The Role of Object Detection, Tracking and Augmentation in Guiding the Remote Assistance Systems. SN COMPUT. SCI. 2, 305 (2021). https://doi.org/10.1007/s42979-021-00679-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-021-00679-5

Keywords

Search

Navigation