Skip to main content
SpringerLink
Log in

Fast and Automatic Object Registration for Human-Robot Collaboration in Industrial Manufacturing

  • Conference paper
  • First Online:
Database and Expert Systems Applications - DEXA 2022 Workshops (DEXA 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1633))

Included in the following conference series:

Abstract

We present an end-to-end framework for fast retraining of object detection models in human-robot-collaboration. Our Faster R-CNN based setup covers the whole workflow of automatic image generation and labeling, model retraining on-site as well as inference on a FPGA edge device. The intervention of a human operator reduces to providing the new object together with its label and starting the training process. Moreover, we present a new loss, the intraspread-objectosphere loss, to tackle the problem of open world recognition. Though it fails to completely solve the problem, it significantly reduces the number of false positive detections of unknown objects.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Code adapted from https://github.com/yhenon/keras-frcnn/ (last pulled 02/03/2020).

  2. 2.

    https://github.com/fchollet/deep-learning-models/releases (pretrained on Imagenet).

  3. 3.

    https://www.xilinx.com/support/documentation/user_guides/ug1327-dnndk-user-guide.pdf.

  4. 4.

    https://docs.opencv.org/4.x/d9/d61/tutorial_py_morphological_ops.html.

  5. 5.

    https://scikit-image.org/docs/dev/api/skimage.filters.html.

References

  1. Bendale, A., Boult, T.: Towards open world recognition. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition, pp. 1893–1902 (2015)

    Google Scholar 

  2. Chen, J., Ran, X.: Deep learning with edge computing: a review. Proc. IEEE 107(8), 1655–1674 (2019)

    Article  Google Scholar 

  3. Dhamija, A.R., Günther, M., Boult, T.: Reducing network agnostophobia. Adv. Neural Inf. Process. Syst. 31, 9157–9168 (2018)

    Google Scholar 

  4. Ge, Z., Liu, S., Wang, F., Li, Z., Sun, J.: YOLOX: exceeding YOLO series in 2021. arXiv preprint arXiv:2107.08430 (2021)

  5. Hassen, M., Chan, P.K.: Learning a neural-network-based representation for open set recognition. In: Proceedings of SIAM International Conference on Data Mining, pp. 154–162. SIAM (2020)

    Google Scholar 

  6. Joseph, K., Khan, S., Khan, F.S., Balasubramanian, V.N.: Towards open world object detection. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition, pp. 5830–5840 (2021)

    Google Scholar 

  7. Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  8. Matheson, E., Minto, R., Zampieri, E., Faccio, M., Rosati, G.: Human-robot collaboration in manufacturing applications: a review. Robotics 8, 100 (2019)

    Article  Google Scholar 

  9. Mehrabi, M., Ulsoy, A., Koren, Y.: Reconfigurable manufacturing systems: key to future manufacturing. J. Intell. Manuf. 11, 403–419 (2000)

    Article  Google Scholar 

  10. Murshed, M.G.S., Murphy, C., Hou, D., Khan, N., Ananthanarayanan, G., Hussain, F.: Machine learning at the network edge: a survey. ACM Comput. Surv. 54(8), 1–37 (2021)

    Article  Google Scholar 

  11. Ramasesh, V.V., Dyer, E., Raghu, M.: Anatomy of catastrophic forgetting: hidden representations and task semantics. In: International Conference on Learning Representation (2021)

    Google Scholar 

  12. Redmon, J., Farhadi, A.: YOLOV3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)

  13. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advanced Neural Information Processing Systems, vol. 28 (2015)

    Google Scholar 

  14. Ryan Fanello, S., et al.: iCuB world: friendly robots help building good vision data-sets. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition Workshops (2013)

    Google Scholar 

  15. Scheirer, W.J., de Rezende Rocha, A., Sapkota, A., Boult, T.E.: Toward open set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1757–1772 (2012)

    Article  Google Scholar 

  16. Zhao, X., Li, W., Zhang, Y., Gulliver, T.A., Chang, S., Feng, Z.: A faster RCNN-based pedestrian detection system. In: VTC2016-Fall, pp. 1–5 (2016)

    Google Scholar 

  17. Zhao, X., Liu, X., Shen, Y., Ma, Y., Qiao, Y., Wang, D.: Revisiting open world object detection. arXiv preprint arXiv:2201.00471 (2022)

Download references

Acknowledgements

The research reported in this paper has been funded by BMK, BMDW, and the State of Upper Austria in the frame of SCCH, part of the COMET Programme managed by FFG.

Author information

Authors and Affiliations

  1. Software Competence Center Hagenberg GmbH, Softwarepark 32a, 4232, Hagenberg, Austria

    Manuela Geiß, Georgios Chasparis & Michael Zwick

  2. KEBA Group AG, Reindlstraße 51, 4040, Linz, Austria

    Martin Baresch

  3. Danube Dynamics Embedded Solutions GmbH, Lastenstraße 38/12.OG, 4020, Linz, Austria

    Edwin Schweiger & Nico Teringl

Authors
  1. Manuela Geiß
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Baresch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Georgios Chasparis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Edwin Schweiger
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nico Teringl
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Michael Zwick
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manuela Geiß .

Editor information

Editors and Affiliations

  1. Johannes Kepler University of Linz, Linz, Oberösterreich, Austria

    Gabriele Kotsis

  2. Technical University of Vienna, Vienna, Austria

    A Min Tjoa

  3. Johannes Kepler University of Linz, Linz, Austria

    Ismail Khalil

  4. Software Competence Center Hagenberg, Hagenberg, Austria

    Bernhard Moser

  5. (WU) Vienna University of Economics and Business, Vienna, Austria

    Alfred Taudes

  6. Johannes Kepler University of Linz, Linz, Austria

    Atif Mashkoor

  7. Johannes Kepler University of Linz, Linz, Austria

    Johannes Sametinger

  8. Software Competence Center Hagenberg, Hagenberg, Austria

    Jorge Martinez-Gil

  9. Software Competence Center Hagenberg, Hagenberg, Austria

    Florian Sobieczky

  10. Software Competence Center Hagenberg, Hagenberg, Austria

    Lukas Fischer

  11. Software Competence Center Hagenberg, Hagenberg, Austria

    Rudolf Ramler

  12. Pak-Austria Fachhochschule - Institute of Applied Sciences and Technology (PAF-IAST), Haripur, Pakistan

    Maqbool Khan

  13. Software Competence Center Hagenberg, Hagenberg, Austria

    Gerald Czech

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Geiß, M., Baresch, M., Chasparis, G., Schweiger, E., Teringl, N., Zwick, M. (2022). Fast and Automatic Object Registration for Human-Robot Collaboration in Industrial Manufacturing. In: Kotsis, G., et al. Database and Expert Systems Applications - DEXA 2022 Workshops. DEXA 2022. Communications in Computer and Information Science, vol 1633. Springer, Cham. https://doi.org/10.1007/978-3-031-14343-4_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-14343-4_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-14342-7

  • Online ISBN: 978-3-031-14343-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation