Recognition of Assembly Parts by Convolutional Neural Networks

  • Kamil ŽidekEmail author
  • Alexander Hosovsky
  • Jan Piteľ
  • Slavomír Bednár
Conference paper
Part of the Lecture Notes in Mechanical Engineering book series (LNME)


The paper describes the experiments with the use of deep neural networks (CNN) for robust identification of assembly parts (screws, nuts) and assembly features (holes), to speed up any assembly process with augmented reality application. The simple image processing tasks with static camera and recognized parts can be handled by standard image processing algorithms (threshold, Hough line/circle detection and contour detection), but the augmented reality devices require dynamic recognition of features detected in various distances and angles. The problem can be solved by deep learning CNN which is robust to orientation, scale and in cases when element is not fully visible. We tested two pretrained CNN models Mobilenet V1 and SSD Fast RCNN Inception V2 SSD extension have been tested to detect exact position. The results obtained were very promising in comparison to standard image processing techniques.


Deep learning Object recognition Augmented reality 



This work was supported by the Agency for Research and Development under the contract no. APVV-15-0602.


  1. 1.
    Agrawal, A.: Putting VR/AR to work (Editorial material). IEEE Comput. Graphics Appl. 38(1), 115–118 (2018)CrossRefGoogle Scholar
  2. 2.
    Takaseki, A.R., Nagashima, R., Kashima, H., Okazaki, T.: Development of anchoring support system using with AR toolkit. In: 2015 7th International Conference on Emerging Trends in Engineering and Technology, pp. 123–127 (2015)Google Scholar
  3. 3.
    Liu, H.Y., Wang, L.H.: An AR-based worker support system for human-robot collaboration. In: 27th International Conference on Flexible Automation and Intelligent Manufacturing, FAIM 2017, vol. 11, pp. 22–30 (2017)CrossRefGoogle Scholar
  4. 4.
    Chuck, C., Laskey, M., Krishnan, S., Joshi, R., Fox, R., Goldberg, K., et al:. Statistical data cleaning for deep learning of automation tasks from demonstrations. In: 13th IEEE Conference on Automation Science and Engineering (IEEE CASE), 20–23 August, pp. 1142–1149. IEEE, Xian (2017)Google Scholar
  5. 5.
    Bosse, S., Maniry, D., Muller, K.R., Wiegand, T., Samek, W.: Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans. Image Process. 27(1), 206–219 (2018)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Zhang, W., Li, C.H., Peng, G.L., Chen, Y.H., Zhang, Z.J.: A deep convolutional neural network with new training methods for bearing fault diagnosis under noisy environment and different working load. Mech. Syst. Signal Process. 100, 439–453 (2018)CrossRefGoogle Scholar
  7. 7.
    Saucedo-Espinosa, M.A., Escalante, H.J., Berrones, A.: Detection of defective embedded bearings by sound analysis: a machine learning approach. J. Intell. Manuf. 28(2), 489–500 (2017)CrossRefGoogle Scholar
  8. 8.
    Patil, K., Kulkarni, M., Sriraman, A., Karande, S.: Deep learning based car damage classification. In: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 50–54 (2017)Google Scholar
  9. 9.
    Liang, J., Mahler, J., Laskey, M., Li, P.S., Goldberg, K.: Using dVRK teleoperation to facilitate deep learning of automation tasks for an industrial robot. In: 13th IEEE Conference on Automation Science and Engineering (IEEE CASE), 20–23 August, pp. 1–8. IEEE, Xian (2017)Google Scholar
  10. 10.
    Komarov, A.S., Buehner, M.: adaptive probability thresholding in automated ice and open water detection from RADARSAT-2 images. IEEE Geosci. Remote Sens. Lett. 15(4), 552–556 (2018)CrossRefGoogle Scholar
  11. 11.
    Rasche, C.: Rapid contour detection for image classification. IET Image Proc. 12(4), 532–538 (2018)CrossRefGoogle Scholar
  12. 12.
    Manzanera, A., Nguyen, T.P., Xu, X.L.: Line and circle detection using dense one-to-one Hough transforms on greyscale images. EURASIP J. Image Video Process. 18 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Faculty of Manufacturing Technologies with a seat in Presov, Department of Industrial Engineering and InformaticsTechnical University of KosicePresovSlovak Republic

Personalised recommendations