Skip to main content
SpringerLink
Log in

Visual guidance of a sealant dispensing robot for online detection of complex 3D-curve seams

  • Original Paper
  • Published:
International Journal on Interactive Design and Manufacturing (IJIDeM) Aims and scope Submit manuscript

Abstract

Dispensing and seam sealing in the manufacturing industry cause some problems related to human health, time consumption and material waste because of inconsistent sealers. However, at present, in small- and medium-sized factories, manual performance by workers and teaching manipulators by using teach-pendant to recognize seam sealing trajectories, especially for 3D complex seam curves, are still the two main approaches. To improve spraying quality, productivity, and repeatable consistency while reducing cost, this research presents an effective method of using robot and computer vision for autonomous detection of the seam sealing curve. First, the camera calibration methodology is proposed based on the laser triangular algorithm. Then, the coordinates of the seam curve are computed by applying the image processing algorithm. After that, the coordinates are processed and converted to the coordinates of the robot and transmitted to the controller to perform the task. By using this methodology, the robot can perform the gluing task without knowing the trajectory in advance; moreover, the system is not overly complicated and has high accuracy as well as a fast processing speed. The efficiency of the system is proven through experiments on 2D and 3D complex curves.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Fanuc, Dispensing Robot Systems | Adhesive, Glue, Epoxy, Urethane & Automotive Seam Sealer Dispensing, https://www.fanucamerica.com/solutions/applications/dispensing-robot-adhesive-dispensing-systems, Accessed 2021–10–11.

  2. Robotics & Automation: Kuka showcases new fully automated sealing and damping system at Audi https://roboticsandautomationnews.com/2020/08/21/kuka-showcases-new-fully-automated-sealing-and-damping-system-at-audi/35454/. Accessed 2021–11–11.

  3. BenchCat: BenchCat Fluid Dispensing Robot Software, https://benchcat.com/benchcat-robot-software.html. Accessed 2021–09–15.

  4. Yaskawa: MotoSim EG-VRC https://www.yaskawa.eu.com/products/software/productdetail/product/motosim-eg-vrc_1686. Accessed: 2021–10–15.

  5. Kuka, K.S.: https://www.kuka.com/en-de/products/robot-systems/software/planning-project-engineering-service-safety/kuka_sim. Accessed 2021–11–15.

  6. Wu, X., Li, Z., Wen, P.: An automatic shoe-groove feature extraction method based on robot and structural laser scanning. Int. J. Adv. Rob. Syst. 14, 1–14 (2017). https://doi.org/10.1177/1729881416678135

    Article  Google Scholar 

  7. Yang, L., Li, E., Long, T., Fan, J., Liang, Z.: A high-speed seam extraction method based on the novel structured-light sensor for arc welding robot: a review. IEEE Sens. J. 18(21), 8631–8641 (2018)

    Article  Google Scholar 

  8. Du, J., Yong, L.Y., Sun, M., Ge, J.S.: Study on stereo vision for 3D reconstruction of welding seam. Adv. Mater. Res. 850–851, 212–216 (2013)

    Article  Google Scholar 

  9. Guoyang, W., Guofeng, W., Kaisheng, X., Tinghao, Y., Yunsheng, F.: 6DOF object positioning and grasping approach for industrial robots based on boundary point cloud features. Math. Problems Eng. 2020, 1–12 (2020). https://doi.org/10.1155/2020/9279345

    Article  Google Scholar 

  10. Silvers, G.A., Fang, G.: Human machine interface in weld seam detection using an RGB-D camera. Adv. Mater. Res. 875–877, 1967–1971 (2014)

    Article  Google Scholar 

  11. Wang, Q., Jiao, W., Yu, R., Johnson, M.T., Zhang, Y.M.: Virtual reality robot-assisted welding based on human intention recognition. IEEE Trans. Autom. Sci. Eng. 17(2), 799–808 (2020). https://doi.org/10.1109/TASE.2019.2945607

    Article  Google Scholar 

  12. Pablo, R.G., Manuel, R.M.: Weld bead detection based on 3D geometric featuresand machine learning approaches. IEEE Access 7, 14714–14727 (2019). https://doi.org/10.1109/ACCESS.2019.2891367

    Article  Google Scholar 

  13. Panasonic Industry: VRPS (Virtual Robot Programming System), https://industrial.panasonic.com/ww/products-fa/welding/industrial-robots/arc-welding-robot/vrps. Accessed: 2021–10–13

  14. Chen, X., Chen, Y., You, B., Xie, J., Najjaran, H.: Detecting 6D poses of target objects from cluttered scenes by learning to align the point cloud patches with the CAD models. IEEE Access 8, 210640–210650 (2020)

    Article  Google Scholar 

  15. Song, K.T., Wu, C.H., Jiang, S.Y.: CAD-based pose estimation design for random bin picking using a RGB-D camera. J. Intell. Rob. Syst. 87, 455–470 (2017). https://doi.org/10.1109/ICCAS.2015.7364621

    Article  Google Scholar 

  16. Li, M., Hashimoto, K.: Fast and Robust Pose Estimation Algorithm for Bin Picking Using Point Pair Fea-ture. 24th International Conference on Pattern Recognition (ICPR), 2018, pp. 1604–1609. https://doi.org/10.1109/ICPR.2018.8545432

  17. Pitteri, G., Bugeau, A., Ilic, S., Lepetit, V.: 3D Object Detection and Pose Estimation of Unseen Objects in Color Images with Local Surface Embeddings. In: Ishikawa H., Liu CL., Pajdla T., Shi J. (eds) Computer Vision – ACCV 2020. ACCV 2020. Lecture Notes in Computer Science, vol 12622. Springer, Cham (2021).

  18. Hanh, L.D., Hieu, K.T.G.: 3D matching by combining CAD model and computer vision for autonomous bin picking. Int. J. Interact. Des. Manuf. 15, 239–247 (2021). https://doi.org/10.1007/s12008-021-00762-4

    Article  Google Scholar 

  19. Santos, P., Ferreira, M.J.: Computer Vision and Fuzzy Rules Applied to a Dispensing Application in an Industrial Desktop Robot. Chapter December 2006, DOI: https://doi.org/10.5772/4910

  20. Ting, Y., Chen, C., Feng, H., Chen, S.: Apply computer vision and neural network to glue dispenser route inspection. Int. Conf. Mechatron. Autom. 2008, 3882–3887 (2007). https://doi.org/10.1109/ICMA.2007.4304195

    Article  Google Scholar 

  21. Lai, M.-Y., Wang, L.-L.: Automatic shoe-pattern boundary extraction by image-processing techniques. Robot. Comput.-Integr. Manuf. 24(2), 217–227 (2008). https://doi.org/10.1016/j.rcim.2006.10.005

    Article  MathSciNet  Google Scholar 

  22. Kai-Fu, Y., Shao-Bing, G., Ce-Feng, G., Chao-Yi, L., Yong-Jie, L.: Boundary detection using double-opponency and spatial sparseness constraint. IEEE Trans. Image Process. 24(8), 2565–2578 (2015). https://doi.org/10.1109/TIP.2015.2425538

    Article  MathSciNet  MATH  Google Scholar 

  23. Chi, Y., Leung, M.K.H.: A general shape context framework for object identification. Comput. Vis. Image Underst. 112, 324–336 (2008). https://doi.org/10.1016/j.cviu.2008.05.009

    Article  Google Scholar 

  24. Wu, W., Wang, X., Xu, D., Yin, Y.: Position and orientation measurement for autonomous aerial refueling based on monocular vision. Int. J. Robot. Autom. 32, 4661–4672 (2017). https://doi.org/10.2316/Journal.206.2017.1.206-4661

    Article  Google Scholar 

  25. Huang, Y., Li, G., Shao, W., Gong, S., Zhang, X.: A novel dual-channel weld seam tracking system for aircraft T-joint welds. Int. J. Adv. Manuf. Technol. 91(4), 751–761 (2017)

    Article  Google Scholar 

  26. Zou, Y., Lan, R., Wei, X., Chen, J.: Robust seam tracking via a deep learning framework combining tracking and detection. Appl. Opt. 59, 4321–4331 (2020)

    Article  Google Scholar 

  27. Zhang, W., Lang, J.: Semi-supervised training for positioning of welding seams. Sensors 21(21), 7309 (2021). https://doi.org/10.3390/s21217309

    Article  Google Scholar 

  28. Yao, D., Zhang, C., Zhu, Z., Huang, J., Bi, J.: Trajectory clustering via deep representation learning. Int. Jt. Conf. Neural Netw. (IJCNN) 2017, 3880–3887 (2017). https://doi.org/10.1109/IJCNN.2017.7966345

    Article  Google Scholar 

  29. Jin, X., Lv, L., Chen, C., Yang, F., Chen, T.: A New Welding Seam Recognition Methodology Based on Deep Learning Model MRCNN. In: 7th International Conference on Information, Cybernetics, and Computational Social Systems (ICCSS), 2020, pp. 767–771. https://doi.org/10.1109/ICCSS52145.2020.9336927.

  30. Pagano, S., Russo, R., Savino, S.: A vision guided robotic system for flexible gluing process in the footwear industry. Robot. Comput.-Integr. Manuf. 65, 1065 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

We acknowledgment the support of time and facilities from Ho Chi Minh City University of Technology (HCMUT), VNU-HCM for this study.

Author information

Authors and Affiliations

  1. Department of Mechatronics, Falculty of Mechanical Engineering, Ho Chi Minh City University of Technology (HCMUT), 268 Ly Thuong Kiet Street, District 10, Ho Chi Minh City, Vietnam

    Le Duc Hanh & Cao Viet Thien

  2. Vietnam National University, HoChiMinh City, Linh Trung Ward, Thu Duc District, Ho Chi Minh City, Vietnam

    Le Duc Hanh & Cao Viet Thien

Authors
  1. Le Duc Hanh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cao Viet Thien
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

L.D.H.: Writing and review. C.V.T.: Coding and testing.

Corresponding author

Correspondence to Le Duc Hanh.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hanh, L.D., Thien, C.V. Visual guidance of a sealant dispensing robot for online detection of complex 3D-curve seams. Int J Interact Des Manuf 16, 1525–1532 (2022). https://doi.org/10.1007/s12008-022-00843-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12008-022-00843-y

Keywords

Search

Navigation