Skip to main content
SpringerLink
Log in

Vision-based localization for cooperative robot-CNC hybrid manufacturing

  • ORIGINAL ARTICLE
  • Published:
The International Journal of Advanced Manufacturing Technology Aims and scope Submit manuscript

Abstract

Wire and arc additive manufacturing (WAAM) has shown promise in recent years for producing large-scale parts with higher deposition rates than other additive processes. WAAM is often combined with subtractive machining to form a hybrid manufacturing process. This hybrid process can be realized by retrofitting computer numerical control (CNC) machines with deposition heads, adding spindles and deposition heads to robots, or developing part localization methods to transfer parts from an additive cell to a CNC machine. Here, a novel, robot-CNC hybrid configuration is introduced where a maneuverable robot is placed in front of a CNC machine to deposit material within the machine envelop. This method removes the need for part localization and the extensive machine modifications required for retrofitting; however, the problem of robot localization is also added. In this work, the effects of error in vision-based, contactless robot localization on machining parameters in a robot-machine hybrid process were studied. Performance was characterized on an implementation of this system using classical computer vision techniques. In addition, machining simulations were conducted to evaluate the effects of image-induced error on chip thickness, material removal rate, and machining allowance. Initial tests show that computer vision could adequately locate a robot for the hybrid WAAM process within .5 mm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

Data availability

All data is available upon request to the corresponding author

References

  1. Singh SR, Khanna P (2021) Wire arc additive manufacturing (WAAM) A new process to shape engineering materials. Mater Today Proc 44:118–128. https://doi.org/10.1016/j.matpr.2020.08.030. https://www.sciencedirect.com/science/article/pii/S2214785320358922

    Article  Google Scholar 

  2. Rodrigues TA, Duarte V, Miranda RM et al (2019) Current status and perspectives on wire and arc additive manufacturing (WAAM). Materials 12(7):1121. https://doi.org/10.3390/ma12071121. https://www.mdpi.com/1996-1944/12/7/1121, number: 7 Publisher: Multidisciplinary Digital Publishing Institute

    Article  Google Scholar 

  3. Flynn JM, Shokrani A, Newman ST et al (2016) Hybrid additive and subtractive machine tools –?research and industrial developments. Int J Mach Tools Manuf 101:79–101. https://doi.org/10.1016/j.ijmachtools.2015.11.007. https://www.sciencedirect.com/science/article/pii/S0890695515300894

    Article  Google Scholar 

  4. Yarrapareddy E, Kovacevic R (2008) Synthesis and characterization of laser-based direct metal deposited nano-particles reinforced surface coatings for industrial slurry erosion applications. Surf Coat Technol 202(10):1951–1965. https://doi.org/10.1016/j.surfcoat.2007.08.032. https://www.sciencedirect.com/science/article/pii/S0257897207008985

    Article  Google Scholar 

  5. He W, Jiang Z, Ming W et al (2021) A critical review for machining positioning based on computer vision. Measurement 184:109,973. https://doi.org/10.1016/j.measurement.2021.109973. https://www.sciencedirect.com/science/article/pii/S0263224121009040

    Article  Google Scholar 

  6. Dinham M, Fang G (2009) A low cost hand-eye calibration method for arc welding robots. In: 2009 IEEE international conference on robotics and biomimetics (ROBIO), pp 1889–1893. https://doi.org/10.1109/ROBIO.2009.5420552

  7. Wu Q, Qin X, Li Y et al (2021) Automatic calibration of work coordinates for robotic wire and arc additive re-manufacturing with a single camera. Int J Adv Manuf Technol 114(9):2577–2589. https://doi.org/10.1007/s00170-021-06664-9

    Article  Google Scholar 

  8. Zhang W, Ma X, Cui L et al (2008) 3 points calibration method of part coordinates for arc welding robot. In: Hutchison D, Kanade T, Kittler J et al (eds) Intelligent robotics and applications, vol 5314. Springer, Berlin, pp 216–224. https://doi.org/10.1007/978-3-540-88513-9_24. series Title: Lecture Notes in Computer Science

  9. Zhang X, Li M, Lim JH et al (2018) Large-scale 3D printing by a team of mobile robots. Autom Constr 95:98–106. https://doi.org/10.1016/j.autcon.2018.08.004. https://www.sciencedirect.com/science/article/pii/S0926580518304011

    Article  Google Scholar 

  10. Shiu Y, Ahmad S (1987) Calibration of wrist-mounted robotic sensors by solving homogeneous transform equations of the form AX=XB. Dep Electr Comput Eng Tech Rep. https://docs.lib.purdue.edu/ecetr/592

  11. Zeng J, Cao GZ, Li WB et al (2019) An algorithm of hand-eye calibration for arc welding robot. In: 2019 16th international conference on ubiquitous robots (UR), pp 1–6. https://doi.org/10.1109/URAI.2019.8768578, iSSN 2325-033X

  12. Tian FP, Feng W, Zhang Q et al (2019) Active camera relocalization from a single reference image without hand-eye calibration. IEEE Trans Pattern Anal Mach Intell 41 (12):2791–2806. https://doi.org/10.1109/TPAMI.2018.2870646. conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence

    Article  Google Scholar 

  13. Feddema J, Mitchell O (1989) Vision-guided servoing with feature-based trajectory generation (for robots). IEEE Trans Robot Autom 5(5):691–700. https://doi.org/10.1109/70.88086. conference Name: IEEE Transactions on Robotics and Automation

    Article  Google Scholar 

  14. Chang WC (2018) Robotic assembly of smartphone back shells with eye-in-hand visual servoing. Robot Comput Integr Manuf 50:102–113. https://doi.org/10.1016/j.rcim.2017.09.010. https://www.sciencedirect.com/science/article/pii/S073658451630271X

    Article  Google Scholar 

  15. Zhou L, Lin T, Chen SB (2006) Autonomous acquisition of seam coordinates for arc welding robot based on visual servoing. J Intell Robot Syst 47(3):239–255. https://doi.org/10.1007/s10846-006-9078-9. http://link.springer.com/10.1007/s10846-006-9078-9

    Article  Google Scholar 

  16. Oyelola O, Crawforth P, M’Saoubi R et al (2016) Machining of additively manufactured parts implications for surface integrity. Procedia CIRP 45:119–122. https://doi.org/10.1016/j.procir.2016.02.066. https://www.sciencedirect.com/science/article/pii/S2212827116003486

    Article  Google Scholar 

  17. Barath Kumar MD, Manikandan M (2022) Assessment of process, parameters, residual stress mitigation, post treatments and finite element analysis simulations of wire arc additive manufacturing technique. Met Mater Int 28(1):54–111. https://doi.org/10.1007/s12540-021-01015-5

    Article  Google Scholar 

  18. de Oliveira AR, Del Conte EG (2021) Concurrent improvement of surface roughness and residual stress of as-built and aged additively manufactured maraging steel post-processed by milling. Int J Adv Manuf Technol 116(7):2309–2323. https://doi.org/10.1007/s00170-021-07527-z

    Article  Google Scholar 

  19. Fuchs C, Baier D, Semm T et al (2020) Determining the machining allowance for WAAM parts. Prod Eng 14(5):629–637. https://doi.org/10.1007/s11740-020-00982-9

    Article  Google Scholar 

  20. Manogharan G (2014) Hybrid manufacturing analysis of integrating additive and subtractive methods. Ph.D., Carolina State University, United States – North Carolina. https://www.proquest.com/docview/1642719354/abstract/A23FCD218CF2402FPQ/1, iSBN:9781321414097

  21. Li F, Chen S, Shi J et al (2017) Evaluation and optimization of a hybrid manufacturing process combining wire arc additive manufacturing with milling for the fabrication of stiffened panels. Appl Sci 7(12):1233. https://doi.org/10.3390/app7121233. https://www.mdpi.com/2076-3417/7/12/1233, number: 12 Publisher: Multidisciplinary Digital Publishing Institute

    Article  Google Scholar 

  22. Norberto López de Lacalle L, Campa FJ, Lamikiz A (2011) 3 – milling. In: Paulo Davim J (ed) Modern machining technology. Woodhead Publishing, pp 213–303. https://doi.org/10.1533/9780857094940.213https://www.sciencedirect.com/science/article/pii/B9780857090997500034

  23. Chernovol N, Sharma A, Tjahjowidodo T et al (2021) Machinability of wire and arc additive manufactured components. CIRP J Manuf Sci Technol 35:379–389. https://doi.org/10.1016/j.cirpj.2021.06.022. https://www.sciencedirect.com/science/article/pii/S1755581721001140

    Article  Google Scholar 

  24. de Araujo PRM, Lins RG (2020) Computer vision system for workpiece referencing in three-axis machining centers. Int J Adv Manuf Technol 106(5):2007–2020. https://doi.org/10.1007/s00170-019-04626-w

    Article  MathSciNet  Google Scholar 

  25. Sharf I, Wolf A, Rubin MB (2010) Arithmetic and geometric solutions for average rigid-body rotation. Mech Mach Theory 45 (9):1239–1251. https://doi.org/10.1016/j.mechmachtheory.2010.05.002. https://www.sciencedirect.com/science/article/pii/S0094114X10000790

    Article  MATH  Google Scholar 

  26. De K, Masilamani V (2013) Image sharpness measure for blurred images in frequency domain. Procedia Eng 64:149–158. https://doi.org/10.1016/j.proeng.2013.09.086. https://www.sciencedirect.com/science/article/pii/S1877705813016007

    Article  Google Scholar 

  27. Nguyen L, Buhl J, Bambach M (2020) Continuous Eulerian tool path strategies for wire-arc additive manufacturing of rib-web structures with machine-learning-based adaptive void filling. Addit Manuf 35:101,265. https://doi.org/10.1016/j.addma.2020.101265. https://www.sciencedirect.com/science/article/pii/S2214860420306370

    Article  Google Scholar 

  28. Wahab MNA, Sivadev N, Sundaraj K (2011) Target distance estimation using monocular vision system for mobile robot. In: 2011 IEEE conference on open systems, pp 11–15. https://doi.org/10.1109/ICOS.2011.6079296

Download references

Acknowledgements

Thank you to Jaime Berez (Georgia Institute of Technology) for providing a point-cloud fitting software library used in a portion of this research.

Funding

This work was supported by the US Department of Energy DE-EE0008303

Author information

Authors and Affiliations

  1. GWW School of Mechanical Engineering, Georgia Institute of Technology, 801 Ferst Dr, Atlanta, GA, 30332, USA

    Jesse Goodwin & Christopher Saldaña

Authors
  1. Jesse Goodwin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher Saldaña
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: Jesse Goodwin, Christopher Saldaña. Methodology: Jesse Goodwin. Validation: Jesse Goodwin, Christopher Saldaña. Analysis: Jesse Goodwin. Writing - original draft: Jesse Goodwin. Writing—review and editing: Christopher Saldaña. Funding acquisition: Christopher Saldaña. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Jesse Goodwin.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Conflict of interest

The authors declare no competing interests

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Goodwin, J., Saldaña, C. Vision-based localization for cooperative robot-CNC hybrid manufacturing. Int J Adv Manuf Technol 126, 241–258 (2023). https://doi.org/10.1007/s00170-023-11009-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00170-023-11009-9

Keywords

Search

Navigation