ARTool—Augmented Reality Human-Machine Interface for Machining Setup and Maintenance

Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 751)

Abstract

In modern production lines, smaller batches to be produced and higher customization level of a single component bring to higher cost, related especially to setup and preparation of machines. The setup of a milling machine is an operation that requires time and may bring to errors that can be catastrophic. In this Chapter, the ARTool Augmented Reality framework for machine tool operations is presented. The framework permits to write and debug part-code in an augmented environment, to identify quicker misalignments and errors in fixing of new blank material, and to support maintenance operations. The ego-localization of the handheld device that depicts the augmented scene in machine work-area is based upon markers. The library that performs marker identification is brand-new and it is benchmarked throughout the Chapter against a state-of-the-art solution (ARUCO) and a ground truth (multi-stereoscopic motion capture). The Chapter also describes the general information flow and the context that brought to the conception of the ARTool framework, and presents a series of applications developed using the framework.

References

  1. 1.
    Apple Inc.: SceneKit Framework. https://developer.apple.com/scenekit/ (2016)
  2. 2.
    Bleser, G., Stricker, D.: Advanced tracking through efficient image processing and visual-inertial sensor fusion. Comput. Graph. 33(1), 59–72 (2009)CrossRefGoogle Scholar
  3. 3.
    Bondrea, I., Petruse, R.: Augmented reality—an improvement for computer integrated manufacturing. Adv. Mater. Res. 628, 330–336 (2013).  https://doi.org/10.4028/www.scientific.net/AMR.628.330
  4. 4.
    Bradski, G., et al.: The opencv library. Doct. Dobbs J. 25(11), 120–126 (2000)Google Scholar
  5. 5.
    Büttner, S., Sand, O., Röcker, C.: Extending the design space in industrial manufacturing through mobile projection. In: MobileHCI 2015—Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, pp. 1130–1133 (2015).  https://doi.org/10.1145/2786567.2794342
  6. 6.
    Chong, J., Ong, S., Nee, A.C., Youcef-Youmi, K.B.: Robot programming using augmented reality: an interactive method for planning collision-free paths. Robot. Comput.-Integr. Manuf. 25(3), 689–701 (2009).  https://doi.org/10.1016/j.rcim.2008.05.002
  7. 7.
    Ćuković, S., Devedžić, G., Pankratz, F., Baizid, K., Ghionea, I., Kostić, A.: Augmented reality simulation of cam spatial tool paths in prismatic milling sequences. IFIP Adv. Inf. Commun. Technol. 467, 516–525 (2015).  https://doi.org/10.1007/978-3-319-33111-9_47
  8. 8.
    Doshi, A., Smith, R., Thomas, B., Bouras, C.: Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. Int. J. Adv. Manuf. Technol. 1–15 (2016).  https://doi.org/10.1007/s00170-016-9164-5
  9. 9.
    Elia, V., Gnoni, M., Lanzilotto, A.: Evaluating the application of augmented reality devices in manufacturing from a process point of view: An ahp based model. Expert Syst. Appl. 63, 187–197 (2016).  https://doi.org/10.1016/j.eswa.2016.07.006
  10. 10.
    Fang, H., Ong, S., Nee, A.: Interactive robot trajectory planning and simulation using augmented reality. Robot. Comput.-Integr. Manuf. 28(2), 227–237 (2012).  https://doi.org/10.1016/j.rcim.2011.09.003
  11. 11.
    Fang, H., Ong, S., Nee, A.: Robot path and end-effector orientation planning using augmented reality. Proc. CIRP 3, 191–196 (2012).  https://doi.org/10.1016/j.procir.2012.07.034
  12. 12.
    Fiorentino, M., Uva, A., Gattullo, M., Debernardis, S., Monno, G.: Augmented reality on large screen for interactive maintenance instructions. Comput. Ind. 65(2), 270–278 (2014).  https://doi.org/10.1016/j.compind.2013.11.004
  13. 13.
    Gilbert, W.: Economics of Machining. Machining Theory and Practice, pp. 465–485 (1950)Google Scholar
  14. 14.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2003)Google Scholar
  15. 15.
    Hou, L., Wang, X., Bernold, L., Love, P.E.: Using animated augmented reality to cognitively guide assembly. J. Comput. Civ. Eng. 27(5), 439–451 (2013)CrossRefGoogle Scholar
  16. 16.
    Jozef, N.M., Miroslav, J., Ludmila, N.M.: Augmented reality aided control of industrial robots. Adv. Mater. Res. 1025–1026, 1145–1149 (2014).  https://doi.org/10.4028/www.scientific.net/AMR.1025-1026.1145
  17. 17.
    Kalpakjian, S., Schmid, S.R., Kok, C.W.: Manufacturing Processes for Engineering Materials. Pearson-Prentice Hall (2008)Google Scholar
  18. 18.
    Lowe, D.G.: Fitting parameterized three-dimensional models to images. IEEE Trans. Pattern Anal. Mach. Intell. 5, 441–450 (1991)CrossRefGoogle Scholar
  19. 19.
    Martínez, H., Laukkanen, S., Mattila, J.: A new hybrid approach for augmented reality maintenance in scientific facilities. Int. J. Adv. Robot. Syst. 1729, 8806 (2013). Ferre, M., Mattila, J., Siciliano, B., Bonnal, P. (eds.)Google Scholar
  20. 20.
    Meden, B., Knodel, S., Bourgeois, S.: Markerless augmented reality solution for industrial manufacturing. In: ISMAR 2014—IEEE International Symposium on Mixed and Augmented Reality—Science and Technology 2014, Proceedings, pp. 359–360 (2014).  https://doi.org/10.1109/ISMAR.2014.6948488
  21. 21.
    Monroy Reyes, A., Vergara Villegas, O., Miranda Bojrquez, E., Cruz Snchez, V., Nandayapa, M.: A mobile augmented reality system to support machinery operations in scholar environments. Comput. Appl. Eng. Educ. 24(6), 967–981 (2016).  https://doi.org/10.1002/cae.21772
  22. 22.
    Nee, A., Ong, S.: Virtual and augmented reality applications in manufacturing. In: IFAC Proceedings Volumes (IFAC-PapersOnline), pp. 15–26 (2013).  https://doi.org/10.3182/20130619-3-RU-3018.00637
  23. 23.
    Olwal, A., Gustafsson, J., Lindfors, C.: Spatial augmented reality on industrial cnc-machines. In: Proc. SPIE 6804, 680, 409–680, 409–9 (2008).  https://doi.org/10.1117/12.760960
  24. 24.
    Ong, S., Pang, Y., Nee, A.B.: Augmented reality aided assembly design and planning. CIRP Ann.—Manuf. Technol. 56(1), 49–52 (2007).  https://doi.org/10.1016/j.cirp.2007.05.014
  25. 25.
    Ong, S., Yuan, M., Nee, A.: Augmented reality applications in manufacturing: a survey. Int. J. Prod. Res. 46(10), 2707–2742 (2008).  https://doi.org/10.1080/00207540601064773
  26. 26.
    PTC Inc.: Product Lifecycle Management (PLM) Software. http://www.ptc.com/product-lifecycle-management (2016). Accessed 05 Oct 2017, 15:40:45
  27. 27.
    Ramrez, H.B., Mendoza, E., Mendoza, M., Gonzlez, E.: Application of augmented reality in statistical process control, to increment the productivity in manufacture. Proc. Comput. Sci. 75, 213–220 (2015).  https://doi.org/10.1016/j.procs.2015.12.240
  28. 28.
    Schaumlöffel, P., Talha, M., Gorecky, D., Meixner, G.: Augmented reality applications for future manufacturing. In: Proceedings of the 5th Manufacturing Science and Education-MSE, vol. 1, no. 5, pp. 2–5 (2011)Google Scholar
  29. 29.
    Schweighofer, G., Pinz, A.: Robust pose estimation from a planar target. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2024–2030 (2006)CrossRefGoogle Scholar
  30. 30.
    Setti, A., Bosetti, P., Ragni, M.: ARTool—Augmented reality platform for machining setup and maintenance. In: Bi, Y., Kapoor, S., Bhatia, R. (eds.) Proceedings of SAI Intelligent Systems Conference (IntelliSys) 2016. IntelliSys 2016. Lecture Notes in Networks and Systems, vol. 15, pp. 457–475, Springer, Cham (2018)Google Scholar
  31. 31.
    Surez-Warden, F., Mendvil, E., Ramrez, H., Garza Njera, L., Pantoja, G.: Mill setup manual aided by augmented reality. In: Mechanisms and Machine Science, vol. 25, pp. 433–441 (2015).  https://doi.org/10.1007/978-3-319-09858-6_41
  32. 32.
    Syberfeldt, A., Danielsson, O., Holm, M., Wang, L.b.: Dynamic operator instructions based on augmented reality and rule-based expert systems. In: Procedia CIRP, vol. 41, pp. 346–351 (2016).  https://doi.org/10.1016/j.procir.2015.12.113
  33. 33.
    Systems, D.: Solidworks Model Based Definitions. http://www.solidworks.it/sw/products/technical-communication/packages.htm (2016). Accessed 26 June 2017
  34. 34.
    Vignais, N., Miezal, M., Bleser, G., Mura, K., Gorecky, D., Marin, F.: Innovative system for real-time ergonomic feedback in industrial manufacturing. Appl. Ergon. 44(4), 566–574 (2013).  https://doi.org/10.1016/j.apergo.2012.11.008
  35. 35.
    Wang, X., Love, P.E., Kim, M.J., Park, C.S., Sing, C.P., Hou, L.: A conceptual framework for integrating building information modeling with augmented reality. Autom. Constr. 34, 37–44 (2013)CrossRefGoogle Scholar
  36. 36.
    Wang, Z., Ong, S., Nee, A.: Augmented reality aided interactive manual assembly design. Int. J. Adv. Manuf. Technol. 69(5–8), 1311–1321 (2013)CrossRefGoogle Scholar
  37. 37.
    Weinert, K., Zabel, A., Ungemach, E., Odendahl, S.: Improved nc path validation and manipulation with augmented reality methods. Prod. Eng. 2(4), 371–376 (2008).  https://doi.org/10.1007/s11740-008-0115-3
  38. 38.
    Wójcicki, T.: Supporting the diagnostics and the maintenance of technical devices with augmented reality. Diagnostyka 15(1), 43–47 (2014)Google Scholar
  39. 39.
    Zhang, J., Ong, S., Nee, A.: A multi-regional computation scheme in an ar-assisted in situ cnc simulation environment. CAD Comput. Aided Des. 42(12), 1167–1177 (2010).  https://doi.org/10.1016/j.cad.2010.06.007

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Pro-M Facility, Trentino Sviluppo S.P.A.RoveretoItaly
  2. 2.Department of Industrial EngineeringUniversity of TrentoTrentoItaly

Personalised recommendations