ARTool—Augmented Reality Human-Machine Interface for Machining Setup and Maintenance

Conference paper
Part of the Studies in Computational Intelligence book series (SCI, volume 751)


In modern production lines, smaller batches to be produced and higher customization level of a single component bring to higher cost, related especially to setup and preparation of machines. The setup of a milling machine is an operation that requires time and may bring to errors that can be catastrophic. In this Chapter, the ARTool Augmented Reality framework for machine tool operations is presented. The framework permits to write and debug part-code in an augmented environment, to identify quicker misalignments and errors in fixing of new blank material, and to support maintenance operations. The ego-localization of the handheld device that depicts the augmented scene in machine work-area is based upon markers. The library that performs marker identification is brand-new and it is benchmarked throughout the Chapter against a state-of-the-art solution (ARUCO) and a ground truth (multi-stereoscopic motion capture). The Chapter also describes the general information flow and the context that brought to the conception of the ARTool framework, and presents a series of applications developed using the framework.


  1. 1.
    Apple Inc.: SceneKit Framework. (2016)
  2. 2.
    Bleser, G., Stricker, D.: Advanced tracking through efficient image processing and visual-inertial sensor fusion. Comput. Graph. 33(1), 59–72 (2009)CrossRefGoogle Scholar
  3. 3.
    Bondrea, I., Petruse, R.: Augmented reality—an improvement for computer integrated manufacturing. Adv. Mater. Res. 628, 330–336 (2013).
  4. 4.
    Bradski, G., et al.: The opencv library. Doct. Dobbs J. 25(11), 120–126 (2000)Google Scholar
  5. 5.
    Büttner, S., Sand, O., Röcker, C.: Extending the design space in industrial manufacturing through mobile projection. In: MobileHCI 2015—Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, pp. 1130–1133 (2015).
  6. 6.
    Chong, J., Ong, S., Nee, A.C., Youcef-Youmi, K.B.: Robot programming using augmented reality: an interactive method for planning collision-free paths. Robot. Comput.-Integr. Manuf. 25(3), 689–701 (2009).
  7. 7.
    Ćuković, S., Devedžić, G., Pankratz, F., Baizid, K., Ghionea, I., Kostić, A.: Augmented reality simulation of cam spatial tool paths in prismatic milling sequences. IFIP Adv. Inf. Commun. Technol. 467, 516–525 (2015).
  8. 8.
    Doshi, A., Smith, R., Thomas, B., Bouras, C.: Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. Int. J. Adv. Manuf. Technol. 1–15 (2016).
  9. 9.
    Elia, V., Gnoni, M., Lanzilotto, A.: Evaluating the application of augmented reality devices in manufacturing from a process point of view: An ahp based model. Expert Syst. Appl. 63, 187–197 (2016).
  10. 10.
    Fang, H., Ong, S., Nee, A.: Interactive robot trajectory planning and simulation using augmented reality. Robot. Comput.-Integr. Manuf. 28(2), 227–237 (2012).
  11. 11.
    Fang, H., Ong, S., Nee, A.: Robot path and end-effector orientation planning using augmented reality. Proc. CIRP 3, 191–196 (2012).
  12. 12.
    Fiorentino, M., Uva, A., Gattullo, M., Debernardis, S., Monno, G.: Augmented reality on large screen for interactive maintenance instructions. Comput. Ind. 65(2), 270–278 (2014).
  13. 13.
    Gilbert, W.: Economics of Machining. Machining Theory and Practice, pp. 465–485 (1950)Google Scholar
  14. 14.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2003)Google Scholar
  15. 15.
    Hou, L., Wang, X., Bernold, L., Love, P.E.: Using animated augmented reality to cognitively guide assembly. J. Comput. Civ. Eng. 27(5), 439–451 (2013)CrossRefGoogle Scholar
  16. 16.
    Jozef, N.M., Miroslav, J., Ludmila, N.M.: Augmented reality aided control of industrial robots. Adv. Mater. Res. 1025–1026, 1145–1149 (2014).
  17. 17.
    Kalpakjian, S., Schmid, S.R., Kok, C.W.: Manufacturing Processes for Engineering Materials. Pearson-Prentice Hall (2008)Google Scholar
  18. 18.
    Lowe, D.G.: Fitting parameterized three-dimensional models to images. IEEE Trans. Pattern Anal. Mach. Intell. 5, 441–450 (1991)CrossRefGoogle Scholar
  19. 19.
    Martínez, H., Laukkanen, S., Mattila, J.: A new hybrid approach for augmented reality maintenance in scientific facilities. Int. J. Adv. Robot. Syst. 1729, 8806 (2013). Ferre, M., Mattila, J., Siciliano, B., Bonnal, P. (eds.)Google Scholar
  20. 20.
    Meden, B., Knodel, S., Bourgeois, S.: Markerless augmented reality solution for industrial manufacturing. In: ISMAR 2014—IEEE International Symposium on Mixed and Augmented Reality—Science and Technology 2014, Proceedings, pp. 359–360 (2014).
  21. 21.
    Monroy Reyes, A., Vergara Villegas, O., Miranda Bojrquez, E., Cruz Snchez, V., Nandayapa, M.: A mobile augmented reality system to support machinery operations in scholar environments. Comput. Appl. Eng. Educ. 24(6), 967–981 (2016).
  22. 22.
    Nee, A., Ong, S.: Virtual and augmented reality applications in manufacturing. In: IFAC Proceedings Volumes (IFAC-PapersOnline), pp. 15–26 (2013).
  23. 23.
    Olwal, A., Gustafsson, J., Lindfors, C.: Spatial augmented reality on industrial cnc-machines. In: Proc. SPIE 6804, 680, 409–680, 409–9 (2008).
  24. 24.
    Ong, S., Pang, Y., Nee, A.B.: Augmented reality aided assembly design and planning. CIRP Ann.—Manuf. Technol. 56(1), 49–52 (2007).
  25. 25.
    Ong, S., Yuan, M., Nee, A.: Augmented reality applications in manufacturing: a survey. Int. J. Prod. Res. 46(10), 2707–2742 (2008).
  26. 26.
    PTC Inc.: Product Lifecycle Management (PLM) Software. (2016). Accessed 05 Oct 2017, 15:40:45
  27. 27.
    Ramrez, H.B., Mendoza, E., Mendoza, M., Gonzlez, E.: Application of augmented reality in statistical process control, to increment the productivity in manufacture. Proc. Comput. Sci. 75, 213–220 (2015).
  28. 28.
    Schaumlöffel, P., Talha, M., Gorecky, D., Meixner, G.: Augmented reality applications for future manufacturing. In: Proceedings of the 5th Manufacturing Science and Education-MSE, vol. 1, no. 5, pp. 2–5 (2011)Google Scholar
  29. 29.
    Schweighofer, G., Pinz, A.: Robust pose estimation from a planar target. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2024–2030 (2006)CrossRefGoogle Scholar
  30. 30.
    Setti, A., Bosetti, P., Ragni, M.: ARTool—Augmented reality platform for machining setup and maintenance. In: Bi, Y., Kapoor, S., Bhatia, R. (eds.) Proceedings of SAI Intelligent Systems Conference (IntelliSys) 2016. IntelliSys 2016. Lecture Notes in Networks and Systems, vol. 15, pp. 457–475, Springer, Cham (2018)Google Scholar
  31. 31.
    Surez-Warden, F., Mendvil, E., Ramrez, H., Garza Njera, L., Pantoja, G.: Mill setup manual aided by augmented reality. In: Mechanisms and Machine Science, vol. 25, pp. 433–441 (2015).
  32. 32.
    Syberfeldt, A., Danielsson, O., Holm, M., Wang, L.b.: Dynamic operator instructions based on augmented reality and rule-based expert systems. In: Procedia CIRP, vol. 41, pp. 346–351 (2016).
  33. 33.
    Systems, D.: Solidworks Model Based Definitions. (2016). Accessed 26 June 2017
  34. 34.
    Vignais, N., Miezal, M., Bleser, G., Mura, K., Gorecky, D., Marin, F.: Innovative system for real-time ergonomic feedback in industrial manufacturing. Appl. Ergon. 44(4), 566–574 (2013).
  35. 35.
    Wang, X., Love, P.E., Kim, M.J., Park, C.S., Sing, C.P., Hou, L.: A conceptual framework for integrating building information modeling with augmented reality. Autom. Constr. 34, 37–44 (2013)CrossRefGoogle Scholar
  36. 36.
    Wang, Z., Ong, S., Nee, A.: Augmented reality aided interactive manual assembly design. Int. J. Adv. Manuf. Technol. 69(5–8), 1311–1321 (2013)CrossRefGoogle Scholar
  37. 37.
    Weinert, K., Zabel, A., Ungemach, E., Odendahl, S.: Improved nc path validation and manipulation with augmented reality methods. Prod. Eng. 2(4), 371–376 (2008).
  38. 38.
    Wójcicki, T.: Supporting the diagnostics and the maintenance of technical devices with augmented reality. Diagnostyka 15(1), 43–47 (2014)Google Scholar
  39. 39.
    Zhang, J., Ong, S., Nee, A.: A multi-regional computation scheme in an ar-assisted in situ cnc simulation environment. CAD Comput. Aided Des. 42(12), 1167–1177 (2010).

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.Pro-M Facility, Trentino Sviluppo S.P.A.RoveretoItaly
  2. 2.Department of Industrial EngineeringUniversity of TrentoTrentoItaly

Personalised recommendations