Skip to main content
SpringerLink
Log in

Exploiting VR and AR Technologies in Education and Training to Inclusive Robotics

  • Conference paper
  • First Online:
Education in & with Robotics to Foster 21st-Century Skills (EDUROBOTICS 2021)

Part of the book series: Studies in Computational Intelligence ((SCI,volume 982))

Included in the following conference series:

  • 699 Accesses

Abstract

Nowadays, robotics applications are no more limited to industrial rigid production cells or research laboratories and are spreading in several domains in which interaction with humans is required. In this context, educating professionals, workers, and students in the use and understanding of robotic systems is paramount to ensure the social acceptance and uptake of interactive robots. At the same time, the request for innovative training and teaching methodologies is growing, and virtual and augmented reality technologies as educational tools are gaining the attention of public institutions, companies, and healthcare facilities. In this paper, we focus on how VR/AR technologies can be exploited to teach robotics and to train in the use of specific robot platforms, providing a review of the available resources and discussing their advantages and disadvantages in the “inclusive robotics” perspective.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://unstats.un.org/unsd/ccsa/documents/covid19-report-ccsa.pdf.

  2. 2.

    https://ec.europa.eu/eurostat/en/web/products-eurostat-news/-/DDN-20200604-1.

References

  1. Schwab, K.: The fourth industrial revolution. Crown (2017)

    Google Scholar 

  2. Alimisis, D., Loukatos, D., Zoulias, E., Alimisi, R.: The role of education for the social uptake of robotics: the case of the eCraft2Learn project. In: Inclusive Robotics for a Better Society, Cham, pp. 180–187 (2020). https://doi.org/10.1007/978-3-030-24074-5_30

  3. Pozzi, M., Prattichizzo, D., Malvezzi, M.: On-line educational resources on robotics: a review. In: Inclusive Robotics for a Better Society, Cham, pp. 141–147 (2020). https://doi.org/10.1007/978-3-030-24074-5_25

  4. Pozzi, M., Malvezzi, M., Prattichizzo, D.: MOOC on the art of grasping and manipulation in robotics: design choices and lessons learned. In: Robotics in Education, Cham, pp. 71–78 (2019). https://doi.org/10.1007/978-3-319-97085-1_7

  5. Ferrarelli, P., et al.: Improving students’ concepts about newtonian mechanics using mobile robots. In: Lepuschitz, W., Merdan, M., Koppensteiner, G., Balogh, R., Obdržálek, D. (eds.) Robotics in Education, vol. 829, pp. 113–124. Springer International Publishing, Cham (2019)

    Chapter  Google Scholar 

  6. Martín-Gutiérrez, J., Mora, C.E., Añorbe-Díaz, B., González-Marrero, A.: Virtual technologies trends in education. Eurasia J Math Sci T 13(2), January 2017. https://doi.org/10.12973/eurasia.2017.00626a

  7. Ibáñez, M.-B., Delgado-Kloos, C.: Augmented reality for STEM learning: a systematic review. Comput. Educ. 123, 109–123 (2018). https://doi.org/10.1016/j.compedu.2018.05.002

    Article  Google Scholar 

  8. Bellalouna, F.: New approach for industrial training using virtual reality technology. Procedia CIRP 93, 262–267 (2020). https://doi.org/10.1016/j.procir.2020.03.008

    Article  Google Scholar 

  9. Chinello, F., Koumaditis, K.: Virtual immersive educational systems: early results and lessons learned. In: SIGGRAPH Asia 2019 Posters, Brisbane QLD Australia, pp. 1–2, November 2019. https://doi.org/10.1145/3355056.3364586

  10. Pozzi, M., Prattichizzo, D., Malvezzi, M.: Accessible educational resources for teaching and learning robotics. Robotics 10(1), 38 (2021). https://doi.org/10.3390/robotics10010038

    Article  Google Scholar 

  11. Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. 77(12), 1321–1329 (1994)

    Google Scholar 

  12. Suh, A., Prophet, J.: The state of immersive technology research: a literature analysis. Comput. Hum. Behav. 86, 77–90 (2018)

    Article  Google Scholar 

  13. Radianti, J., Majchrzak, T.A., Fromm, J., Wohlgenannt, I.: A systematic review of immersive virtual reality applications for higher education: design elements, lessons learned, and research agenda. Comput. Educ. 147, (2020)

    Article  Google Scholar 

  14. Koumaditis, K., Chinello, F., Mitkidis, P., Karg, S.: Effectiveness of virtual versus physical training: the case of assembly tasks, trainer’s verbal assistance, and task complexity. IEEE Comput. Grap. Appl. 40(5), 41–56 (2020). https://doi.org/10.1109/MCG.2020.3006330

    Article  Google Scholar 

  15. Koumaditis, K., Chinello, F., Venckute, S.: Design of a virtual reality and haptic setup linking arousals to training scenarios: a preliminary stage. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Reutlingen, pp. 1–2, March 2018. https://doi.org/10.1109/vr.2018.8446528

  16. Chittaro, L., Buttussi, F.: Assessing knowledge retention of an immersive serious game vs. a traditional education method in aviation safety. IEEE Trans. Vis. Comput. Graph. 21(4), 529–538 (2015)

    Article  Google Scholar 

  17. Krokos, E., Plaisant, C., Varshney, A.: Virtual memory palaces: immersion aids recall. Virtual Reality 23(1), 1–15 (2019)

    Article  Google Scholar 

  18. Carlson, P., Peters, A., Gilbert, S.B., Vance, J.M., Luse, A.: Virtual training: learning transfer of assembly tasks. IEEE Trans. Vis. Comput. Graph. 21(6), 770–782 (2015)

    Article  Google Scholar 

  19. Vaughan, N., John, N., Rees, N.: ParaVR: paramedic virtual reality training simulator. In: 2019 International Conference on Cyberworlds (CW), pp. 21–24 (2019)

    Google Scholar 

  20. Butt, A.L., Kardong-Edgren, S., Ellertson, A.: Using game-based virtual reality with haptics for skill acquisition. Clin. Simul. Nursing 16, 25–32 (2018)

    Article  Google Scholar 

  21. Shorey, S., Ang, E., Ng, E.D., Yap, J., Lau, L.S.T., Chui, C.K.: Communication skills training using virtual reality: a descriptive qualitative study. Nurse Educ. Today 94, (2020)

    Article  Google Scholar 

  22. McFaul, H., FitzGerald, E.: A realist evaluation of student use of a virtual reality smartphone application in undergraduate legal education. Br. J. Educ. Technol. 51(2), 572–589 (2020)

    Article  Google Scholar 

  23. Webel, S., Bockholt, U., Engelke, T., Gavish, N., Olbrich, M., Preusche, C.: An augmented reality training platform for assembly and maintenance skills. Robot. Auton. Syst. 61(4), 398–403 (2013)

    Article  Google Scholar 

  24. Gavish, N., et al.: Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks. Interact. Learn. Environ. 23(6), 778–798 (2015)

    Article  Google Scholar 

  25. Catal, C., Akbulut, A., Tunali, B., Ulug, E., Ozturk, E.: Evaluation of augmented reality technology for the design of an evacuation training game. Virtual Reality, 1–10 (2019)

    Google Scholar 

  26. Rojas-Muñoz, E., et al.: Evaluation of an augmented reality platform for austere surgical telementoring: a randomized controlled crossover study in cricothyroidotomies. NPJ Digital Med. 3(1), 1–9 (2020)

    Article  Google Scholar 

  27. Simeone, A.L., Speicher, M., Molnar, A., Wilde, A., Daiber, F.: LIVE: the human role in learning in immersive virtual environments. In: Symposium on Spatial User Interaction, pp. 1–11 (2019)

    Google Scholar 

  28. Allcoat, D., von Mühlenen, A.: Learning in virtual reality: effects on performance, emotion and engagement. Res. Learn. Technol. 26 (2018)

    Google Scholar 

  29. Rahman, Y., Asish, S.M., Fisher, N.P., Bruce, E.C., Kulshreshth, A.K., Borst, C.W.: Exploring eye gaze visualization techniques for identifying distracted students in educational VR. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 868–877 (2020)

    Google Scholar 

  30. Tobar-Muñoz, H., Baldiris, S., Fabregat, R.: Augmented reality game-based learning: enriching students’ experience during reading comprehension activities. J. Educ. Comput. Res. 55(7), 901–936 (2017)

    Article  Google Scholar 

  31. Chen, C.-M., Tsai, Y.-N.: Interactive augmented reality system for enhancing library instruction in elementary schools. Comput. Educ. 59(2), 638–652 (2012)

    Article  Google Scholar 

  32. Erbas, C., Demirer, V.: The effects of augmented reality on students’ academic achievement and motivation in a biology course. J. Comput. Assist. Learn. 35(3), 450–458 (2019)

    Article  Google Scholar 

  33. Sahin, D., Yilmaz, R.M.: The effect of augmented reality technology on middle school students’ achievements and attitudes towards science education. Comput. Educ. 144, (2020)

    Article  Google Scholar 

  34. Roldán, J.J., Peña-Tapia, E., Martín-Barrio, A., Olivares-Méndez, M.A., Del Cerro, J., Barrientos, A.: Multi-robot interfaces and operator situational awareness: study of the impact of immersion and prediction. Sensors 17(8), 1720 (2017)

    Article  Google Scholar 

  35. Pérez, L., Diez, E., Usamentiaga, R., García, D.F.: Industrial robot control and operator training using virtual reality interfaces. Comput. Ind. 109, 114–120 (2019)

    Article  Google Scholar 

  36. Haruna, M., Ogino, M., Koike-Akino, T.: Proposal and evaluation of visual haptics for manipulation of remote machine system. Front. Robot. AI 7, (2020). https://doi.org/10.3389/frobt

    Article  Google Scholar 

  37. Pai, Y.S., Yap, H.J., Dawal, S.Z.M., Ramesh, S., Phoon, S.Y.: Virtual planning, control, and machining for a modular-based automated factory operation in an augmented reality environment. Sci. Rep. 6, 27380 (2016)

    Article  Google Scholar 

  38. Peral-Boiza, M., Gomez-Fernandez, T., Sanchez-Gonzalez, P., Rodriguez-Vila, B., Gómez, E.J., Gutiérrez, Á.: Position based model of a flexible ureterorenoscope in a virtual reality training platform for a minimally invasive surgical robot. IEEE Access 7, 177414–177426 (2019)

    Article  Google Scholar 

  39. Wang, F., et al.: The application of virtual reality training for anastomosis during robot-assisted radical prostatectomy. Asian J. Urol. (2019)

    Google Scholar 

  40. Knopp, S., Lorenz, M., Pelliccia, L., Klimant, P.: Using industrial robots as haptic devices for VR-training. In: 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 607–608 (2018)

    Google Scholar 

  41. Mariani, A., Pellegrini, E., Enayati, N., Kazanzides, P., Vidotto, M., De Momi, E.: Design and evaluation of a performance-based adaptive curriculum for robotic surgical training: a pilot study. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 2162–2165 (2018)

    Google Scholar 

  42. Raison, N., et al.: Competency based training in robotic surgery: benchmark scores for virtual reality robotic simulation. BJU Int. 119(5), 804–811 (2017)

    Article  Google Scholar 

  43. de la Iglesia, D.H., Mendes, A.S., González, G.V., Jiménez-Bravo, D.M., de Paz Santana, J.F.: Connected elbow exoskeleton system for rehabilitation training based on virtual reality and context-aware. Sensors, 20(3), 858 (2020)

    Google Scholar 

  44. Grimm, F., Naros, G., Gharabaghi, A.: Closed-loop task difficulty adaptation during virtual reality reach-to-grasp training assisted with an exoskeleton for stroke rehabilitation. Front. Neurosci. 10, 518 (2016)

    Google Scholar 

  45. Christensen, N.H., et al.: Depth cues in augmented reality for training of robot-assisted minimally invasive surgery. In: Proceedings of the 21st International Academic Mindtrek Conference, pp. 120–126 (2017)

    Google Scholar 

  46. Chowriappa, A., et al.: Augmented-reality-based skills training for robot-assisted urethrovesical anastomosis: a multi-institutional randomised controlled trial. BJU Int. 115(2), 336–345 (2015)

    Article  Google Scholar 

  47. Crespo, R., García, R., Quiroz, S.: Virtual reality application for simulation and off-line programming of the mitsubishi movemaster RV-M1 robot integrated with the oculus rift to improve students training. Procedia Comput. Sci. 75, 107–112 (2015)

    Article  Google Scholar 

  48. Theofanidis, M., Sayed, S.I., Lioulemes, A., Makedon, F.: Varm: using virtual reality to program robotic manipulators. In: Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments, pp. 215–221 (2017)

    Google Scholar 

  49. Román-Ibáñez, V., Pujol-López, F.A., Mora-Mora, H., Pertegal-Felices, M.L., Jimeno-Morenilla, A.: A low-cost immersive virtual reality system for teaching robotic manipulators programming. Sustainability 10(4), 1102 (2018)

    Article  Google Scholar 

  50. Jara, C.A., Candelas-Herías, F.A., Fernández, M., Torres, F.: An augmented reality interface for training robotics through the web (2009)

    Google Scholar 

  51. Cheli, M., Sinapov, J., Danahy, E.E., Rogers, C.: Towards an augmented reality framework for k-12 robotics education (2018)

    Google Scholar 

  52. Krajník, T., Vonásek, V., Fišer, D., Faigl, J.: AR-drone as a platform for robotic research and education. In: International Conference on Research and Education in Robotics, pp. 172–186 (2011)

    Google Scholar 

  53. Quintero, C.P., Li, S., Pan, M.K., Chan, W.P., Van der Loos, H.M., Croft, E.: Robot programming through augmented trajectories in augmented reality. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1838–1844 (2018)

    Google Scholar 

  54. Ostanin, M., Mikhel, S., Evlampiev, A., Skvortsova, V., Klimchik, A.: Human-robot interaction for robotic manipulator programming in Mixed Reality. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 2805–2811 (2020)

    Google Scholar 

  55. Asín-Prieto, G., et al.: Haptic adaptive feedback to promote motor learning with a robotic ankle exoskeleton integrated with a video game. Front. Bioeng. Biotechnol. 8, 113 (2020)

    Article  Google Scholar 

  56. Mubin, O., Alnajjar, F., Jishtu, N., Alsinglawi, B., Al Mahmud, A.: Exoskeletons With virtual reality, augmented reality, and gamification for stroke patients’ rehabilitation: systematic review. JMIR Rehabil. Assistive Technol. 6(2), e12010 (2019)

    Google Scholar 

  57. Nagai, K.: Learning while doing: practical robotics education. IEEE Robot. Autom. Mag. 8(2), 39–43 (2001)

    Article  Google Scholar 

  58. Alimisis, D.: Educational robotics: open questions and new challenges. Themes Sci. Technol. Educ. 6(1), 63–71 (2013)

    Google Scholar 

  59. Potkonjak, V., et al.: Virtual laboratories for education in science, technology, and engineering: a review. Comput. Educ. 95, 309–327 (2016)

    Article  Google Scholar 

  60. Corke, P.: Robotics, vision and control: fundamental algorithms in MATLAB® second, completely revised, vol. 118. Springer (2017)

    Google Scholar 

Download references

Funding

This work was supported by the EU Horizon 2020 research and innovation programme under GA No 780073 INBOTS (Inclusive Robotics for a Better Society).

Author information

Authors and Affiliations

  1. University of Siena, Siena, Italy

    Maria Pozzi & Monica Malvezzi

  2. Aarhus University, Aarhus, Denmark

    Unnikrishnan Radhakrishnan, Konstantinos Koumaditis & Francesco Chinello

  3. Spanish Research Council, Madrid, Spain

    Ana Rojo Agustí & Juan C. Moreno

Authors
  1. Maria Pozzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Unnikrishnan Radhakrishnan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ana Rojo Agustí
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Konstantinos Koumaditis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Francesco Chinello
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Juan C. Moreno
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Monica Malvezzi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors equally contributed to the paper.

Corresponding author

Correspondence to Maria Pozzi .

Editor information

Editors and Affiliations

  1. Department of Information Engineering and Mathematics, University of Siena, Siena, Italy

    Monica Malvezzi

  2. European Lab for Educational Technology, Edumotiva, Sparta, Greece

    Dimitris Alimisis

  3. Department of Information Engineering, Università degli Studi di Padova, Padova, Italy

    Michele Moro

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pozzi, M. et al. (2021). Exploiting VR and AR Technologies in Education and Training to Inclusive Robotics. In: Malvezzi, M., Alimisis, D., Moro, M. (eds) Education in & with Robotics to Foster 21st-Century Skills. EDUROBOTICS 2021. Studies in Computational Intelligence, vol 982. Springer, Cham. https://doi.org/10.1007/978-3-030-77022-8_11

Download citation

Publish with us

Policies and ethics

Search

Navigation