Skip to main content
SpringerLink
Log in

Designing User-Guidance for eXtendend Reality Interfaces in Industrial Environments

  • Chapter
  • First Online:
Human-Technology Interaction

  • 744 Accesses

Abstract

Modern industrial environments evolve from collections of physical machines into networked extended reality environments, where in addition to the physical machines digital content becomes increasingly significant. Examples of such digital content include not only the necessary information to control the production, but also digital instructions and product information, information derived from embedded sensors or predictive models, as wells as digital twins of products, machines and entire factories.

This means that user interface designers in such environments have to design interfaces that allow access to and interaction with both the virtual and physical aspects. The design of user interfaces for such eXtendend Reality (XR) environments faces a number of challenges including the change of the implementation platform from a “standardized” PC GUI to I/O devices and paradigms that are suitable for XR. Examples include mobile and wearable devices as well as virtual and augmented reality. There is still a lack of standards and best-practices regarding the design of XR interfaces, especially when interaction and data visualization techniques are required to support work across different platforms.

This chapter focuses on guidance techniques that help users navigate the complex information and interaction tasks in XR environments, as this is an area for which little information is currently available. The chapter explains why different forms of user guidance are required in XR environments, why these pose a challenge for user interface designers, and presents techniques that can be used to aid the design process.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 84.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Dörner, R., Broll, W., Grimm, P., & Jung, B. (2016). Virtual reality und augmented reality (VR/AR). Informatik-Spektrum, 39(1), 30–37. https://doi.org/10.1007/s00287-014-0838-9

    Article  Google Scholar 

  2. Azuma, R. T. (1997). A survey of augmented reality’. Presence: Teleoperators and Virtual Environments, 6(4).

    Google Scholar 

  3. Milgram, P., Takemura, H., Utsumi, A., & Kishino, F. (1995). Augmented reality: A class of displays on the reality-virtuality continuum. In Paper presented at the Telemanipulator and Telepresence Technologies (Vol. 2351, p. 282). IEEE. https://doi.org/10.1117/12.197321

    Chapter  Google Scholar 

  4. Ishii, H., & Ullmer, B. (1998). Tangible bits: Towards seamless interfaces between people, bits and atoms. In Conference on Human Factors in Computing Systems - Proceedings. IEEE. https://doi.org/10.1145/258549.258715

    Chapter  Google Scholar 

  5. Paelke, V., Nebe, K., & Geiger, C. (2012). Multi-modal, multi-touch interaction with maps in disaster management applications. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XXXIX-B8, XXII.

    Google Scholar 

  6. Moran, T. P., & Dourish, P. (2001). Introduction to this special issue on context-aware computing. Null, 16(2–4), 87–95. https://doi.org/10.1207/S15327051HCI16234_01

    Article  Google Scholar 

  7. Caudell, T. P., & Mizell, D. W. (1992). Augmented reality: An application of heads-up display technology to manual manufacturing processes. In Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences (pp. 659–669). ACM. https://doi.org/10.1109/HICSS.1992.183317

    Chapter  Google Scholar 

  8. Darken, R. P. (1995). Wayfinding in large-scale virtual worlds. In Paper presented at the conference companion on human factors in computing systems, Denver, Colorado, USA (pp. 45–46). ACM. https://doi.org/10.1145/223355.223419

    Chapter  Google Scholar 

  9. Boud, A. C., Haniff, D. J., Baber, C., & Steiner, S. J. (1999). Virtual reality and augmented reality as a training tool for assembly tasks. In Paper presented at the 1999 IEEE international conference on information visualization (pp. 32–36). IEEE. https://doi.org/10.1109/IV.1999.781532

    Chapter  Google Scholar 

  10. Reiners, D., Stricker, D., Klinker, G., Müller, S., & Igd, F. (1999, November). Augmented reality for construction tasks: Doorlock assembly. In Proceedings of the first International Workshop on Augmented Reality (IWAR’98), San Francisco (pp. 31–46). A K Peters.

    Google Scholar 

  11. Tang, A., Owen, C., Biocca, F., & Mou, W. (2003). Comparative effectiveness of augmented reality in object assembly. In Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Ft. Lauderdale, Florida, USA, pp. 73–80. https://doi.org/10.1145/642611.642626

  12. Henderson, S., & Feiner, S. (2011). Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Transactions on Visualization and Computer Graphics, 17(10), 1355–1368. https://doi.org/10.1109/TVCG.2010.245

    Article  Google Scholar 

  13. Khan, A., Matejka, J., Fitzmaurice, G., & Kurtenbach, G. (2005). Spotlight: Directing users’ attention on large displays. In Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Portland, Oregon, USA (pp. 791–798). IEEE. https://doi.org/10.1145/1054972.1055082

    Chapter  Google Scholar 

  14. Biocca, F., Owen, C., Tang, A., & Bohil, C. (2007). Attention issues in spatial information systems: Directing mobile users’ visual attention using augmented reality. Journal of Management Information Systems, 23(4), 163–184. https://doi.org/10.2753/MIS0742-1222230408

    Article  Google Scholar 

  15. Baudisch, P., & Rosenholtz, R. (2003). Halo: A technique for visualizing off-screen objects. In Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Ft. Lauderdale, Florida, USA (pp. 481–488). ACM. https://doi.org/10.1145/642611.642695

    Chapter  Google Scholar 

  16. Renner, P., & Pfeiffer, T. (2017). Attention guiding techniques using peripheral vision and eye tracking for feedback in augmented-reality-based assistance systems. In Paper presented at the 2017 IEEE Symposium on 3D User Interfaces (3DUI) (pp. 186–194). IEEE. https://doi.org/10.1109/3DUI.2017.7893338

    Chapter  Google Scholar 

  17. Renner, P., Blattgerste, J., & Pfeiffer, T. (2018). A path-based attention guiding technique for assembly environments with target occlusions. In Paper presented at the 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (pp. 671–672). IEEE. https://doi.org/10.1109/VR.2018.8446127

    Chapter  Google Scholar 

  18. Radkowski, R., Herrema, J., & Oliver, J. H. (2015). Augmented reality-based manual assembly support with visual features for different degrees of difficulty. International Journal of Human-Computer Interaction, 31, 337–349.

    Article  Google Scholar 

  19. Imbert, J., Hodgetts, H. M., Parise, R., Vachon, F., Dehais, F., & Tremblay, S. (2014). Attentional costs and failures in air traffic control notifications. Ergonomics, 57(12), 1817–1832. https://doi.org/10.1080/00140139.2014.952680

    Article  Google Scholar 

  20. Blattgerste, J., Strenge, B., Renner, P., Pfeiffer, T., & Essig, K. (2017). Comparing conventional and augmented reality instructions for manual assembly tasks. In Proceedings of PETRA’17 (pp. 75–82). Association for Computing Machinery. https://doi.org/10.1145/3056540.3056547

    Chapter  Google Scholar 

  21. Büttner, S., Funk, M., Sand, O., & Röcker, C. (2016). Using head-mounted displays and in-situ projection for assistive systems: A comparison. In Proceedings of 9th ACM international conference on pervasive technologies related to assistive environments (p. 44). ACM.

    Google Scholar 

  22. Funk, M., Kosch, T., & Schmidt, A. (2016). Interactive worker assistance: Comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions. In Proceedings of 2016 ACM international joint conference on pervasive and ubiquitous computing (Vol. 2016, pp. 934–939). ACM.

    Chapter  Google Scholar 

  23. Korn, O., Schmidt, A., & Hörz, T. (2013). The potentials of in-situ-projection for augmented workplaces in production: A study with impaired persons. In Extended Abstracts CHI’13 (pp. 979–984). Association for Computing Machinery. https://doi.org/10.1145/2468356.2468531

    Chapter  Google Scholar 

  24. Hein, P., Bernhagen, M., & Bullinger, A. C. (2019). Improving visual attention guiding by differentiation between fine and coarse navigation. In Paper presented at the 11th International Conference on Virtual Worlds and Games for Serious Applications, VS-Games 2019, Vienna, Austria, September 4–6, 2019 (pp. 1–4). ACM. https://doi.org/10.1109/VS-Games.2019.8864539

    Chapter  Google Scholar 

  25. Renner, P., & Pfeiffer, T. (2020). AR-glasses-based attention guiding for complex environments: Requirements, classification and evaluation. In Paper presented at the Proceedings of the 13th ACM International Conference on Pervasive Technologies Related to Assistive Environments, Corfu, Greece. IEEE. https://doi.org/10.1145/3389189.3389198

    Chapter  Google Scholar 

  26. Rothe, S., Buschek, D., & Hußmann, H. (2019). Guidance in cinematic virtual reality-taxonomy, research status and challenges. Multimodal Technologies and Interaction, 3(1). https://doi.org/10.3390/mti3010019

  27. Büttner, S., Mucha, H., Funk, M., Kosch, T., Aehnelt, M., Robert, S., & Röcker, C. (2017). The design space of augmented and virtual reality applications for assistive environments in manufacturing: A visual approach. In Proceedings of 10th International conference on pervasive technologies related to assistive environments (pp. 433–440). ACM.

    Chapter  Google Scholar 

  28. Funk, M., Kosch, T., Greenwald, S. W., & Schmidt, A. (2015). A benchmark for interactive augmented reality instructions for assembly tasks. In Proceedings of 14th International Conference on Mobile and Ubiquitous Multimedia (MUM’15) (pp. 253–257). Association for Computing Machinery. https://doi.org/10.1145/2836041.2836067

    Chapter  Google Scholar 

  29. Hou, L., Wang, X., & Truijens, M. (2015). Using augmented reality to facilitate piping assembly: An experiment-based evaluation. Journal of Computing in Civil Engineering, 29, 1. (2015). https://doi.org/10.1061/(ASCE)CP.1943-5487.0000344

    Article  Google Scholar 

  30. Blattner, J., Wolfartsberger, J., Lindorfer, R., Froschauer, R., Pimminger, S., & Kurschl, W. (2021). A standardized approach to evaluate assistive systems for manual assembly tasks in industry. In Proceedings of Conference on Learning Factories (CLF). ACM. https://doi.org/10.2139/ssrn.3858632

    Chapter  Google Scholar 

  31. Illing, J., Klinke, P., Grünefeld, U., Pfingsthorn, M., & Heuten, W. (2020). Time is money! Evaluating augmented reality instructions for time-critical assembly tasks. In 19th international conference on mobile and ubiquitous multimedia. ACM.

    Google Scholar 

  32. Büttner, S., Prilla, M., & Röcker, C. (2020). Augmented reality training for industrial assembly work - Are projection-based AR assistive systems an appropriate tool for assembly training? In Proceedings of CHI’20 (pp. 1–12). Association for Computing Machinery. https://doi.org/10.1145/3313831.3376720

    Chapter  Google Scholar 

  33. Furmanski, C., Azuma, R., & Daily, M. (2002). Augmented-reality visualizations guided by cognition: Perceptual heuristics for combining visible and obscured information. In Paper presented at the Proceedings. International Symposium on Mixed and Augmented Reality (pp. 215–320). ACM. https://doi.org/10.1109/ISMAR.2002.1115091

    Chapter  Google Scholar 

  34. Kruijff, E., Swan, J. E., & Feiner, S. (2010). Perceptual issues in augmented reality revisited. In Paper presented at the 2010 IEEE international symposium on mixed and augmented reality (pp. 3–12). IEEE. https://doi.org/10.1109/ISMAR.2010.5643530

    Chapter  Google Scholar 

  35. Roda, C. (2010). Attention support in digital environments. nine questions to be addressed. New Ideas in Psychology, 28(3), 354–364. https://doi.org/10.1016/j.newideapsych.2009.09.010

    Article  Google Scholar 

  36. Hata, H., Koike, H., & Sato, Y. (2016). Visual guidance with unnoticed blur effect. In Paper presented at the Proceedings of the International Working Conference on Advanced Visual Interfaces, Bari, Italy (pp. 28–35). ACM. https://doi.org/10.1145/2909132.2909254

    Chapter  Google Scholar 

  37. Kalkofen, D., Sandor, C., White, S., & Schmalstieg, D. (2011). Visualization techniques for augmented reality (pp. 65–98). IEEE. https://doi.org/10.1007/978-1-4614-0064-6_3

    Book  Google Scholar 

  38. Schwaber, K. & Sutherland, J. (2020). The scrum guide. Retrieved November 9, 2021, from https://scrumguides.org/docs/scrumguide/v2020/2020-Scrum-Guide-US.pdf

  39. ISO 9241-210:2019(en). Retrieved from https://www.iso.org/standard/77520.html

  40. Dhiman, H., Martinez, S., Paelke, V., & Röcker, C. (2018). Head-mounted displays in industrial AR-applications: Ready for prime time? In Proceedings of HCI International 2018, Las Vegas, NV, USA. ACM. https://doi.org/10.1007/978-3-319-91716-0\_6

    Chapter  Google Scholar 

  41. IEEE Xplore. Retrieved from https://ieeexplore.ieee.org/

  42. Oculus developers guide. Retrieved from https://developer.oculus.com/learn/

  43. Microsoft guidelines for hololens AR systems. Retrieved from https://docs.microsoft.com/en-us/windows/mixed-reality/design/app-patterns-landingpage

  44. Nielsen, J. (1989). Usability engineering at a discount. In Paper presented at the proceedings of the third international conference on human-computer interaction on designing and using human-computer interfaces and knowledge based systems (2nd ed., pp. 394–401). IEEE.

    Google Scholar 

  45. Fischertechnik. Retrieved from https://www.fischertechnik.de/de-de/produkte

  46. Lego. Retrieved from https://www.lego.com/

  47. Makeblock. Retrieved from https://www.makeblock.com/

  48. Paelke, V., Röcker, C., & Bulk, J. (2019). A test platform for the evaluation of augmented reality head mounted displays in industrial applications. In Proceedings of advances in manufacturing, production management and process (pp. 25–35). IEEE.

    Google Scholar 

  49. Glassner, A. (1990). In A. S. Glassner (Ed.), Graphics gems. Academic Press.

    MATH  Google Scholar 

  50. UI pattern library. Retrieved from http://ui-patterns.com/

Download references

Author information

Authors and Affiliations

  1. Hochschule Bremen, Human-Computer-Interaction, ZIMT, Bremen, Germany

    Volker Paelke & Jendrik Bulk

Authors
  1. Volker Paelke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jendrik Bulk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Volker Paelke .

Editor information

Editors and Affiliations

  1. Institute Industrial IT, OWL University of Applied Sciences and Arts, Lemgo, Germany

    Carsten Röcker

  2. Institute Industrial IT, OWL University of Applied Sciences and Arts, Lemgo, Germany, Human-Centered Information Systems, Clausthal University of Technology, Clausthal-Zellerfeld, Germany

    Sebastian Büttner

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Paelke, V., Bulk, J. (2023). Designing User-Guidance for eXtendend Reality Interfaces in Industrial Environments. In: Röcker, C., Büttner, S. (eds) Human-Technology Interaction. Springer, Cham. https://doi.org/10.1007/978-3-030-99235-4_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-99235-4_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-99234-7

  • Online ISBN: 978-3-030-99235-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation