Skip to main content
SpringerLink
Log in

A Domain-Specific Visual Modeling Language for Augmented Reality Applications Using WebXR

  • Conference paper
  • First Online:
Conceptual Modeling (ER 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14320))

Included in the following conference series:

  • 548 Accesses

Abstract

Augmented reality (AR) is a technology that overlays digital information onto real-world objects using devices like smartphones, tablets, or head-mounted displays to enrich human comprehension and interaction with the physical environment. The creation of AR software applications requires today advanced coding skills, particularly when aiming to realize complex, multifaceted scenarios. As an alternative, we propose a domain-specific visual modeling language for designing AR scenarios, enabling users to define augmentations and AR workflows graphically. The language has been implemented on the ADOxx metamodeling platform, together with a software engine for running the AR applications using the W3C WebXR Device API for web-based augmented reality. The language and the AR application are demonstrated through a furniture assembly use case. In an initial evaluation, we show, via a comprehensive feature comparison, that the proposed language exhibits a more extensive coverage of AR concepts compared to preceding model-based approaches.

Financial support is gratefully acknowledged by the Smart Living Lab funded by the University of Fribourg, EPFL, and HEIA-FR.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://library.vuforia.com/.

  2. 2.

    https://developer.apple.com/augmented-reality/arkit/.

  3. 3.

    https://developers.google.com/ar.

  4. 4.

    https://github.com/Microsoft/MixedRealityToolkit-Unity.

  5. 5.

    https://www.unitear.com/.

  6. 6.

    https://adobe.com/products/aero.html.

  7. 7.

    https://caniuse.com/webxr.

  8. 8.

    https://microsoft.com/en-us/hololens.

  9. 9.

    https://github.com/WebKit/WebKit/tree/main/Source/WebCore/Modules/webxr.

  10. 10.

    https://www.apple.com/apple-vision-pro/.

  11. 11.

    https://github.com/WEKIT-ECS/MIRAGE-XR.

  12. 12.

    http://docs.opengeospatial.org/is/12-132r4/12-132r4.html.

  13. 13.

    https://github.com/spi-fm/ARE4DSL.

  14. 14.

    https://alter-ar.github.io/.

  15. 15.

    https://github.com/mrdoob/three.js/.

  16. 16.

    https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html.

References

  1. IKEA Online Shop (2023). https://www.ikea.com/us/en/p/knarrevik-nightstand-black-30381183/. Accessed 28 Apr 2023

  2. Azuma, R.T.: A survey of augmented reality. Presence Teleoperators Virtual Environ. 6(4), 355–385 (1997)

    Article  Google Scholar 

  3. Bock, A.C., Frank, U.: Low-code platform. Bus. Inf. Syst. Eng. 63(6), 733–740 (2021). https://doi.org/10.1007/s12599-021-00726-8

    Article  Google Scholar 

  4. Bork, D., Roelens, B.: A technique for evaluating and improving the semantic transparency of modeling language notations. Softw. Syst. Model. 20(4), 939–963 (2021). https://doi.org/10.1007/s10270-021-00895-w

    Article  Google Scholar 

  5. Brunschwig, L., Campos-Lopez, R., Guerra, E., de Lara, J.: Towards domain-specific modelling environments based on augmented reality. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), Madrid, ES, pp. 56–60. IEEE (2021). https://doi.org/10.1109/ICSE-NIER52604.2021.00020

  6. Campos-López, R., Guerra, E., de Lara, J.: Towards automating the construction of augmented reality interfaces for information systems. In: Insfrán, E., et al. (eds.) Information Systems Development: Crossing Boundaries Between Development and Operations (DevOps) in Information Systems (ISD2021 Proceedings), Valencia, Spain, 8–10 September 2021. Universitat Politècnica de València/Association for Information Systems (2021). https://aisel.aisnet.org/isd2014/proceedings2021/hci/6

  7. Dalton, J., Gillham, J.: Seeing is believing (2019). https://www.pwc.com/gx/en/industries/technology/publications/economic-impact-of-vr-ar.html. Accessed 09 Mar 2023

  8. Di Ruscio, D., Kolovos, D., de Lara, J., Pierantonio, A., Tisi, M., Wimmer, M.: Low-code development and model-driven engineering: two sides of the same coin? Softw. Syst. Model. 21(2), 437–446 (2022). https://doi.org/10.1007/s10270-021-00970-2

    Article  Google Scholar 

  9. Doerner, R., Broll, W., Grimm, P., Jung, B. (eds.): Virtual and Augmented Reality (VR/AR). Springer, Cham (2022). https://doi.org/10.1007/978-3-030-79062-2

    Book  Google Scholar 

  10. Fill, H.-G., Härer, F., Muff, F., Curty, S.: Towards augmented enterprise models as low-code interfaces to digital systems. In: Shishkov, B. (ed.) BMSD 2021. LNBIP, vol. 422, pp. 343–352. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-79976-2_22

    Chapter  Google Scholar 

  11. Fill, H., Karagiannis, D.: On the conceptualisation of modelling methods using the ADOxx meta modelling platform. Enterp. Model. Inf. Syst. Archit. Int. J. Concept. Model. 8(1), 4–25 (2013). https://doi.org/10.18417/emisa.8.1.1

  12. Fill, H.-G., Redmond, T., Karagiannis, D.: Formalizing meta models with FDMM: the ADOxx case. In: Cordeiro, J., Maciaszek, L.A., Filipe, J. (eds.) ICEIS 2012. LNBIP, vol. 141, pp. 429–451. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40654-6_26

    Chapter  Google Scholar 

  13. Frank, U.: Domain-specific modeling languages: requirements analysis and design guidelines. In: Reinhartz-Berger, I., Sturm, A., Clark, T., Cohen, S., Bettin, J. (eds.) Domain Engineering, Product Lines, Languages, and Conceptual Models, pp. 133–157. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36654-3_6

    Chapter  Google Scholar 

  14. Götzinger, D., Miron, E.-T., Staffel, F.: OMiLAB: an open collaborative environment for modeling method engineering. In: Domain-Specific Conceptual Modeling, pp. 55–76. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39417-6_3

    Chapter  Google Scholar 

  15. Grambow, G., Hieber, D., Oberhauser, R., Pogolski, C.: A context and augmented reality BPMN and BPMS extension for industrial Internet of Things processes. In: Marrella, A., Weber, B. (eds.) BPM 2021. LNBIP, vol. 436, pp. 379–390. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-94343-1_29

    Chapter  Google Scholar 

  16. Gulden, J., Yu, E.: Toward requirements-driven design of visual modeling languages. In: Buchmann, R.A., Karagiannis, D., Kirikova, M. (eds.) PoEM 2018. LNBIP, vol. 335, pp. 21–36. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02302-7_2

    Chapter  Google Scholar 

  17. Jannaber, S., Riehle, D.M., Delfmann, P., Thomas, O., Becker, J.: Designing a framework for the development of domain-specific process modelling languages. In: Maedche, A., vom Brocke, J., Hevner, A. (eds.) DESRIST 2017. LNCS, vol. 10243, pp. 39–54. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59144-5_3

    Chapter  Google Scholar 

  18. Jones, B., Goregaokar, M., Cabanier, R.: WebXR device API. W3C candidate recommendation draft, work in progress, World Wide Web Consortium (2023). https://www.w3.org/TR/2023/CRD-webxr-20230303/

  19. Karagiannis, D., Mayr, H.C., Mylopoulos, J. (eds.): Domain-Specific Conceptual Modeling, Concepts, Methods and Tools. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39417-6

    Book  Google Scholar 

  20. Karsai, G., Krahn, H., Pinkernell, C., Rumpe, B., Schneider, M., Völkel, S.: Design guidelines for domain specific languages. In: Rossi, M., Sprinkle, J., Gray, J., Tolvanen, J.P. (eds.) Proceedings of the 9th OOPSLA Workshop on Domain-Specific Modeling (DSM 2009), Orlanda, vol. B-108, pp. 7–13. Helsingin Kauppakorkeakoulu (2009)

    Google Scholar 

  21. Lechner, M.: ARML 2.0 in the context of existing AR data formats. In: Latoschik, M.E., Reiners, D., Blach, R., Figueroa, P.A., Wingrave, C.A. (eds.) 6th Workshop on Software Engineering and Architectures for Realtime Interactive Systems, SEARIS 2013, Orlando, FL, USA, 17 March 2013, pp. 41–47. IEEE Computer Society (2013). https://doi.org/10.1109/SEARIS.2013.6798107

  22. Mernik, M., Heering, J., Sloane, A.M.: When and how to develop domain-specific languages. ACM Comput. Surv. 37(4), 316–344 (2005). https://doi.org/10.1145/1118890.1118892

    Article  Google Scholar 

  23. Moody, D.L.: The physics of notations: toward a scientific basis for constructing visual notations in software engineering. IEEE Trans. Softw. Eng. 35(6), 756–779 (2009). https://doi.org/10.1109/TSE.2009.67

    Article  Google Scholar 

  24. Muff, F., Fill, H.: Initial concepts for augmented and virtual reality-based enterprise modeling. In: Lukyanenko, R., Samuel, B.M., Sturm, A. (eds.) Proceedings of the ER Demos and Posters 2021 Co-Located with 40th International Conference on Conceptual Modeling (ER 2021), St. John’s, NL, Canada, 18–21 October 2021. CEUR Workshop Proceedings, vol. 2958, pp. 49–54. CEUR-WS.org (2021). https://ceur-ws.org/Vol-2958/paper9.pdf

  25. Muff, F., Fill, H.: ADOxx Library and UseCase Models for the ER23 Publication: A Domain-Specific Visual Modeling Language for Augmented Reality Applications Using WebXR (2023). https://doi.org/10.5281/zenodo.8207639

  26. Muff, F., Fill, H.: Past achievements and future opportunities in combining conceptual modeling with VR/AR: a systematic derivation. In: Shishkov, B. (ed.) BMSD 2023. LNBIP, vol. 483, pp. 129–144. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-36757-1_8

    Chapter  Google Scholar 

  27. Nguyen, T.: 4 impactful technologies from the gartner emerging technologies and trends impact radar for 2021 (2021). https://www.gartner.com/smarterwithgartner/4-impactful-technologies-from-the-gartner-emerging-technologies-and-trends-impact-radar-for-2021. Accessed 09 Mar 2023

  28. Roo, J.S., Hachet, M.: One reality: augmenting how the physical world is experienced by combining multiple mixed reality modalities. In: Gajos, K., Mankoff, J., Harrison, C. (eds.) Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, UIST 2017, Quebec City, QC, Canada, 22–25 October 2017, pp. 787–795. ACM (2017). https://doi.org/10.1145/3126594.3126638

  29. Ruiz-Rube, I., Baena-Pérez, R., Mota, J.M., Sánchez, I.A.: Model-driven development of augmented reality-based editors for domain specific languages. IxD &A 45, 246–263 (2020). https://doi.org/10.55612/s-5002-045-011

  30. Ruminski, D., Walczak, K.: Dynamic composition of interactive AR scenes with the carl language. In: Bourbakis, N.G., Tsihrintzis, G.A., Virvou, M. (eds.) IISA 2014, The 5th International Conference on Information, Intelligence, Systems and Applications, Chania, Crete, Greece, pp. 329–334. IEEE (2014). https://doi.org/10.1109/IISA.2014.6878808

  31. Saxena, D., Verma, J.K.: Recreating reality: classification of computer-assisted environments. In: Verma, J.K., Paul, S. (eds.) Advances in Augmented Reality and Virtual Reality. SCI, vol. 998, pp. 3–9. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-7220-0_1

    Chapter  Google Scholar 

  32. Schmalstieg, D.: Augmented Reality: Principles and Practice. Addison-Wesley, Boston (2016)

    Google Scholar 

  33. Seiger, R., Kühn, R., Korzetz, M., Aßmann, U.: HoloFlows: modelling of processes for the Internet of Things in mixed reality. Softw. Syst. Model. 20(5), 1465–1489 (2021). https://doi.org/10.1007/s10270-020-00859-6

    Article  Google Scholar 

  34. Siau, K., Rossi, M.: Evaluation techniques for systems analysis and design modelling methods - a review and comparative analysis. Inf. Syst. J. 21(3), 249–268 (2011). https://doi.org/10.1111/j.1365-2575.2007.00255.x

    Article  Google Scholar 

  35. Visic, N., Fill, H., Buchmann, R.A., Karagiannis, D.: A domain-specific language for modeling method definition: from requirements to grammar. In: 9th IEEE International Conference on Research Challenges in Information Science, RCIS 2015, Athens, Greece, 13–15 May 2015, pp. 286–297. IEEE (2015). https://doi.org/10.1109/RCIS.2015.7128889

  36. Wild, F., Perey, C., Hensen, B., Klamma, R.: IEEE standard for augmented reality learning experience models. In: Mitsuhara, H., et al. (eds.) IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2020, Takamatsu, Japan, 8–11 December 2020, pp. 1–3. IEEE (2020). https://doi.org/10.1109/TALE48869.2020.9368405

  37. Wild, F., et al.: Towards data exchange formats for learning experiences in manufacturing workplaces. In: Kravcik, M., Mikroyannidis, A., Pammer, V., Prilla, M., Ullmann, T.D., Wild, F. (eds.) Proceedings of the 4th Workshop on Awareness and Reflection in Technology-Enhanced Learning in conjunction with the 9th European Conference on Technology Enhanced Learning: Open Learning and Teaching in Educational Communities, ARTEL@EC-TEL 2014, Graz, Austria, 16 September 2014. CEUR Workshop Proceedings, vol. 1238, pp. 23–33. CEUR-WS.org (2014). http://ceur-ws.org/Vol-1238/paper2.pdf

  38. Yin, K., He, Z., Xiong, J., Zou, J., Li, K., Wu, S.T.: Virtual reality and augmented reality displays: advances and future perspectives. J. Phys. Photonics 3(2), 022010 (2021). https://doi.org/10.1088/2515-7647/abf02e

    Article  Google Scholar 

  39. Zhou, F., Duh, H.B., Billinghurst, M.: Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR. In: 7th IEEE and ACM International Symposium on Mixed and Augmented Reality, ISMAR 2008, Cambridge, UK, 15–18 September 2008, pp. 193–202. IEEE Computer Society (2008). https://doi.org/10.1109/ISMAR.2008.4637362

Download references

Author information

Authors and Affiliations

  1. Research Group Digitalization and Information Systems, University of Fribourg, Fribourg, Switzerland

    Fabian Muff & Hans-Georg Fill

Authors
  1. Fabian Muff
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hans-Georg Fill
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fabian Muff .

Editor information

Editors and Affiliations

  1. Federal University of Espírito Santo, Vitória, Brazil

    João Paulo A. Almeida

  2. Universidade de Lisboa, Lisbon, Portugal

    José Borbinha

  3. University of Twente, Enschede, The Netherlands

    Giancarlo Guizzardi

  4. University of Auckland, Auckland, New Zealand

    Sebastian Link

  5. Stockholm University, Kista, Sweden

    Jelena Zdravkovic

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Muff, F., Fill, HG. (2023). A Domain-Specific Visual Modeling Language for Augmented Reality Applications Using WebXR. In: Almeida, J.P.A., Borbinha, J., Guizzardi, G., Link, S., Zdravkovic, J. (eds) Conceptual Modeling. ER 2023. Lecture Notes in Computer Science, vol 14320. Springer, Cham. https://doi.org/10.1007/978-3-031-47262-6_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47262-6_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47261-9

  • Online ISBN: 978-3-031-47262-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation