Combining Large Language Models and Immersive Technologies to Represent Cultural Heritage in the Metaverse Context

Abstract

This chapter delves into the intersection of large language models, immersive technologies, and cultural heritage representation in the metaverse. Advancements in natural language processing (NLP) and deep learning enable immersive learning experiences using extended reality (XR) to teach global cultural heritage. Specifically, we propose a model that integrates generative AI, NLP, and XR, incorporating multi-sensory feedback with haptics and olfactory virtual reality (OVR) to engage users in a dialogical relationship with diverse cultures and challenge postcolonial narratives. We explore the potential of cultural heritage to resurrect famous historical personalities and overlooked indigenous peoples using generative AI and metahumans. Use cases in art history are presented, highlighting scaffolded experiences in virtual learning environments (VLEs) for deeper engagement with historical figures and events. Additionally, we address recent safety concerns and limitations of large language models that may inadvertently compromise historical veracity. Ethical implications of misrepresenting historical viewpoints are discussed, emphasizing the need for expert collaboration to ensure historical accuracy and appropriateness. The chapter also elucidates issues of ownership, representation, and cultural appropriation in the context of cultural heritage. It underscores the potential of combining large language models and immersive technologies to offer captivating and educational cultural heritage experiences.