Preface to the UMUAI special issue on personalized delivery of cultural heritage content: perspectives on 7 years of progress in the field
Digital cultural heritage is now a mature field, in which novel information and communication technologies (ICT) are used in the service of preserving cultural heritage and supporting its discovery by the public. Following the extensive digitization efforts conducted since the mid 1990s, and the many uses of supporting technologies that followed since the 2000s in education and tourism, we found ourselves in an era where state-of-the-art ICT enables the delivery of personalized content to visitors of cultural heritage sites in a context-aware manner. Moreover, these new developments redefine the way cultural heritage information can be managed, stored and delivered to visitors and it fits well into the new field of Digital Humanities.
From the point of view of the Humanities, we see new approaches to the examination and presentation of culture and history, including greater emphasis in developing storytelling strategies with which to stimulate reflection, curiosity and even reinterpretation (Wong 2015). From the point of view of ICT, we have unprecedented reasoning and analysis power via solutions such as deep learning, developments in smart interconnected devices that revolutionize the notion of space and the way we interact with it, and advances in Augmented Reality and Virtual Reality technologies that open new ways to the delivery of content. Finally, from the point of view of cooperation between Humanities and ICT, we are in the era when the boundaries are starting to fall, bridges are being formed and true, deep, robust synergies appear, leading to new opportunities and perspectives.
Perhaps most stimulating amongst these developments—and certainly for us and the audience of this journal—is the progress achieved and the further potential uncovered in the direction of personalization of the cultural heritage experience. The amounts of information in the hands of the cultural heritage venues, the many stories that can be told thereof, and the possibility of retrieving additional contents from online repositories (e.g. Wikipedia, Europeana and other Linked Open Data resources) and social networks, altogether yield a landscape in which it is strategically important to deliver information in the way that best matches the interests, preferences, needs and context of each individual user (a student, a museum visitor or a tourist in a city) or each group ( a class, a family, a community of interest, a group of friends or the crowd in general). In doing so, the role of Humanities experts as mediators between the knowledge bases and the users’ end devices is also magnified, since they are the key actors in ensuring that the ensembles of contents presented to the audiences are meaningful, historically rigorous and relevant. Accordingly, the experts must be given proper tools not only to develop catalogues and knowledge repositories for cultural heritage venues, but also to curate the interconnections among these and with the aforementioned online repositories. Those tools can also become the medium to systematize the creation of stories as drivers of the users’ experiences.
The state-of-the-art in personalization technologies applied to cultural heritage has made steady progress over the years, as presented by the survey by Ardissono et al. (2012) and continued since. Given the rich history, the number of museums and sites, and the unmatched research pedigree, Europe was and remains the leader in the application of cultural heritage informatics and in the research pushing the field forward. Tellingly, 2018 was designated as the European Year of Cultural Heritage, underlining the European Union’s intent to further push forward in this domain. Following suit, we have spent this year editing this Special Issue, inviting works that presented and/or reviewed advances in areas of data analytics, semantics, information retrieval, user interaction, user modeling, recommender systems and related technologies, as well as in the design and evaluation of new, exciting and stimulating cultural heritage experiences.
Three papers (by Alexandridis et al., Raptis et al., and Sansonetti et al.) cover advances in recommender systems for cultural heritage activities, coupled with ideas in implementing context and topic sensitiveness, modeling cognition-related phenomena and enriching the recommendations by harnessing information from social networks and Linked Open Data resources (namely DBPedia and Europeana).
The paper by Not and Petrelli focuses on the tool support needed by curators, educators, guides and designers to configure activities, objectives, contents, \(\ldots \) and therefrom deliver adaptive narratives, grounded on an extensive study of previous ideas and of the work practices, ambitions and expectations of those experts. Such tools are also the subject of the paper by Raptis et al., with a proposal that aims to tailor different types of cultural heritage activities to the cognitive characteristics of the end users.
Finally, the paper by Rajaonarivo et al. presents the concept of a “3D living museum” metaphor, with interesting innovations related not only to the user interactions (immersion, embodiment) but also to the semantic reasoning technologies that underpin the selection of personalized contents.
The second observation from the survey conducted by Ardissono et al. (“Before, during and after the visit”) highlighted that, by 2012, it was already commonplace that researchers viewed the cultural heritage experiences as ongoing ones, thus caring about capturing and retaining user preferences over time, both online and onsite (i.e. at the museum, archaeological site or relevant outdoor setting). The papers included in this special issue address an enabling aspect: whilst location is the main personalization axis during a visit, adaptive storytelling (as posed in the paper by Not and Petrelli) is probably the key ingredient to ensure a meaningful interaction continuum from the moment that a user expresses interest in visiting a cultural or historical place, and also after the visit to fully integrate information, observations and personal experience. The persistence of the user profiles could also realize the vision of a system that delivers successive related narratives that link together different cultural heritage experiences—most likely, at different places—in a lifelong chain. This would face issues of privacy, data ownership and user control that are seldom mentioned in journal articles; however, as discussed tangentially in the papers by Raptis et al. and Sansonetti et al., they must be part of any experiments and deployments due to the development of data protection regulations.
The third highlight by Ardissono et al. (“Not me but us: Social aspects”) referred to two main aspects. On the one hand, there was the fact that visiting a cultural venue is many times a social experience, so the personalization engines should be able to deal not only with individual user profiles, but also group profiles. On the other hand, the survey echoed the growing phenomenon of online social networks, that museums and cultural heritage institutions were using actively to support the sharing of experiences among visitors and the publication of user-generated material, aiming to establish long-term relationships with people. The trends about group personalization and social networks have indeed intensified since 2012, both in research and practice in all the areas of application of personalization systems. Thus, we have seen numerous variations on the idea of how to build and manage group profiles (Delic et al. 2016) and how to feed profiles from information posted on Facebook, Twitter and other sites (Piao and Breslin 2018)—it must be noted that standardization is still to be achieved in these approaches, too. In this special issue, the paper by Sansonetti et al. develops and evaluates a system that enhances cultural recommendations through social media and Linked Open Data resources, that we consider a good representative of what can be achieved today, harnessing the new tools provided by graph databases such as Neo4j to bring together user profiles, social relationships, semantic knowledge about exhibits, \(\ldots \) into a unique formalism. Additionally, the paper by Not and Petrelli looks at the modeling needs to create adaptive narratives for different target user groups, as well as the tool support expected by the groups of professionals involved, which is a notion of “group” that has received relatively little attention hitherto.
By 2012, Ardissono et al. noticed that, with just a few exceptions, the delivery mode in personalized cultural heritage experiences had been based on the metaphor of hypermedia with content organized in pages and links, followed by multimedia presentations on mobile devices. Under the observation “Beyond hypermedia and multimedia” they foresaw that new possibilities would emerge as technology evolves, which has been particularly true in the last few years thanks to the noticeable increase in the computing, visualization and interaction capabilities of smart mobile devices. Augmented Reality and Virtual Reality are no longer a realm for fancy but awkward prototypes, and a number of systems have been presented and deployed for different purposes, including education, exhibition enhancement, exploration, reconstruction and virtual museums (Chatzopoulos et al. 2017; Bekele et al. 2018). The new concept of micro-augmentations (Antoniou et al. 2015) arose in the meantime as a way to use minimum stimulation (e.g. via short audio effects or vibrations) in carefully-selected locations or points of a narrative, showing potential to achieve significant emotional response and increase visitor engagement. Nevertheless, as argued by Čopič Pucihar et al. (2018), there is a general lack of support for personalization and communication activities in these “augmented” systems, failing to enable features such as context-aware bookmarking, curation of augmentations for the exhibits, new forms of body interaction for embodiment, etc. Besides, the quality of support for analytical activity is low, since the applications often fail to show interesting information that is there but cannot be seen by the naked eye. In sum, it may be said that the limiting factor nowadays is not in the devices, but in the human–content interaction aspects. The prototype presented in this special issue by Rajaonarivo et al. brings a step forward in this direction, by developing the metaphor of a 3D “living museum” with the capability of recommending cultural heritage objects dynamically, according to a user profile that is computed online from the user’s interactions. The paper is grounded on the paradigms of constructivism and enaction, assuming circular dependency between the the sense making of the museum visitors and the selection and the arrangement of the exhibits.
The final highlight in the 2012 survey (“Evaluation: A serious issue”) relates to our decision of prioritizing for this special issue the papers that contained extensive experimental evaluation. Ardissono et al. noted that, as for all other aspects of personalization research (particularly, in the cultural heritage realm), the system and user evaluation would benefit from a common effort and the creation of a shared understanding of which new systems should be evaluated, and how. They noticed that a shared, structured evaluation framework was missing, that would support the comparison of different solutions, the emergence of best practices and the reuse of components, thus leading to faster progress in the field. They also highlighted common methodological flaws, such as lack of objectivity (as when comparing the adaptive system with a static counterpart not optimized for normal interaction), lack of distinction between usability and adaptivity issues, or incorrect sampling of participants. As explained in a 2017 survey by Koutsabasis (2017), some flaws have worsened in the last few years as the purpose and value of many cultural heritage experiences (in relation to exploration, reflection, learning and the like) became secondary to the mastering of impressive interactive technologies of AR/VR, kinesthetic control, etc. In addition, whilst the dimensions of evaluation have been extending to include different notions of effectiveness, engagement, fun or attractiveness (just to name a few), questions related to the “cultural value” conveyed remain largely unanswered: “Do the interactive systems help users reflect on the heritage of places, people and activities and draw analogies with today?”, “Does interactive technology contribute to the sensitization of users about the value of preserving and learning about heritage?”, “Do these systems help visitors interpret the messages conveyed from old situations into today’s needs?”. Admittedly, these questions are particularly hard to answer in short-term and small-to-medium scale evaluation approaches because the “cultural value” still gives way to traditional metrics related to the accuracy of the recommender systems (in the papers by Alexandridis et al. and Sansonetti et al.), to the outcomes of usability polls (in the papers by Rajaonarivo et al. and Not and Petrelli) and to indirect metrics derived from eye-tracking (in the paper by Raptis et al.). Notwithstanding, it is important to note that researchers are increasingly aware of the methodological caveats, and that such good practices as making the research data available in public datasets are becoming mainstream, thus paving the road for comparative, more objective and insightful studies.
Overall, we see that the five open directions identified 7 years ago are still the focus of the field. Gains have been made in all five directions, more in some and less in others, but all still have unanswered challenges and new opportunities. We believe the selection of papers in this special issue provides a good overview of the state-of-the-art of personalization in cultural heritage, representing the progress attained over the period, providing new fascinating ideas to keep working on, and evidencing the many research and development opportunities that remain in this field. We hope that the traditional gap between the involved disciplines continues to shrink for the sake of the “cultural value”, and from our side we intend to keep working in this direction in many ways, including the continued organization of focused exchange meeting such as PATCH (soon to be organized for the 10th time), SMAP (now in its 14th edition), Cultural Informatics (moving on its 3rd edition) and so on.
We are thankful to the many reviewers that shared their expertise in areas of ICT and Humanities, and to the authors that trusted us with their works.
Enjoy the read!
- Antoniou, A., O’Brien, J., Bardon, T., Barnes, A., Virk, D.: Micro-augmentations: situated calibration of a novel non-tactile, peripheral museum technology. In: Proceedings of the 19th Panhellenic Conference on Informatics (PCI), Athens, Greece (2015)Google Scholar
- Bekele, M.K., Pierdicca, R., Frontoni, E., Malinverni, E.S., Gain, J.: A survey of augmented, virtual, and mixed reality for cultural heritage. ACM J. Comput. Cult. Herit. 11(7), 1–36 (2018)Google Scholar
- Čopič Pucihar, K., Kljun, M.: ART for art: augmented reality taxonomy for art and cultural heritage. In: Augmented Reality Art, Springer Series on Cultural Computing (2018)Google Scholar
- Delic, A., Neidhardt, J., Ricci, F.: Research methods for group recommender systems. In: Proceedings of Workshop on Recommenders in Tourism (RecTour), in Conjunction with 10th ACM Conference on Recommender Systems (RecSys), Boston, MA, USA (2016)Google Scholar
- Ekstrand, M.D.: The LKPY package for recommender systems experiments: next-generation tools and lessons learned from the LensKit project (2018). arXiv:1809.03125
- Guo, G., Zhang, J., Sun, Z., Yorke-Smith, N.: LibRec: a java library for recommender systems. In: Proceedings of 23rd Conference on User Modeling, Adaptation, and Personalization (UMAP), Dublin, Ireland (2015)Google Scholar
- Wong, A.: The whole story, and then some: ‘digital storytelling’ in evolving museum practice. In: Proceedings of Annual Conference on Museums and the Web, Chicago, IL, USA (2015)Google Scholar
- Zheng, Y., Mobasher, B., Burke, R.: CARSKit: a java-based context-aware recommendation engine. In: Proceedings of 15th IEEE International Conference on Data Mining (ICDM), Atlantic City, NJ, USA (2015)Google Scholar