Augmented reality has seen an increased popularity among the last decades due to technological advances and, a consequent growth of the amount of augmented reality systems, became available. However, in order to diffuse this technology successfully, understand users’ feelings when using augmented reality applications is considered a major issue. This study implemented this technology in a cultural heritage outdoor context and tested it in-situ to evaluate user’s perspective regarding to personal satisfaction – including cultural enrichment acquired –, ease of use and their intention to use it. The results obtained, through questionnaires, presented the visitors acceptance regarding the usage of this type of solutions among a cultural heritage context, since it may become future visits more pleasant and desirable. The user’s majority expressed the request to use this technology more often in cultural heritage spaces.
- Mobile augmented reality
- Cultural heritage
- User tests
Museums are striving for having their spaces full of visitors whom, by their side, want to learn, enrich themselves and have fun. It is known, through an internal survey made in the Conimbriga Monographic Museum-National Museum in 2014, that the main reason that motivates visitors to come to Conimbriga is cultural enrichment (50,6%). Other motivations are related to leisure time (20,2%), to improve their knowledge about history and archaeology (17,7%), to Ruins’ mosaics, architecture and urbanism (6,5%), and to the object collection in the museum (5,0%).
To fulfil the expectations of both – museums and visitors – museums are attempting to provide information by means of different and innovative methods. Aside from their historic artefacts and infrastructures (or what remain of them), they usually provide images added to specific places (digital and real), mock-ups reconstructions, multimedia content (audio-visual information), etc. Therefore, new technologic approaches, such as augmented reality, has been exploited in the last few years among different museums. The Monographic Museum of Conimbriga-National Museum, an archaeological space with the ruins of an important ancient Roman City, presented a report in 2016 , where visitors stated that this museum is interesting from the historic point of view (9,4/10) and worthwhile a visit to the ruins (9,2/10). However, less satisfactory results were obtained when visitors were questioned if Conimbriga knows how to use technology in the space in a way to provide a more interesting visit (6,9/10) and when questioned about the enjoyability for children while visiting Conimbriga (6,7/10). Hence, a proper and user-oriented use of technology should be prospected to cultural heritage contexts, profiting visitors and institutions. Notwithstanding, the way that visitors from archaeological spaces look and feel about technology applications in cultural heritage spaces, may differ in a way that technology could be considered as intrusive and unexploited instead of being profitable.
AR has been recognized as a good solution to use in distinct areas and its usage has a positive impact in users’ perspective (Liu, Zhang and Bao 2016).
The main purpose of this study is to better comprehend the users’ opinion about the use of AR technology when visiting a cultural heritage space regarding to its satisfaction, the ease of use, and their desire to use this technology.
This research presents a case study whereby the visitors of a space – an archaeological space – the Roman Ruins of the Conimbriga Monographic Museum-National Museum – were invited to test an AR application for mobile devices. The usage of a smartphone for the experiment brings this technology closer to people since there is no need to resort to further gadgets as head-mounted devices, being less intrusive to visitors whilst saves money for cultural institutions. The visitors who accepted this challenge, used a smartphone themselves to test the AR app with no time limit and without restrictions. The outcome obtained across this experiment are the results acquired from questionnaires on a Likert-type five-level scale.
2 State of the Art
This chapter intends to introduce the concept of augmented reality and to present relevant conclusions from studies related to AR and users. A summary of recent studies where this technology was tested with users is made.
2.1 Virtual and Augmented Reality
To allude the concept of augmented reality, it is suitable to mention virtual reality as well, since augmented reality is a correlation between real environment and virtual reality. In 1994, Milgram and Kishino  presented the concept of Virtuality Continuum: a term used to describe the concept through a continuous scale, ranging the totally virtual and the totally real environment. Thus, according to their concept, while in virtual reality the user is totally immersed in a virtual environment, the experience with augmented reality allows the user to be aware of both environments: virtual and real.
Augmented Reality (AR), despite its early appearance during the 60’s , it was among the last decades that has become more popular, acquiring preponderance and prominence. Due to technological advances, today it is possible to access to AR experiences using handy and ubiquitous devices, like smartphones or tablets, bringing this technology closer to general public. AR has been known as a good solution to use in distinct areas and its usage has a positive impact in users’ perspective . Although this technology has seen popularity grow and being present among a vast variety of areas, such as publicity , entertainment , education , , medicine , architecture , manufacturing  and, the particular case for this study, in cultural heritage [13,14,15], it is not very common to find AR experiences available for visitors in archaeological spaces whereby they can experience it among their visits. This lack is even more noticeable for engaging AR experiences in outdoor spaces, a fact that sparked interest to unleash this research.
2.2 Previous Studies with Users Using AR
In the first years of the new millennium, a mobile augmented reality implementation in cultural heritage sites, arose as one of the pioneers projects named Archeoguide . That same year, Vlahakis et al.  published the first results of the Archeoguide project. This study, where younger users appeared as the most enthusiastic, some problems using the system were detected. As examples of these problems, they found the fact that users felt uncomfortable while using the technological devices and the difficulty of visualizing the digital information outdoors due to the sunlight.
Meanwhile, mobile AR became lighter to carry, since today is implemented in single handheld devices, various approaches were developed, and several cultural institutions have made their own mobile augmented reality applications. Some examples are found in Philadelphia Department of Records, the city of Christchurch, the Museum of London, the Netherlands Architecture Institute and the Powerhouse Museum in Sydney.
Recently, several studies tested the usage of AR in cultural heritage contexts using different approaches to understand the acceptance and the intention to use the technology. Hence, the necessity to understand how other researches evaluated their systems arose.
All in all, a collection of recent studies – from 2014 up to now – related to AR applications developed in order to, somehow, preserve or disseminate cultural heritage spaces, was made and is summarised in Table 1.
Observing the previous studies where AR was implemented and tested with users within diverse cultural heritage purposes, it is noticeable that the most common instrument used to evaluate the systems implemented – for those that present results related to users – is the questionnaire and the number of participants vary amid 16 and 200 volunteers.
A mobile AR prototype was developed to allow the visitors of the archaeological space in study – the Roman Ruins of Conimbriga – to experience mobile AR technology. This prototype, named DinofelisAR, was developed for Android devices and an image was used as mark to detect the position and orientation of the user in order to place the virtual information in the right position when overlapped to the real scenario perceived trough the smartphone’s camera. Using this prototype in-situ, the user could see a virtual reconstruction of the Forum superimposed over its existing ruins.
3.1 Prototype Development
Due to a large quantity and diversity of frameworks available to support the development of AR systems, a research study was made in order to identify which were the frameworks that would fulfil the requirements needed for this project . Considering this prototype requirements, three frameworks seemed comply the requisites: Vuforia SDKFootnote 1, Kudan SDKFootnote 2 and, Wikitude SDKFootnote 3. The referred study revealed that, for the specific tests made, Vuforia SDK was the one which best fulfilled the needs for this prototype. In addition to the good performance tests achieved by Vuforia SDK, this framework was the only one that, in the free version for developers, allowed to track the camera with natural features when the marker gets out of the field of view with a feature named by Vuforia as Extended Tracking. Although this technique appeared to be less stable when compared to the marker-based tracking, especially with restricted light conditions, this feature enhanced the application flexibility and freedom to navigate in space.
DinofelisAR, the prototype created to be tested in-situ, was developed in Unity 3DFootnote 4 and using a Vuforia package for Unity. An image was used as mark (Fig. 1) and the functionality Extended Tracking was activated in order to allow users to explore the surroundings even if the image used as mark, got out of the field of view.
Since the museum requested for infographics in place to inform visitors about the place of the AR experiment, the image presented in Fig. 1 was created for two purposes: as an informative image about the existence of an AR experience in the space; and as a mark for the AR app.
3.2 Local of the Experiment
The intention to evaluate DinofelisAR app by users when used in cultural heritage contexts, took us to test the prototype in Conimbriga.
Conimbriga is a Roman provincial city which was abandoned during the medieval age and it was inhabited between centuries IX B.C. and A.D. VII–VIII 
DinofelisAR app was tested in the location where used to be edified the Forum – a public monument with large dimensions which was virtually reconstructed – and its position corresponds to the centre of the ancient city (Fig. 2). Its dimensions were approximately 48 × 96 m and it would be around 9 m high.
Within the Forum’s ruins, an ample and open space, the experience occurred in square’s centre, as illustrated on Fig. 3.
A green circle in Fig. 3 identifies the place where the experience with the DinofelisAR app took place. The volunteers who agreed to participate in this study, could visualize, in the smartphone and around their surroundings, the virtual building corresponding to the Roman Forum that used to be edified there during the Roman Era. In the beginning of the experience, Forum’s Temple would be right in front of the users and the surroundings would be composed by the peristyle, with columns and walls. The Forum was edified all around the square which means that all its structure, composed by columns, roofs and walls, established a closed space.
3.3 Tests Procedure
The tests occurred between 17th and 19th of May of 2017 in the Ruins of the Conimbriga Monographic Museum-National Museum. For this AR experience, a smartphoneFootnote 5 was handed to each participant and they were invited to perform several steps that are hereinafter described.
1. First, to launch the application DinofelisAR.
2. Second, to point the smartphone’s camera to the image provided (this image is the mark used as target for the AR experience) until a virtual cube appears over the referred image as illustrated in Fig. 4. Technically, this moment was where the user’s position and orientation were identified.
3. To explore the surroundings with the smartphone to visualize the virtual model over its ruins (Fig. 5 [A] and [B]). Users were able to navigate in a 360° angle.
4. For the last step, participants were invited to fill a questionnaire (English or Portuguese version) on a five-point scale.
Looking at Fig. 5, it is possible to observe that the ground is part of the real content around user. The virtual model is overlapped over the ruins to give the user a glimpse of the building’s appearance back in the Roman Era.
4 Presentation and Discussion of Results
The visitors of Conimbriga’s Roman Ruins which accepted the invitation to be part of this study, after testing AR experience by their hands, filled in a questionnaire whose questions intended to ascertain the users’ opinion related to (1) satisfaction, which includes the pleasure, dynamism and activity among the visit to the Forum, as well as the level of engagement in the visit and achievements obtained; (2) the ease of use of AR technology; and (3) the desire to use this technology in cultural heritage spaces.
A total of 90 participants accepted to be part of this study (51% female and 49% male). Among this heterogeneous group of participants, 44% of them were more than 55 years old, 19% between 40 and 55 years old, 21% between 25 and 39 years old, 9% between 18 and 24 years old and, 7% were less than 18 years old. Considering their level of education, while 1% preferred not to answer this question, 73% of the respondents were higher educated, contrasting to 26% that were not.
The participants were also questioned about why they decided to be part of this experiment and, the majority (72%), stated the interest of augmented reality applied to archaeological context. Even the participants that affirmed not to know the technology in study, they got curious to try it (18%) while 10% asserted other reasons. These results help to predict the tendency of people when invited to use AR.
4.1 User’s Satisfaction
The results collected disclosed that the participants considered that AR contributed for a more dynamic and active visit to the Forum (94%) and more pleasant as well (94%).
Moreover, 92% of the participants agreed or strongly agreed with the allegation of being more involved with this tour zone when using the AR app.
Keeping in mind the motivation of visitors about people’s motivation to visit Conimbriga – the majority, for cultural enrichment –, it was asked about the cultural enrichment provided by using the AR app when they were visiting the Forum. The results presented a majority agreement: 97% of the participants agreed or strongly agreed that AR applications can contribute to cultural enrichment during their visits.
Considering the Forum features acquired, 91% of participants stated that became able to describe better the Forum characteristics – e.g. size, colours, architecture.
In short, participants emphasized a significant level of satisfaction using AR technology in Conimbriga’s Ruins. Using DinofelisAR when visiting the Forum, the clear majority of the participants stated that their visit became more dynamic and active, more pleasant, they felt more involved, acquired new findings related to the referred building, and facilitated their cultural enrichment.
4.2 Ease of Use
An interesting fact about the use of AR technology, is that even people who never used this technology before this experiment (61%), considered it easy to use, as well as those who already had tried it before (Fig. 6).
Figure 6 displays that, regardless the previous experience using AR, the majority affirmed that this technology is easy to use. In fact, 96% of the participants agreed or strongly agreed that this technology is easy to use.
The majority (84%) didn’t feel lost during the experience, not knowing very well where to point the smartphone in order to see the virtual model.
Furthermore, most of the participants (88%) assumed that after some short moments using AR technology, they already were comfortable using it and understanding how to use it correctly.
In brief, the usage of AR technology was revealed as a very easy and rapid learning tool to handle, independently of the individual participant’s characteristic – age, gender or level of expertise.
4.3 Desire to Use
And because the participants’ intention to use this technology is also a point of interest for this study, the visitors were asked about their interest in installing an app similar to the one provided in the experiment in their own smartphones. The results were also positive, and the majority stated that they would be interested (Fig. 7).
In Fig. 7 is shown that, regardless of the average age of this study sample, participants confirmed their desire to install an AR mobile app in their phones with the intension to use it among cultural heritage spaces.
As visitors of Conimbriga, it was asked if the use of this technology could contribute to an increase number of visitors to this archaeological space where 90% of the participants agreed or strongly agreed.
Additionally, through open-ended questions related to suggestions and issues found during their experience, the participants specifically requested to extend AR technology to other places – inside Conimbriga’s Roman Ruins and other archaeological spaces as well – (33%); to freely explore the space without the need of being close to the mark (28%); to look through a bigger screen (13%) – this suggestion was made mainly (83%) from people with more than 55 years old –; to provide a more detailed virtual model; and to add people/avatars to the virtual scene. Other requests made by the participants included the use of sounds and the ability to zoom the virtual model in order to observe it with more detail.
In sum even though the future installation of these AR applications implies the usage of their personal mobile devices, these results reveal that most participants confirm their interest using AR technology when visiting cultural heritage spaces. Their intent is also noticeable among their suggestions presented in the open-ended questions.
This study intended to implement and test a mobile AR application with visitors of a cultural heritage space, which in our case is the Roman Ruins of Conimbriga, and to evaluate its use with visitors. This rollout was specifically guided to evaluate an application of mobile AR used in outdoor spaces, among an archaeological space, where a virtual building was superimposed in its correspondent ruins.
A total of 90 visitors participated in this study which was accomplished in the centre of the ancient city Forum’s square. These volunteers used a smartphone (an Android device was provided) to run the mobile AR app named DinofelisAR. They were able to perceive and explore the virtual reconstruction of the Forum overlapped on its real ruins. Each user could observe, in real-time and in a 360-degree view, the virtual model while perceiving real images trough the camera. After this experience, the participants filled a questionnaire and the results had exposed that this technology may be an excellent opportunity to improve and spotlight cultural heritage spaces. Analysing participants’ opinions related to their satisfaction, including the achievements acquired, their opinion related to the ease of use, and their intention to use AR technology in this context, had highlighted an unequivocal benefit from users’ perspective.
Looking ahead, the importance of deeply understand users’ perspective when using a new technology such as mobile AR, is well known to accomplish better approaches that would fit the visitors’ expectations and needs when visiting cultural heritage spaces. Hence, arises also the necessity of comprehend the acceptance and the intention to use technology among other cultural heritage spaces beyond the Ruins of Conimbriga.
Analysing the results presented among this paper, it is our opinion that AR technology has great potential to engage people and seemed to be easy to use amid everyone who tested it. Therefore, dwell on AR technology is appropriate and stimulates opportune and useful new fields of research for scientific community and for society.
More information about Vuforia SDK on https://developer.vuforia.com/.
More information about Kudan SDK on https://www.kudan.eu/overview/.
More information about Wikitude SDK on https://www.wikitude.com/.
More information about Unity 3D on https://unity3d.com/pt.
The smartphone used for this experiment was a Motorola 3rd Generation 5.0’’ (Quad-core 1.4 GHz Cortex-A53, 1 GB RAM, camera 13 MP, f/2.0).
Santander Chair- Universidad Complutense de Madrid. Conimbriga’s Visitor Satisfaction (2016)
Milgram, P., Takemura, H., Utsumi, A., Kishino, F.: Augmented reality : a class of displays on the reality-virtuality continuum. In: SPIE - The International Society for Optical Engineering, pp. 282–292 (1994)
Sutherland, I.E.: The ultimate display. In: Proceedings of the IFIP Congress, pp. 506–508 (1965)
Bae, H., Walker, M., White, J., Sun, Y., Golparvar-Fard, M.: Fast and scalable structure-from-motion based localization for high-precision mobile augmented reality systems fast and scalable structure-from-motion based localization for high-precision mobile augmented reality systems. mUX J. Mob. User Exp. 5(1), 4 (2016)
Liu, H., Zhang, G., Bao, H.: Robust keyframe-based monocular SLAM for augmented reality. In: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (2016)
Kim, Y., Kim, W.: Implementation of augmented reality system for smartphone advertisements. Int. J. Multimed. Ubiquit. Eng. 9(2), 385–392 (2014)
Hamasaki, M., Takeda, H., Nishimura, T.: Network analysis of massively collaborative creation of multimedia contents. In: Proceedings of the 1st International Conference on Designing Interactive User Experiences for TV and Video, UXTV 2008, pp. 165–168 (2008)
Yuen, S.C., Yaoyuneyong, G., Johnson, E.: Augmented reality: an overview and five directions for AR in education. J. Educ. Technol. Dev. Exch. 4(1), 119–140 (2011)
Bower, M., Howe, C., McCredie, N., Robinson, A., Grover, D.: Augmented reality in education – cases, places and potentials. EMI. Educ. Media Int. 51(1), 1–15 (2014)
Paolis, L.T.: augmented visualization as surgical support in the treatment of tumors. In: Rojas, I., Ortuño, F. (eds.) IWBBIO 2017, Part I. LNCS, vol. 10208, pp. 432–443. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-56148-6_38
Meža, S., Turk, Ž., Dolenc, M.: Measuring the potential of augmented reality in civil engineering. Adv. Eng. Softw. 90, 1–10 (2015)
Regenbrecht, H., Baratoff, G., Wilke, W.: Augmented reality projects in automotive and aerospace industry. IEEE Comput. Graph. Appl. 25(6), 48–56 (2005)
Canciani, M., Conigliaro, E., Del Grasso, M., Papalini, P., Saccone, M.: 3D Survey and augmented reality for cultural heritage. The case study of Aurelian wall at Castra Praetoria in Rome. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 41, 931–937 (2016)
Fidas, C., Sintoris, C., Yiannoutsou, N., Avouris, N.: A survey on tools for end user authoring of mobile applications for cultural heritage. In: Proceedings of the 6th International Conference on Information, Intelligence, Systems and Applications (IISA), pp. 1–5 (2015)
Han, D.-I., Jung, T., Gibson, A.: Dublin AR: implementing augmented reality in tourism. In: Xiang, Z., Tussyadiah, I. (eds.) Information and Communication Technologies in Tourism 2014, pp. 511–523. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-03973-2_37
Gleue, T., Dähne, P.: Design and Implementation of a mobile device for outdoor augmented reality in the archeoguide project. In: Proceedings of the 2001 Conference on Virtual Reality, Archeology, and Cultural Heritage, VAST 2001, pp. 161–168 (2001)
Vlahakis, V., et al: Archeoguide: first results of an augmented reality, mobile computing system in cultural heritage sites. In: Proceedings of the 2001 Conference on Virtual Reality, Archeology, and Cultural Heritage (VAST 2001), pp. 131–140 (2001)
Kourouthanassis, P., Boletsis, C., Bardaki, C., Chasanidou, D.: Tourists responses to mobile augmented reality travel guides: the role of emotions on adoption behavior. Pervasive Mob. Comput. 18, 71–87 (2015)
Chung, N., Han, H., Joun, Y.: Tourists’ intention to visit a destination: The role of augmented reality (AR) application for a heritage site. Comput. Hum. Behav. 50, 588–599 (2015)
Jung, T., tom Dieck, M.C., Lee, H., Chung, N.: Effects of virtual reality and augmented reality on visitor experiences in museum. In: Inversini, A., Schegg, R. (eds.) Information and Communication Technologies in Tourism 2016, pp. 621–635. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-28231-2_45
Střelák, D., Škola, F., Liarokapis, F.: Examining user experiences in a mobile augmented reality tourist guide. In: Proceedings of the 9th ACM International Conference on Pervasive Technologies Related to Assistive Environments (PETRA 2016) (2016)
Pendit, U.C., Zaibon, S.B., Bakar, J.A.A.: Measuring enjoyable informal learning using augmented reality at cultural heritage site. J. Telecommun. Electron. Comput. Eng. 8(10), 13–21 (2016)
Kasapakis, V., Gavalas, D., Galatis, P.: Augmented reality in cultural heritage: field of view awareness in an archaeological site mobile guide. J. Ambient Intell. Smart Environ. 8(5), 501–514 (2016)
Oleksy, T., Wnuk, A.: Augmented places: an impact of embodied historical experience on attitudes towards places. Comput. Hum. Behav. 57, 11–16 (2016)
Tussyadiah, I.P., Jung, T.: Embodiment of wearable augmented reality technology in tourism experiences. J. Travel Res. 57, 597–611 (2017)
Marto, A.G.R.: Realidade Aumentada Móvel num Contexto de Herança Cultural. Faculty of Engineering, University of Porto (2017)
Correia, V.H.: A Arquitectura Doméstica de Conimbriga e as Estruturas Económicas e sociais da Cidade Romana. Simões & Linhares Lda (2013)
This work was supported by national funds through the Portuguese Foundation for Science and Technology (FCT) under the project UID/CEC/04524/2016.
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
© 2019 The Author(s)
About this paper
Cite this paper
Marto, A., Gonçalves, A., de Sousa, A.A. (2019). DinofelisAR: Users’ Perspective About a Mobile AR Application in Cultural Heritage. In: Duguleană, M., Carrozzino, M., Gams, M., Tanea, I. (eds) VR Technologies in Cultural Heritage. VRTCH 2018. Communications in Computer and Information Science, vol 904. Springer, Cham. https://doi.org/10.1007/978-3-030-05819-7_7
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05818-0
Online ISBN: 978-3-030-05819-7
eBook Packages: Computer ScienceComputer Science (R0)