Keywords

1 Introduction

With the dawning of the new museology, in the early 1970’s, museums began a process of change in their methods of communication with their users, now encouraging them to interact more with the exhibition thus making them abandon their classical role as passive spectators, to become the agents of their own learning experience in a proactive and entertaining way. For this purpose, many different emerging technologies have been applied, from multimedia content to virtual reality environments.

Nowadays, game engines are becoming popular as a mean to develop high-end, real-time presentations of virtual environments not necessarily related to the game industry. Many examples of their use in the field of architectural and urban visualization can be found. Therefore, virtual reconstruction of historical heritage using these engines has been the logical next step in the direction of their use in virtual archaeology and the application of such technology in the field of museology.

Natural interaction is another emerging trend with direct application for museum installations. It is also used in the game industry to enhance the gaming experience by means of specialized hardware designed to obtain information of the user’s intention without using traditional devices such as a computer mouse or a gamepad. Depth cameras, such as the Kinect family, allow the computer to be aware of the user’s movements.

Many interesting examples showing how game engines, sometimes with the use of a depth camera, displays visual recreations of archaeological reconstructions can be found today [1–6].

Despite that, few studies can be found that takes into respect the usability of the different paradigms for movement and actuation that are implemented in those systems. There is a variety of interaction mechanism to transform the user gestures into computer commands, some of them more easy to use and intuitive than others. Their functioning mechanic ranges from a single user gesture to control one specific action, such as forward displacement, to a complex set of user gestures to control a wide variety of movements and actions.

Natural interaction paradigms trend to provide the user with the confidence of using familiar, everyday actions to control the functioning of the computer application. These actions mimic gestures and behaviors utilized in the real life to achieve different goals in the virtual realm. Depending on the conceptual relations to the everyday gestures that they mirror, these actions can be classified as follows:

  • Natural: The user applies gestures for the exact same action in the real world. (i.e. the action of grabbing to handle a virtual object.)

  • Metaphoric: Actions that evokes the desired behavior of the system by the existence of a correspondence and similarity. In a metaphoric interface there is a mapping of concepts and operations between to domains, in this case the virtual world and the reality so that an interaction suggested by the metaphor source domain corresponds to the execution of the application implementing the metaphor target domain [7] (i.e. moving a hand left and right in the air to browse a sequence of pictures in a projection screen).

  • Symbolic: In symbolic natural interaction the objects are represented by their visual, aural, and maybe in the future touch sensitive clones. They are naturally manipulated, but they are still representations and not real things [8] (i.e. driving a virtual car by moving hands to control a virtual steering wheel).

In the exploration of architectural reconstructions of archaeological remains, the user movement inside the model is the most important interaction. The examples aforementioned found in museums and exhibitions present different approaches to translate the visitor intentions to the movement of his or her representation in the virtual world but there are not enough case studies that supports either the preference of one over the others, nor the simple adequacy of a paradigm for what is intended. Indeed, the authors’ experience on the use of some of these installations is that they are sometimes too cumbersome for the visitor to understand.

2 Objectives

The experiment we present here has the general objective of testing and evaluating the advantages and limitations of two different paradigms of movement inside an archaeological example in terms of engagement and usability in order to be used in a museum environment.

The test was done on a virtual model of a 4th century Roman villa [9, 10], constructed as a virtual installation for an interpretation center near Seville (Spain).

The virtual model was intended to accomplish two objectives. On the one hand, it was to display a complete recreation of the mosaics found in the nearby excavation, allowing the visitors to contemplate the appearance of the pavements in their full size instead of just fragments. On the other hand, the villa model was designed to act as a built-in environment that could provide context for the interpretation of the mosaics (Figs. 1, 2 and 3).

Fig. 1.
figure 1

Atrium entrance with “The Judgment of Paris” mosaic put in place

Full size image
Fig. 2.
figure 2

Atrium with the central impluvium

Full size image
Fig. 3.
figure 3

Lararium

Full size image

The digital reconstruction was implemented to be explored by means of natural interaction schemes. To achieve this goal, a Kinect sensor for Xbox One also known as Kinect V2 was used, a device designed for videogame interaction that detects the presence and motion of players (Fig. 4).

Fig. 4.
figure 4

Kinect interaction

Full size image

The virtual representation of the domus can be fully visited. The user can walk freely throughout all of the complex, enjoying not only the architecture of the building, but also the wall paintings, furniture, mosaics, and other elements of material culture (amphorae for oil and wine, tegulae, oil lamps, etc.). The setting of the different spaces (atria, peristila, lararium, triclinium, tablinium, etc.) helps interpreting daily life in such facilities. The main focus is the mosaic of the “Judgement of Paris” since this piece is unique in Hispania, being one of only five known cases found in all of the Roman Empire depicting this theme [11].

In line with attaining a quality evaluation of the user experience, we should consider various aspects in terms of usability, effectiveness and satisfaction. This evaluation can be carried out by means of interviews regarding the ease or difficulty of its use during the visit to the virtual museum. The questions included aspects about the environment and the main objects, including the mosaics.

The collected results should contribute to the study of paradigms of navigation inside virtual architectural environments such as those used in the field of virtual archaeology.

3 Movement Paradigms. Implementation

The installation captures the user’s gestures by means of a Kinect depth camera attached to the system. The game engine that is used to visualize the virtual model (Unreal engine 4) incorporates a Kinect 4 Unreal library developed by Opaque Multimedia. The authors used that library to code the different gestures and actions that users perform to move and interact inside the virtual building.

The Kinect system is capable to feed the game engine with a continuous flow of data describing the body configuration of a user in front of the device. This data contains the location of a set of characteristic points in the user’s body, called joints, together with their topological relations that describe a simple skeleton that depicts the user’s pose. The Kinect for Unreal libraries can be queried to obtain both position and orientation of some joints related to other joints and to the Kinect device.

The two different interaction schemes described in this paper have been implemented as described below.

  • Metaphorical. Rise-hand scheme: This approach uses the movement of the user’s arm to control both displacement and orientation. The player controller analyses how much the user lifts his or her hand (Fig. 4) and measures the angle formed by the wrist and the elbow, both in the horizontal plane (yaw) relative to the direction to the Kinect Device, which is used for turning, and in the vertical plane (pitch) relative to the vertical, which is used to move forward and control speed. An idle arm, pointing to the floor means a zero angle in both directions.

    The pitch angle may be negative, thus allowing for backward displacement by pointing the arm just slightly backwards.

The user can change his or her orientation at the same time by pointing sideways with the same arm. Hence, the user can turn and control the displacement speed simultaneously.

  • Natural. Step/Twist scheme: This second approach is designed to offer the user a way to explore the virtual building by using movements similar to those used when walking. In this scheme, users can increase or decrease the virtual walkthrough speed by stepping forward or backward and perform turns by twisting the upper body clockwise for a right turn and counterclockwise for a left turn.

Marks in the ground depicting the usual arrow icons used in audio and video playback indicated locations for the start, move forward at slow pace, move forward quickly and move backward commands. Users were placed in the starting position to begin the test and were then given a brief description of the movement scheme prior to letting them move alone.

4 Usability Test

The authors applied a qualitative, user-centred methodology, based on measurement and systematic analysis of the values used by ISO 9241-11 to define usability; namely effectiveness, efficiency and satisfaction [12].

4.1 Methodology

The authors chose to apply a custom methodology in this experiment which could combine the measure of the perception of the user of the mechanical aspects of the system with their experience of the virtual visit of the virtual villa. This second aspect was studied both from the perspective of the accessibility of the subject to the architectural environment as well as their satisfaction as a museum visitor. Aside from collecting data, users were filmed while performing the tests. This allowed the authors to gather useful information to be applied in future enhancements of the system.

The experiment was organized as follows: First, users gave their usual demographic data as well as a self-assessment of their skills as videogame players. Then, they received a general explanation of the Roman villa and walked through it using both paradigms, first trying to accomplish a task and then moving freely afterwards.

Subjects were interviewed to evaluate their perception of the architecture of the building, the objects and signs placed inside it and their perception regarding the attention consciously employed in the use of each movement scheme. Next, in order to concentrate the study to a specific user profile, namely the museum visitor, the authors applied the System Usability Scale SUS [13]. SUS is a Likert scale that gives great importance to the subjective opinion of the users, and at the same time being a reliableFootnote 1 method to obtain usability measurements.

4.2 Selection of the Data Sample

The data sample for this experiment was defined through a selection of key informers marked as potential museum visitors. A pilot test carried out previously used a convenience sample close to this profile. In order to stablish the percentages that define the profile of a museum visitor, the authors applied the data obtained by a study performed by the Spanish Laboratorio Permanente de Público de Museos [19]. The result of this distribution of statistical variables, from a combinatory optimization perspective gives optimum results for a sample size of 38 individuals. For the case described here, and considering that the SUS methodology has demonstrated to be reliable for a sample size of 8–12 people, the sample was recalculated and the number 13 was used as the next confidence value for the aforementioned percentages. This resulted in the following distribution is as follows.

  • Number of individuals 13 (mean error: 1.29 pp)

  • Gender: 6 males, 7 females

  • Age: less than 25:2; 26 to 45: 6; 46 to 65: 4; 66 and older: 1

  • Education: Primary: 1; High school: 6; University: 4: PhD: 2

5 General Results

The results of the test carried out for both paradigms of movement yielded the following results, collected in Table 1, expressed in a value range from 0 to 10.

Table 1. General results
Full size table

Regarding SUS, both paradigms yielded a result over 70, which placed them in the interval between Good and Excellent in the adjective rating elaborated for SUS by Bangor et al. [17].

Table 1 shows a comparison between both paradigms regarding their effectiveness for the visitor of a virtual museum. They show that architecture of the place is appreciated easily in both paradigms. On the other hand, both paradigms presented problems when users tried to get to a place to read signs with descriptive text, which is a task that requires a more precise placement in front of the sign and the right distance. That points to a clear necessity to improve the system in that regard.

The easiness of contemplating the objects placed in the scene received a good valuation in both paradigms. Contemplation of the mosaics, which are located on the floor presented more difficulties for the rise-hand paradigm than for the step/twist paradigm (Fig. 5).

Fig. 5.
figure 5

Comparison between step/twist and rise-rand paradigm for every poll question

Full size image

It is interesting to note the ratio of attention consciously invested in enjoying the museum compared to the attention that is consciously invested in controlling the movement (Fig. 6). We can tell that the step/twist paradigm shows to be more appropriate behavior for a museum installation, displaying the need of less attention invested in controlling movements and more attention could be invested in the experience.

Fig. 6.
figure 6

Comparison of conscious attention invested in both paradigms

Full size image

The analysis of the SUS from a gender perspective presents a good valuation for both paradigms, but females give specially good grades to the step/twist scheme (Fig. 7). The standard deviation for females is also smaller, indicating a clear agreement in that sense.

Fig. 7.
figure 7

Gender comparison of SUS results

Full size image

6 Discussion

6.1 Effectiveness

The effectiveness of both of the paradigms can be measured by analyzing their suitability to contribute to the installation in order to achieve its purpose as a virtual museum.

In this sense, regarding the easiness to perceive the virtual environment, both paradigms offered good results in the architectural aspect (question 1), and object contemplation (question 3). The rise-hand paradigm presents slightly better results in this regard.

A clear drawback appears when it comes to the task of reading informative signs (question 2), the rise-hand seems to be the worse of the two paradigms in this regard. Two users remarked their difficulty to reach a proper location in order to read the descriptive signs correctly. Regarding the contemplation of the mosaics (question 4), both paradigms receive a medium valuation, although the step/twist scheme is clearly superior.

6.2 Efficiency

The efficiency in this case can be described as the use of attentional resources invested in achieving a concrete task or goal. In this particular aspect, the step/twist scheme surpasses the rise-hand scheme since it allows the user to move in the virtual environment while giving more attention to the enjoyment of the experience than to the control of the movement in the digital space (Fig. 6).

6.3 Satisfaction

The SUS results permits the measuring of the global satisfaction for a museum user profile that utilizes the system. Both paradigms are valuated in the range of Good to Excellent. That offers a positive support for the implementation of this kind of installations in a museum environment.

7 Conclusions

This experiment concludes the suitability of Kinect based natural interaction for museum installations both for natural and metaphorical paradigms, specifically for rise-hand and step/twist movement schemes. In this regard, results yield very good valuations for both schemes in regard of effectiveness, efficiency and satisfaction when the experiment is applied on a museum user profile sample. The system is good for experiencing digital architectural environments and for contemplating objects such as those present in the virtual museums seen in the field of virtual archaeology. Nevertheless, some effort has to be invested in the design aspect of the virtual signs located on those environments, which are somehow difficult to reach and read by users.

In general, the step/twist scheme gave better results, which points in the direction that natural paradigms of natural interaction could be more adequate than metaphoric ones for museum installations.