Tool UTAUT Applied to Measure Interaction Experience with NAO Robot

  • Adrián VegaEmail author
  • Kryscia Ramírez-BenavidezEmail author
  • Luis A. GuerreroEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11568)


This research described the utilization of the Unified Theory of Acceptance and Use of Technology (UTAUT) to evaluate a high-level interactive scenario of using Nao Robot in the role of co-presenting in an educative context. Describing the registered result of construct such as Intention of Use, Perceived Enjoyment (PE), Perceived Sociability (PS), and Trust.

The study also describes the process of elaboration of this interaction. Multiple User-Centered Design techniques applied to the Human Interaction Robot field. Also, describes how the interaction of the robot with the participant was accomplished by using Wizard of Oz techniques.


Human and computer interaction User Centered Design User Experience Unified Theory of Acceptance and Use of Technology Wizard of OZ Nao robot 

1 Introduction

Developments in robotics can be explained as a long process of historical evolution [1]. Starting from the methods which humans interact with objects and the control we get over them. Since ancient civilizations such as Egyptian and Roman exist records of art sample of human simulations represented by puppets, paintings, and statues. Alongside this process are the records of mechanical devices created to simplify human tasks. A long evolution of technologies such as hydraulic, steam, oils, electric… was required to integrate kinesthetic functions to what might be described as “human automation.” Those basic devices bring about basic functions, but they were more related to the entertainment industry and science fiction. It is until the development of the last 50 years that it was allowed the automation of complex cognitive and physical functions. Currently, there are plenty of examples of robotic devices competing with people in activities such as manufacturing, sports, data analysis, among many others.

Nowadays, the robotics field is even growing its application coverage. However, additional challenges have been added besides the functionality, aesthetics, and complexity of the robots. One of the main current challenges is related to find more emphatic ways the people interact naturally with the robots in everyday environments. Thus, developing social engaging interactions with robots.

Robotics has proven to have diverse application in areas such as education [2, 3, 4], relate [5], health [6, 7], among others. Many research described different elements to consider in the design of interactions with socially intelligent robots [7, 8, 9, 10]. Theory of “Social Intelligent Robots” [7] describes that the interaction with a robot must meet four different criteria. Socially evocative rely on anthropomorphizing and capitalize on feelings evoked. Socially Situated able to react to other social agents and object in the environment. Sociable proactively engage with humans to satisfy internal social aims. Socially Intelligent shows deep models of human cognition and social competence during robot interaction.

Many different requirements needed to be considered in the design of more Social Intelligent Robots. The Human-Robot Interaction (HRI) uses different techniques to improve the design and evaluation of effective interactions. One frequent sample of interaction technique in HRI, especially over early stages of development, is the Wizard of Oz (WoZ). Meanwhile, a sample of HRI evaluation technique is the Unified Theory of Acceptance and Use of Technology (UTAUT).

About the design of the interaction, one of the most current approaches is User-Centered Design (UCD). Through this approach, it has been possible to design meaningful interactions [2] with robots further than the current state of the technology. The UCD propose focus design of technologies on how people use those technologies instead of just technology itself. It is by integrating knowledge and techniques from different fields such as Design, Anthropology, Psychology, Robotics, Engineering, among others, that UCD shifts the focus of technology development cycle in the interaction with the users. Tools such as prototyping, scripts, and pilot testing are every day more frequently used in designing desktop software, mobile apps, websites, and currently robotics interactions. Despite the technologic limitations or budget resources, the UCD allows the validation of the next required stages of technology to accomplish users’ expectations and requirements.

The current research contains the design of a presentation using the Robot Nao as a co-presenter during a presentation. The presentation was designed by using UCD techniques with the goal to generate awareness about the risks of using weapons in the resolution of conflicts. The presentation was presented to four groups of secondary students using the Woz technique and evaluated by using UTAUT constructs. The objectives of this research aim to (1) Evaluate the use and acceptance of the robot during the activity. (2) Evaluate the context in which the presentation is carried out. (3) Evaluate the impact of the activity on the population.

2 Theoretical Framework

In the HRI, we find multiple designs and evaluation techniques. The WoZ is one of the most frequently utilized techniques in the design of interactive prototype due its flexibility and adaptability. Meanwhile, the UTAUT offers solid theoretical support to create an evaluation of interaction among different technologies.

2.1 Wizard of Oz (WoZ)

The Woz [11] is utilized in multiple areas of the design of technologies, especially under development. It is especially useful in cases where the implementation has a high cost, or the technology is still emerging.

In HRI the WoZ technique [12] involves simulating the end state of the technology operating through the assistance of people operating the robot. This simulation creates the illusion that the robot works autonomously. The operator of the robot works as part of the design group. They are managing the robot in a way that cannot be perceived for the study participant. Usually, the operator controls the robot remotely using robot sensors or cameras to interact with the participants.

Through the advantages of using this technique are [11, 12] that you can create interactions to be ready on the early stages of the project. Also, it is possible to evaluate the multiples iteration of the same project since its design stage. Finally, it is an effective way to evaluate participant responses identifying possible future problems without complex developments.

Some points that might consider when applying this technique are [11, 12] you will require to train the operator of the robot. Also, adding additional costs on ad extended stages of the design process. Moreover, because of being a technique based on the natural interaction with the participant, it can be challenging to be consistent in the application with different groups. Finally, it is pointed out that there is a risk of designing interaction beyond current technology capabilities.

2.2 Unified Theory of Acceptance and Use of Technology (UTAUT)

UTAUT [13] is a theoretical framework to evaluate the acceptance and usage of technologies. The main advantage of this framework is that it can be applied to different types of technologies. Founded in the Model of Technology Acceptance (TAM) [14], it integrates elements of other theories such as Theory of the reasoned Action, Motivation Model, Theory of Social Cognition, Innovation Diffusion Theory, among others.

The surveys developed from this theory consist of multiple questions grouped by constructs. Each question is elaborated in the format of Likert scales sorted in a random order to bring reliability to the different constructs.

Previous research [14] have evaluated the application of these constructs in cross-cultural scenarios and languages. These evaluations support the reliability and validity of the constructs among six different countries. Also, this framework has been previously applied in the robotics field, in samples such as the evaluation of the acceptance of assistive social robot by older adults [15].

Among the available constructs in this framework are use/usage, trust, anxiety, facilitating conditions, intention to use, perceived enjoyment, perceived sociability, social presence, among others [15]. Because in this study the only direct interaction of the participant with the robot is through dialoging during the presentation, constructs such as anxiety, facilitating conditions, or usage were not included in the evaluation.

3 Methodology

The current research consists of the design of a presentation using the Robot Nao as a co-presenter during a presentation. The presentation was designed by using UCD techniques with the goal to generate awareness about the risks of using weapons to solve violent situations. The presentation was presented to four groups of secondary students under the same conditions. In was applied the WoZ technique for interacting with the robot. The presentation was executed between a presenter and the robot using a script during the presentation. At the end of the presentation, the participant interacted directly with the robot by asking questions about the presented topic. The evaluation of the activity was fulfilled by mainly using UTAUT constructs.

The objectives of this research are to (1) Evaluate the use and acceptance of the robot during the activity. (2) Evaluate the context in which the presentation is carried out. (3) Evaluate the impact of the activity on the population.

The data collection was made in collaboration with the Social Program of the Universidad de Costa Rica and Fundación Fundamentes. In these programs, the university offers robotics workshops and trains in specific topics to the community. The Foundation raised the need to promote awareness about the danger of using weapons in the resolution of conflicts.

Two questionnaires were applied before and after the presentation to collect the data. The process begins with the first questionnaire, which consisted of a single question evaluating the perception of using weapons in the resolution of conflicts. After the first questionnaire, the person participates different activities such as workshops, tours inside the university facilities, and others. Finally, the participants participated in the presentation related to the risk of using weapons in the resolution of conflicts. There is 3 h difference between each questionnaire application.

The second questionnaire contained an evaluation of the interaction with the robot during the presentation. Multiple questions based on the UTAUT framework were implemented to evaluate this interaction. Also, designed additional questions to assess the presenter and content of the presentation. The final question evaluated any change in the perception of using weapons in the resolution of conflicts (Table 1).
Table 1.

Activities table.

3.1 Nao Robot

We used the Robot Model Nao of SoftBank Robotics. Among its main characteristics of this robot are: the ability to move, limited manipulation of objects, verbal communication capacity, and humanoid appeal (Fig. 1).
Fig. 1.

Imagen del Robot NAO

3.2 Population

The population was composed of four groups of high school students attending 7, 8 and 9 levels. Students ages were between 12 and 17 years old. These groups were pre-selected by Fundación Fundamentes from high schools at social risk they supported.

Participation in the experiment was anonymous and voluntary. For this reason, no records were kept of the identity of any of the participants. All students participated in the first questionnaire. Only the students who attended the presentation participated in the second survey. So, 100% of the students participated in the first questionnaire, while only 77% of the population participated in the second questionnaire. The average age of the population of the first questionnaire was 13.53 years. Meanwhile, in the second questionnaire, it was 13.59 years old (Table 2).
Table 2.

Groups by Gender (F = Female, M = Male, and O = Other).


G 1

G 2

G 3

G 4


Questionnaire I

T = 24

F = 14 

M = 9

O = 1

T = 30

F = 13 

M = 17

T = 19

F = 5 

M = 14

T = 28

F = 17 

M = 11

T = 101

F = 49 

M = 51

O = 1

Questionnaire II

T = 16

F = 11 

M = 5

T = 28

F = 13 

M = 15

T = 15

F = 5 

M = 10

T = 18

F = 12 

M = 6

T = 77

F = 41 

M = 36

3.3 Design of the Presentation

Different types of resources were utilized to design the presentation. Utilized concepts and methods of Human and Robot Interactive (HRI), User-Centered Design (UCD) and User Experience (UX). To support the interaction with the robot, we included visual resources such as powerpoint slides and the collaboration of a presenter.

By the collaboration of multidisciplinary groups of HCI specialist, Psychologists and Engineers the presentation script was created. Selecting the dialogues and behaviors of the robot during the presentation. This process was done by adapting Storytelling and Sketching techniques to the interaction with Nao robots. Two different testing groups evaluated the prototypes of the script. Creating a script capable of present the topic and that could be consistently shared between the different groups (Fig. 2).
Fig. 2.

Sketch of the script of the presentation

In this stage, some issues were identified as needing attention. The first need identified was creating alternative ways of representing the robot’s emotions at different moments of the presentation. It was required due to the limitations of the Nao robot with facial expressions and voice accents – especially in Spanish -. During the presentations to the test groups, they pointed out the need highlighting the emotions of the robot to achieve empathy with the stories presented. The solution included adding slides in the presentation representing the internal emotions of the robot during the presentation.

The second requirement identified was including a human presenter with the robot. At initial stages of the project, it was expected to have two Nao robots along interacting during the presentation. However, cause limitations such as the speaking speed of the robots, it was difficult to keep the attention of the public for a long time. Moreover, it was risking having two people coordinating the interaction of two different robots at the same time with the current resources.

3.4 Robot Script and the WoZ

This process required the application of mixed techniques during the script and open questions of the presentation. For this reason, the technical team implemented an interface that would allow movement and dialogues pre-programmed for NAO during the presentation, as well as independent movements and dialogs in order to freely and spontaneously interact with the participants.

To make the presentation consistent among the different groups, it was necessary that both the presenter and the robot presented the contents in the same way on each group. For this reason, the script was created. Both movements and dialogs of the presenter and robot were predefined and coordinated using the script as the reference in multiple rehearsals.

Finally, in order to provide the sensation of spontaneity and direct interaction of the participants with the robot. At the end of the presentation, there was a space of open questions regarding the subject presented. In this space, the participants could ask random questions to the robot controlled by using the WoZ technique (Fig. 3).
Fig. 3.

Robot Nao and presenter during a presentation

4 Evaluation

As mentioned before, one of the central evaluations was based on the UTAUT framework. It is a broad model that seeks to encompass the evaluation of different types of interactions with technologies. For this reason, the required construct should be selected according to the specific technology and interaction required.

According to the scope of this research, four fundamental constructs were selected [15]. Intention of Use (IU) which evaluates the intent to use the technology for an extended time This is especially important to know if the contribution of the robot in the presentation must be maintained over time. Perceived Enjoyment (PE) which consists in the perception of pleasure felt by the user while using the technology. It is contributing to evaluating if users enjoyed their interaction with the robot. Perceived Sociability (PS) refers to the ability of the system to develop social behaviors. It is very important to evaluate its relationship with the perception of the robot as socially intelligent. Finally, Trust (T) refers to the reliability and security that the interaction with technology transmits. In the case of the robot, it is important to verify that the information presented is reliably when presented by the robot (Table 3).
Table 3.

English to Spanish translation of the constructs selected to replicate from the research of Heerink et al. [15]





I’m planning to use the robot during the next few days

Consideraría participar con robots similares si tuviera la oportunidad


I think I’ll use the robot during the next few days

Recomendaría seguir utilizando al robot en actividades similares


I am certain to use the robot during the next few days

Quisiera ver más robots como este en diferentes actividades


I enjoy the robot talking to me

Disfruté hablando con el robot


I find the robot fascinating

Encontré al robot fascinante


I find the robot enjoyable

Encontré al robot agradable


I enjoy doing things with the robot

Disfruté interactuando con el robot


I find the robot boring

Pienso que el robot NO es aburrido


I feel the robot understands me

Siento que el robot me ha entendido


I consider the robot a pleasant conversational partner

Pienso que este robot ha sido un presentador agradable


I think the robot is nice

Pienso que el robot ha sido amable


I find the robot pleasant to interact with

Encuentro que la interacción con el robot ha sido satisfactoria


I would follow the advice the robot gives me

Estoy dispuesto(a) a seguir los consejos que el robot pudiera darme


I would trust the robot if it gave me advice

Confiaría en el robot si me brindara algún consejo

In addition to the UTAUT constructs, we also evaluated. The information presented, resources utilized and presenter attitude. It is also required to evaluate the context where the presentation is applied.

5 Results


Analyzed the preselected constructs previously mentioned identified Perception of Enjoyment (PE), Perception of Sociability (PS), Intention to Use (IU) y Trust (T), using variance analysis sorted by group, gender, and age.

In order to know the reliability of the four constructs (PE, PS, IU, and T) utilized a Cronbach Alpha Analysis. As described the Table 4, the result of this analysis shows acceptable reliability in all constructs.
Table 4.

UTAUT Cronbach Alpha


Cronbach Alpha














To analyze the data utilized a Likert scale with the parameters between −2 to 2. Considering 2 as the higher value. The different constructs describe a positive perception of the interaction with the Nao. According to Table 5, the higher value identified was IU (M = 1.72, SD = .42), follow by a high PS (M = 1.68, SD = .42), PE (M = 1.59, SD = .43), and a consistent T (M = 1.29, SD .74).
Table 5.

UTAUT descriptive analysis





Std Dev





















The lower mean identified in T requires future research to identify the causes in the decrease of this value. It is also important to consider if the topic presented matched the Nao appeal and the relationship of T when the robot presents mature topics such as the risks of using weapons (Table 6).
Table 6.

Anova analysis UTAUT x Group X Gender X Age













F(3,73) = 2.9

p = .041

F(3,73) = .94

p = .427

F(3,73) = .26

p = .853

F(3,73) = .53

p = .662








F(1,75) = .92

p = .341

F(1,75) = .62

p = .432

F(1,75) = .45

p = .506

F(1,75) = 1.33

p = .253








F(4,72) = 1.59

p = .186

F(4,72) = 1.66

p = .170

F(4,72) = .86

p = .495

F(4,72) = .44

P = .781

Alpha  =  .05

According to the variance analysis, the perception of the robot among groups was not significant. This value is important to be considered in the evaluation of consistence of the information presented among the four groups. As mentioned before, one of the main challenges of the research was creating a script consistent to be present among all four groups. According to the variance analysis, the only significant difference identified among groups was PE (F (3, 73) = 2.9 p = .041). Future research is required to identify if the context elements might have impacted this value. However, the post-hoc analysis of each group criteria revealed no significant relationships identified. Finally, no significant difference of gender or age was identified between the constructs of the experiment.

5.2 Context

To evaluate other elements of the context where the robot performed, we also evaluated: the attitude of the presenter, accessibility of the information presented, and utility of the information presented. Using parameters between −2 to 2, considering 2 as the higher value. Table 7 describes the performance of each variable:
Table 7.

Context analysis





Std Dev

Affordability of the information





Presenter Attitude





Usability of the information in daily life





The usability of the presented information evaluated as M = 1.74, SD = .52. The presenter attitude performed a M = 1.69, SD = .47. Finally, the usability of the presented information in everyday life registered a M = 1.56, SD = .66. It might be possible to consider evaluating element to increase the utility of the information presented in the context of everyday life.

5.3 Topic

A one-tailed two-sample independent means t-test, equal variance assumed, revealed that after the presentation the topic impact recorded (M = 1.7, SD = .54), it was not significantly higher without the presentation value (M = 1.54, SD = .81), F(3, 73) = 2.06, p = .113. Even though it was detected an increase in the attitude to lower the use of violence and weapons in our society, that increment was not significant enough to reject the null hypothesis.

6 Conclusion

Using methodologies such as Sketching, and Story Telling from the User-Centered Design applied to the Human Interaction Robot it is possible to offer a higher quality of interactive experience with the robot. Identifying alternatives to possible improvement areas of robots such as NAO. For example, it was essential to evaluate the need to create alternative solutions that highlight the emotional expressions in the limited facial and oral expression in the NAO robot. Using these techniques brought more positive feedback in the testing groups, and participants of the experiment PE, PS, IU, and T.

The applied research demonstrated a functional application of the UTAUT framework in the evaluation of HRI. By using UTAUT constructs, UCD design techniques, and WoZ interaction, we could design an appealing interaction of using the Nao robot as co-presenter of information. It was also possible to evaluate how consistent was the interaction among the different groups—elaborating an interaction inclusive in factors such as age and gender. Also, by using UTAUT constructs simplified the evaluation process by making the constructs more reliable and confiable.

It is required a deeper evaluation in future research of the impact on the perception of T in the interaction with robots. It might require to study the different levels of T using robots to present information. Identifying the effect in T of the presence of the robot in the presentation and the topic.

About the impact of the presentation, it is necessary to mention that no statistically significant variability was identified. However, an increased mean can be observed in the population during the evaluation. The high value in the mean presented in the assessment before the presentation shows high awareness of the participants regarding the issue of the risks of the use of weapons. This element might be considered in order to evaluate the effectiveness of the presentation.

Finally, it is required to highlight the positive perception of the interaction with the robot perceived among the groups in the context of assisting the presentation of information.



This work was supported by ECCI-UCR (Escuela de Ciencias de la Computación e Informática) and by CITIC-UCR (Centro de Investigaciones en Tecnologías de la Información y Comunicación) both at Universidad de Costa Rica (UCR). Grand No. 834-B7-267. Additionally, thanks to the User Interaction Group (USING) for providing ideas to refine and complete the research. Finally, thanks to the Fundación Fundamentes for the collaboration collection the data.


  1. 1.
    Oh, C.G., Park, J.: From mechanical metamorphosis to empathic interaction: a historical overview of robotic creatures. J. Hum.-Robot Interact. 331(10) (2014)Google Scholar
  2. 2.
    Ramírez-Benavides, K., López, G., Guerrero, L.A.: Designing tools that allows children in the early childhood to program robots. In: Zaphiris, P., Ioannou, A. (eds.) LCT 2017. LNCS, vol. 10296, pp. 71–89. Springer, Cham (2017). Scholar
  3. 3.
    Bravo, F.A., González, A.M., González, E.: Interactive drama with robots for teaching non-technical subjects. J. Hum.-Robot Interact. 6(2), 48–69 (2017)CrossRefGoogle Scholar
  4. 4.
    Baxter, P., Ashurst, E., Read, R., Kennedy, J., Belpaeme, T.: Robot education peers in a situated primary school study: Personalisation promotes child learning. PLoS ONE 12(5), 1–23 (2017)CrossRefGoogle Scholar
  5. 5.
    Torta, E., Oberzaucher, J., Werner, F., Cuijpers, R.H., Juola, J.F.: Attitudes Towards Socially Assistive Robots in Intelligent Homes: Results From Laboratory Studies and Field Trials. J. Human-Robot Interact. 1(2), 76–99 (2013)CrossRefGoogle Scholar
  6. 6.
    Wood, L.J., Dautenhahn, K., Rainer, A., Robins, B., Lehmann, H., Syrdal, D.S.: Robot-mediated interviews - how effective is a humanoid robot as a tool for interviewing young children? PLoS ONE 8(3), e59448 (2013)CrossRefGoogle Scholar
  7. 7.
    Dautenhahn, K.: Socially intelligent robots: dimensions of human-robot interaction. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362(1480), 679–704 (2007)CrossRefGoogle Scholar
  8. 8.
    Fortunati, L.: Social robots from a human perspective. Soc. Robot Hum. Perspect. 73, 1–144 (2015)Google Scholar
  9. 9.
    Dautenhahn, K.: Roles and functions of robots in human society: implications from research in autism therapy. Robotica 21(4), S0263574703004922 (2003)CrossRefGoogle Scholar
  10. 10.
    Vlachos, E., Jochum, E., Demers, L.-P.: The effects of exposure to different social robots on attitudes toward preferences. Interact. Stud. 17(3), 390–404 (2016)CrossRefGoogle Scholar
  11. 11.
    Usability Body of Knowledge, “Wizard of Oz” (2012). Accessed 13 Apr 2017
  12. 12.
    Riek, L.D.: Wizard of Oz studies in HRI: a systematic review and new reporting guidelines. J. Hum.-Robot Interact. 1(1), 119–136 (2012)CrossRefGoogle Scholar
  13. 13.
    Heerink, M., Kröse, B., Evers, V., Wielinga, B.: Measuring acceptance of an assistive social robot: a suggested toolkit. In: Proceedings of IEEE International Symposium on Robot and Human Interactive Communication, pp. 528–533 (2009)Google Scholar
  14. 14.
    Oshlyansky, L., Cairns, P., Thimbleby, H.: Validating the Unified Theory of Acceptance and Use of Technology (UTAUT) Tool Cross-CulturallyGoogle Scholar
  15. 15.
    Heerink, M., Kröse, B., Evers, V., Wielinga, B.: Assessing acceptance of assistive social agent technology by older adults: the almere model. Int. J. Soc. Robot. 2(4), 361–375 (2010)CrossRefGoogle Scholar
  16. 16.
    Fundamentes, “¿Quienes somos?—Fundamentes” (2019). Accessed 11 Feb 2019

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Universidad de Costa RicaSan JoséCosta Rica

Personalised recommendations