1 Introduction

Serious games are defined as “games that do not have entertainment, enjoyment or fun as their primary purpose” [32]. Instead, they offer educational content to users in an enjoyable way by simulating scenarios which promote learning. In this sense, serious games stimulate the “flow state” i.e. the mental state associated to the enjoyment of an activity. This concept is associated to the mental state that people feel when they are completely engaged in an activity and have an optimal experience while performing it, leading to a feeling of joy that emerges from a sense of control on the action [15]. According to this theory, it is conceivable that the game can elicit enjoyment and pleasure in children [58]. Moreover, through the action of playing, children can acquire new information and in a more long-lasting perspective. The result is that they can change their attitude toward a specific skill or knowledge [28, 4, 45, 47].

Furthermore, by applying the theory of “reasoned action” [5], people can change their attitude and get positive beliefs especially when they perceive their sense of efficacy by feeling able to control and behave accordingly. A way to increase one’s own sense of efficacy is concretely try and experience the conduct as in the case of the so called learning by doing strategy in a positive context, for example, by means of serious games. On the other side, serious games represent a class of persuasive technologies that can be effectively employed to promote positive behaviours [17, 26, 72]. Indeed, any technology, in addition to conveying positive emotions, sense of efficacy and belief on usefulness, should rely also on relational factors like trust and empathy [73] [54].

Based on all these observations, we can state that serious games represent an effective way to acquire knowledge and problem-solving skills with better performance and long-lasting attributes than traditional approaches [32]. In this work we focus on the use of serious games in the field of ecological sustainability, that represents one of the most pressing problems in the world today. There is an increasing awareness of the need of adopting a correct lifestyle sustaining an ecologically-balanced environment. Reducing and avoiding the negative impact of waste on the environment is a basic and starting point for supporting the environment, being the waste recycle an important contribution to limiting greenhouse gas emissions. In this context, serious games can be useful tools that put together engagement and training for the adoption of a sustainable behaviour especially in children [32, 41, 63, 65, 14, 64, 70].

In addition to serious games, social robots have also recently emerged as effective technologies [74]. Recently, social robots that have been successfully used in education [2].

Social robots are embodied autonomous intelligent agents that interact with people in everyday environments, implementing social behaviours typical of humans [43]. In the context of educational technology, social robots have been shown to be effective in supporting knowledge and skills acquisition [41] by engaging users during the learning process [70]. Moreover, they have been used successfully in supporting children with special needs in learning and acquiring new skills [14, 52].

With the idea of combining the power of serious games and social robots as effective technologies for promoting attitute toward ecological sustainability, we designed and implemented a serious game to be played using the social robot Pepper aiming to help children in learning how to recycle waste material properly. The game, called PeppeRecycle, has been designed as a challenge between a user and the Pepper robot, that plays the role of a game opponent. The mission of the game is to recognize the material of a waste item and select the correct waste bin in which to throw it. To endow the robot with the capability of seeing the waste item and classifying its material correctly, we developed a recognition module based on a Convolutional Neural Network (CNN) that is trained to classify different classes of waste material. A preliminary version of PeppeRecycle was presented in [12]. During the development stages of the game, a first evaluation of the preliminary version was made informally with 12 participants of which only 4 were children [18]. The results of this preliminary formative evaluation were used as starting point to refine the game and improve its user experience. In this new study we present the complete version of the game and describe a formal experiment that has been carried out to test its effectiveness with a sample of 51 children. The experiment was designed to achieve the following goals:

  1. 1.

    To measure the user experience during the game;

  2. 2.

    To evaluate the interaction of children with the robot;

  3. 3.

    To evaluate cognitive and affective elements that can form the attitude toward ecological sustainability and the children recycling skills.

The results of the experiment conducted in this work confirm that playing the game with Pepper actually represents a case of “learning with enjoyment”. We observed that the use of PeppeRecycle improves children’s flow in recycling attitude since they generally enjoyed themselves during the game. Also, the game helped children to better memorize useful information and to increase their sense of mastery (they felt more able to recycling). Finally, the use of real objects as waste items during the game turned out to be a successful approach since it is perceived as highly trustworthy and believable.

2 Background

The development of the PeppeRecycle game leverages a combination of methodologies coming from different research areas, namely serious games, social robots and computer vision. In the following we investigate the state of the art of each area with a specific focus on the task of waste recycling, in order to understand how to face the challenge of improving children’s environmental sustainability attitude through the game.

2.1 Game and Learning

Considerable empirical evidence confirms that gaming is effective in improving the acquisition of both knowledge and competences in several contexts [11, 56, 14, 44, 52, 70, 4, 22, 24, 47, 8]. The gaming dimension allows learners to be more motivated and engaged in the learning process, and it fosters a deeper level of understanding [27]. Also, during the game, the students have the chance to gain experience in a protected (real or simulated) environment that allows them to fail and correct their behavior to succeed in a task. This is one of the main features of constructivist approaches [58]: learners builds their own knowledge out of their experiences. The game allows learners to gain experience on a specific topic in a more realistic fashion and, thus, to acquire effective knowledge on it. In particular, following the constructionist theory [28] students learn concepts from working with materials or building artifacts, rather than by direct instruction. In the context of environmental sustainability, the use of this approach can improve the attitude to adopt eco-sustainable behaviours [49].

The Game4sustainability portalFootnote 1 is one of most representative examples of using serious games for sustainability. The main goal is to support sustainability teaching and learning using serious games since it is widely recognized that the sustainability education is a basic skill for new generations. A number of digital and non digital games have been classified according to learning objectives and age range of intended users. Go Goals!Footnote 2, for example, is a table game developed by the UN Regional Information Centre for Western Europe (UNRIC) to help children aged 8-10 to understand the 17 Sustainable Development Goals of Agenda 2030. The game uses the same approach of the classic game “Snakes and Ladders” to induce participants to think about sustainable behaviours, such as how to beat world hunger, how to use green energy and how to pro1mote responsible consumption and production.

Other solutions employ more advanced technologies. An example is ARGY [6], which uses Augmented Reality to promote consumption of electronic devices in a greener manner. Virtual Reality has also been successfully used to train users in selecting and sorting the waste [46]. The users are asked to drag the rubbish in the right bin placed in the room, and the game is able to recognize the answers using a motion sensing input device. Mobile technologies and humanoid robots are used in [61] to let pupils acquire topographical orientation and recycling skills. The game runs on a mobile device that children use to help the robot MecWilly to associate the garbage to the right recycling container. The results were positive both in terms of engagement and effectiveness.

The empirical evidence given by the literature on serious games confirms the validity of this approach to acquire sustainable behaviors, even if developed using different kinds of technologies. The game dimension allows the acquisition process to be more engaging, while technology helps to promote and enhance the physical and cognitive perception of the game. The present work contributes to the advancement of the state of the art in this field, by developing a serious game for recycling to be played with a social robot, which is able to promote social interaction with humans.

2.2 Social Robots

A social robot is a physically embodied, autonomous agent that communicates and interacts with humans on a social and emotional level. Social robots represent an emerging field of research focused on developing a “social intelligence” in order to maintain the illusion of dealing with a human being [3, 25]. They have been successfully applied in several domains such as elderly care [9, 59], autism therapy [55, 57, 77, 80], education [51, 66], public places [39], domestic and work environments.

To be believable, social robots have to exhibit social intelligence and adapt their behavior to the current situation, therefore they should be aware of the context and of the user’s characteristics, and understand the user’s intentions, emotions, interests and so on. Based on the acquired information, the robot should make decisions on how to react appropriately according to different social situations and according to its role [1]. To increase the robot social believability, personalization is a key factor. Indeed, personalization has proven to be quite effective in terms of improving the acceptability and the trust of a robot since users perceive the robot as a more socially competent companion [38]. For instance, the robot could adapt the level of formality and the lexicon to the user’s age and gender [19]. In this regard, robots capable of exhibiting sociability and achieving widespread societal acceptance are being used more and more often in human-centered environments.

The use of social robots in education has been shown to be successful for increasing attention, engagement, learning gain and compliance, which are critical components of successful learning [33, 43, 62, 68] [30]. In [50], for example, the NAO robot was introduced as teaching assistant to revise a topic in Mathematics. The comparison with a human teaching assistant revealed no significant differences in tests scores, but children were much more engaged when interacting with the NAO robot. In [75] the authors investigate the effectiveness of using a social robot to help young children to learn English vocabulary as a foreign language. Also in this case, the NAO robot was used, and the results confirm that children can remember better with the help of a robot than with traditional approaches. Moreover, the experiment measured higher learning gain when the robot produced iconic gestures to represent the target word in an iconic way. In [48] a new educational robot, called Kiddo, is used. Kiddo is conceived to assist young child in acquiring handwriting skills, thus it is equipped with a robotic arm and a whiteboard where children and the robot can write in a shared environment.

On the overall, social robots open up new possibilities in the learning process that were previously unavailable, leaving space to explore novel aspects of teaching. Their success in education relies on the fact that children tend to be affected by social robots [76] and to connect with them on a friend-like basis [31].

As far as their role in the educational process is concerned, social robots often embody the role of tutor, facilitator or peer [31, 81]. When robots are introduced in learning games as peers, they may play the role of a supporter, a companion or an opponent of the child, challenging him to achieve a higher performance (e.g. [32, 58, 62]). When robots act as a companion, they may establish a collaboration with the child during the learning process by building a social relationship [71, 77] and helping the player by suggesting serious game activities or by supporting the decision process. When they are introduced as opponents in the game, it is desirable to design the robot playing behavior in such a way that the robot could provide affective and attention cues, since the emotional behaviour embedded in the robot helps the users to have a better perception of the game [40].

Moreover, the robot should make some mistakes so as to appear less knowledgeable than the child in order to avoid frustration and evoke caregiving behaviour in the child [71]. In such cases, the child may act as a tutor, correcting the robot’s mistakes [4, 68]. As concerns compliance, the literature on robot persuasiveness clearly demonstrates that robots can promote “virtuous” compliance toward the technology requests [29, 67], especially when the persuaded subjects are children [33].

In summary, promising results have been found for the use of social robots as tutors in educational contexts. Social robots open up new possibilities in teaching that were previously unavailable, such as the robot taking the role of a peer in serious games. According to this view, in our recycling game, Pepper is one of the players and it plays the role of the opponent.

2.3 Waste Recognition

Nowadays, trash has become a serious problem for the ecosystem of our planet. An answer is recycling by re-using the materials. Correct identification of recyclable waste is the first stage of recycling. Unfortunately, most people are not able to identify and separate waste correctly. This makes it more difficult to recycle waste due to the high economic cost of a further separation process. Hence there is an urgency of developing automatic tools that can help or educate people (especially children) to correctly separate waste material. Despite the importance of the problem, there are few works in the literature that address the automatic recognition of waste material for recycling purposes. Solving such a task requires the development of robust methods for object detection and material recognition that are invariant to different shapes, orientation, colours, lighting conditions and occlusions. Humans can solve this problem using their senses such as touch but above all vision. They mainly leverage visual cues to make the decision of how to differentiate the waste material. This human perception process evokes the idea that an artificial intelligent system or a robot could succeed to accomplish the same task by using computer vision techniques.

A pioneer work was presented in [16]. The authors designed an autonomous recycling robot which can separate objects made from plastic, glass, or aluminium. It classifies waste using a metal detector and a switch that triggers by weight. In [53] an intelligent waste separator (TrashCan) is developed that is intended to replace the recycle bins. It incomes waste and places it into different containers by using a multimedia embedded processor, image processing, and pattern recognition in order to make the selection of garbage. This prototype can identify aluminium cans, plastic bottles, and plastic cutlery. In [35] the authors use computer vision and image processing to solve the problem of automatic recognition of empty recyclable containers (bottles and cans). A robot called Recyclebot is proposed in [13]. The robot automatically segregates recyclable and non-recyclable waste and tries to create awareness among people about the benefits of recycling. In [79] they develop a waste recycling robot for nails and screws. They use fast R-CNN methods to find scattered nails and screws in real time, so that the robot can automatically recycle these very specific materials. All the above mentioned works developed specific-purpose robots to be used in industrial environments or commercial applications. Hence, unlike social robots, they are not suitable to encourage people to recycle nor to educate children to understand how to adopt a sustainable behaviour by reducing the impacts of waste on the environment. So far, to our knowledge, the capability to robustly perform separate waste recycling has not been implemented in social robots. The present work represents the first attempt to equip a social robot with this capability. Specifically, our goal was to give Pepper the ability to recognize the material of objects as a basic task necessary to accomplish the recycling game.

The image classification module developed for PeppeRecycle leverages a Convolutional Neural Network (CNN) that is properly trained to recognize different materials. CNNs have been successfully applied to solve many computer vision problems involving object detection and image classification as basic tasks [42, 69]. CNNs are a class of deep learning models that extend fully connected feed-forward Neural Networks by introducing additional operations to the activation function of neurons, specifically convolution and pooling [37]. Neurons in a convolutional layer apply a set of filters along the whole input image to process small local parts and extract a map of relevant features. The pooling layer serves to compress the feature map produced by the convolutional layer. This makes features translation invariant and tolerant to minor differences of patterns in the input image. Higher layers use more broad filters that work on lower resolution feature maps to process more complex parts of the input. Top fully connected layers finally combine features coming from all previous layers to perform the classification of the overall input image. This hierarchical organization, which resembles the hierarchical stages of the visual processing in the human visual cortex, enables a CNN to extract local structures in the input image using supervised learning algorithms.

In particular, to accomplish the task of waste material classification we trained a state-of-art CNN, namely the VGG-16 neural network. The network was trained to classify images according to six classes of waste material contained in the scene, namely glass, paper, cardboard, plastic, metal, and organic trash. Implementation details and experimental results of the network training are given in Sect. 4.

Fig. 1
figure 1

Pepper playing the game: a showing waste bins on its tablet, colors are those commonly used in Italy; b observing a plastic bottle for recognition; c showing scores on its tablet

Full size image

3 Game Design and Playing Rules

The designed game has a twofold aim:

  1. 1.

    to educate children on the importance of recycling and train them to adopt a sustainable behavior;

  2. 2.

    to investigate whether the interaction with a social robot and the use of real waste material may favor the adoption of sustainable behavior in real-world scenarios.

The guideline during the design of the game was to enable the player to learn about recycling in an engaging and realistic way. The game has been designed as made up of two parts. The first one is dedicated to providing information about how important recycling is for our planet. This was made following the storytelling approach, which is not discussed in this paper. The second part of the game consists in a recycling challenge between a user (the child) and the Pepper robot. We adopted the challenge as game strategy since it has been proved to be more effective in terms of learning gains than other strategies. Actually, the challenge created by the game is a key factor both for engagement and for learning through the game [8, 58]. According to psycho-pedagogical theories such as the Malone theory [44], students are more engaged if they are involved in a challenge. Actually, the challenge. In particular, the presence of a challenge component in computer-based learning environments makes such environments more engaging. During the recycling challenge of PeppeRecycle, the Pepper robot plays the role of a game opponent. This choice is driven by some research works indicating that this makes the gameplay more enjoyable and increases engagement [36].

The mission of the game is to recognize the material of a waste item and select the correct waste bin in which throwing it. The game play requires that Pepper and a child challenge each other to recycle correctly the waste disposed on a table. The waste items may belong to the following recycling categories: 1) glass, 2) paper and cardboard, 3) plastic and metals and 4) organic trash. A human judge, who knows the recycling rules, has been introduced in the game to give a feeling of fairness, and to allow both players to verify the correctness of their answers.

3.1 Game Rules

The game consists of 3 rounds for each player, so that each player has 3 attempts to guess the correct waste bin. At the beginning, the game engine selects randomly the player who has to start the game. When the game starts, the human judge selects a waste item, among ten items available on the table. We used the following 10 items during the experiment: 2 glass bottles, 1 used sheet of paper, 2 fruit juice carton packs, 2 plastic bottles, 1 banana peel and 2 cans. Both the players (Pepper and the child) have to guess the proper bin in which the object should be placed.

When it is the turn of the child, he gives his answer by touching the icon of the proper waste bin displayed on the Peppers tablet (Fig. 1a). For the waste bins we use different colours commonly used in Italy to recycle different materials: brown for organic trash, blue for paper and cardboard, yellow for plastic and metals, and green for glass. Once the child has selected the bin, Pepper asks the human judge to confirm whether the choice of the child is correct. Then Pepper waits for the judge to answer Yes Pepper or No Pepper.

Fig. 2
figure 2

Architecture of the VGG-16 CNN used for image classification. In green the fine-tuned layers, in red and dashed the replaced layers

Full size image

When it is the turn of Pepper, the child puts the waste item in front of the robot’s head camera that is close to its eyes (Fig. 1(b)). Pepper, using the image classification module, tries to recognize the material of the item and takes a decision about the appropriate waste bin. Pepper says the sentence “Judge, this object must be placed in the blue/yellow/green/brown bin”. The judge may answer Yes Pepper or No Pepper depending on whether Pepper has guessed or not the correct bin.

Both players get one point for each correct answer (Fig.1c). To increase Peppers sociality and believability some sentences will be pronounced according with the users results. When the child gives a wrong answer, the robot says motivational sentences like try again”, ”next time you will make it. In case of correct answer, Pepper congratulates the user using sentences like you are a serial recycler. In some cases the pronunciation of the sentence is accompanied by appropriate gestures. For instance, when Pepper is recognizing the waste item it performs a typical thinking gesture by holding its fingers at the forehead and temples and reproduce a ticking sound effects. Or, when the child wins the game, Pepper performs the clapping gesture and reproduces the clapping sound. In case of a draw, Pepper says “Cool, we are both talented in recycling, next time I hope to do better than you”. This is made to motivate children to play again to beat the robot.

Fig. 3
figure 3

Sample images for each class in the dataset

Full size image

To reinforce the user knowledge about recycling, Pepper gives some additional information to the user after each turn, depending on the last action made in the game. As an example, if the last item used in the game is a plastic bottle, Pepper says Did you know that a plastic bottle takes from 100 to 1000 years to decompose itself?. The game ends after three turns. According to the collected scores, Pepper declares the winner by showing the final score on its tablet. Then it plays an appropriate behavior. Since Pepper is equipped with a tablet, we use it to reduce reliance on automatic speech recognition since the interaction can be controlled also using the touch-based interaction on Pepper’s tablet.

4 Waste Recognition

To make Pepper able to recognize the waste material during the serious game, we created a computer vision module based on a Convolutional Neural Network. In particular, the VGG-16 neural network [69] pre-trained on the ImageNet dataset [20] was considered for this task. VGG-16 is a deep convolutional neural network used for image classification. The original architecture and its blocks are shown in Fig. 2. We used a transfer learning technique based on fine-tuning to train the network on the Garbage Classification Dataset that is publicly available on Kaggle.Footnote 3 This dataset contains 2527 waste images belonging to 6 different classes: glass, paper, cardboard, plastic, metal, and organic trash. Each class is encoded by an integer number in the range from 1 to 6.

For each class the images are distributed as follows:

  1. 1.

    Glass: 501 images

  2. 2.

    Paper: 594 images

  3. 3.

    Cardboard: 403 images

  4. 4.

    Plastic: 482 images

  5. 5.

    Metal: 410 images

  6. 6.

    Trash: 137 images

The pictures of the objects were taken by placing them on a white background with variable light coming from both natural sunlight and artificial room light. The objects have also been captured in different positions, thus introducing variability in the dataset. Each picture was resized to \(512\times 384\) pixels. An example of images for each class is shown in Fig. 3.

The creators of the dataset divided it into a training set, a validation set, and a test set, according to the following proportions 70:13:17. Hence the training set consists of 1768 images, the validation set of 328 images and the test set of 431 images. A preprocessing step was required to make the images processable by the neural network. All the images were resized to \(224\times 224\) pixels and the RGB average of the ImageNet dataset was subtracted from them. Due to the reduced size of the training set, only the weights of the last convolution block of the network were updated. Moreover, we replaced the three dense layers in the last block with two new dense layers. The first one with 1024 neurons and ReLU as the activation function, the second one with 6 neurons and Softmax as the activation function. This last layer is used to compute the output class. For the training phase, the Adam optimizer [34] was used with a learning rate of 1.25e-4. The loss function is the categorical cross-entropy. The maximum number of training epochs was set to 30. During the training phase, we monitored the accuracy and the value of the loss function on both the training set and the validation set at each epoch. We decided to consider only the model with the highest accuracy measured on the validation set. At the 29th training epoch we obtained the best performance, with an accuracy value of 0.89, therefore we chose this model for the development of the waste recognition module to be deployed in the game.

Table 1 Performance Metrics on Test Set
Full size table
Table 2 Confusion matrix of waste image classification
Full size table

To measure the performance of this model on the test set we considered standard metrics: accuracy, precision, recall and F1. The performance metrics for each of the 6 classes and the average performance are shown in Table 1. It can be seen that in general the selected neural network model has a good classification accuracy on the test set. Only in case of the “trash” class we observe a low precision that is essentially due to the limited number of training images belonging to this class. Indeed, from the confusion matrix (table 2) it can be seen that only 2 out of the 17 testing images of organic waste are wrongly classified as “paper”. Hence overall we can consider the waste classifierFootnote 4 sufficiently reliable for the purpose of the game.

5 Evaluation of the Game

This section describes the experimental study performed to measure the efficacy of the PeppeRecycle game in terms of applied strategies and user experience during game play. The experiment was also designed to understand the potential of the proposed approach in terms of user’s perceived efficacy in recycling.

In our previous work [12] we performed a formative evaluation study aiming at testing the usability and experience during the game play with Pepper. In that study, the considered sample was composed of 12 high school students, 4 children (aged from 7 to 11 years) and 5 adults. All the users played the game with Pepper in the wild during different open days events organized in the Department of Computer Science in Bari. At the end of the game session, we briefly interviewed each participant by asking them to evaluate their experience with Pepper on a scale from 1 (very low) to 5 (very high).

The overall results were positive since all users felt engaged in playing the game with the robot, and they considered Pepper to be socially believable. In addition, this preliminary evaluation showed that children were really enthusiastic to play with Pepper using real objects and they asked to play the game several times. The key findings of the preliminary formative evaluation were that the combined use of Pepper, as opponent player, and real objects was the main strength of the designed game. However, the sample used in [12] was too small to make general claims, thus requiring further investigation. Then, after this preliminary evaluation, a few minor problems were corrected in the game and a new experimental study was set up, hence these data were not used in the experiment described in the present work but they represented the result of a formative test for developing the current version of the game. The new study, that is the focus of the present work, was aimed at going deeper in the evaluation by measuring the user experience during the game play, the experience with the robot, and the cognitive and affective elements that can have an impact on the attitude toward ecological sustainability and on recycling skills.

Fig. 4
figure 4

An example of the questionnaire item

Full size image
Fig. 5
figure 5

Recycling Skills: percentage of correct answers in choosing waste bins colour

Full size image

5.1 Participants and Procedure

The study involved 51 children, 25 females and 26 males, aged from 7 to 9 y.o. They were students attending three different primary schools in the metropolitan area of Bari in Italy. All children played and interacted with Pepper using real waste material. In this specific case, the organic trash was excluded due to hygienic reasons. Before running the game, a pre-test session was performed to collect the following information about the children: their assent and the permission of their parents to participate in the study, their socio-anagraphic characteristics, their confidence with robots, the importance they give to the environment, and self-efficacy in recycling. Then, each child was invited to play against Pepper individually. At the end of the game session, the participants were asked to evaluate their experience with PeppeRecycle by answering a questionnaire created ad-hoc and based on theoretical assumptions concerning the cognitive, emotional, relational factors that induce a robot to increase virtuous attitude toward environment [17, 73, 26]. Specifically, we investigated the following issues:

  1. 1.

    Recycling skills: capability to link the waste material to the correct bin colour;

  2. 2.

    User eXperience (UX) while playing: likability and easiness in playing with Pepper, how much they achieved in doing things, how much the game was easy to understand, the easiness in talking with Pepper, the likability of robot speech, clarity, the usefulness of the images, the easiness and clarity of the game, the easiness to remember the tablet usage, the usefulness of the real waste objects, and the difficulty of remembering the game dynamics;

  3. 3.

    Emotional factors: concentration, easiness, enjoyment, feeling able, challenge;

  4. 4.

    Relational factors associated to Pepper: empathy, trust, credibility, sympathy;

  5. 5.

    Attitude toward ecological sustainability: how is it useful to recycle, how much they feel able to recycling, how much they are willing to recycling even if this means losing time, how important is nature, how much new information they have acquired, how much they think that their behavior affects the environment.

The children were asked to give a score between 1 (very low) to 4 (very high). To help children to answer the questionnaire, very simple questions were formulated and the answers were represented using some expressive emoticons (Fig. 4). The complete questionnaire (translated in English) is attached as an annex to this paper.

5.2 Results and Discussion

In this section we discuss the results obtained by analysing the data collected in the new study for each investigated issue.

5.2.1 Recycling Skills

First of all, we evaluated whether the children acquired recycling skills thanks to the game. To do this we asked them to link each material (plastic, glass, and paper) with the right colour of the bin (yellow, green and blue). Figure 5 plots the frequency of correct answers given by children. It can be seen that children were very skilled to differentiate waste material, with percentage of correct answers equal to 84.3% for plastic and metal, 90.2% for glass and 84.3% for paper.

These results confirm the expectation on the positive effects of using a social robot combined with a serious game on improving the attitude toward ecological sustainability in children.

5.2.2 User EXperience (UX)

We evaluated the UX of PeppeRecycle in terms of the following items: likability, easiness, achievement, understanding of robot speech, easiness in talking, likability of Pepper speech, clear images, useful images, easy to remember, clearness of the game, usefulness of the game, usefulness of employing real objects, difficulty to remember. We performed repeated Anova measures \([F(13, 48)= {37.49}; p< {0.000}]\) and the results (Fig. 6) pointed out that children showed high levels of appreciation for the game interaction. Children felt high levels of likability and easiness in understanding Pepper and in playing and discussing with Pepper. The children also found the pictures on Peppers tablet very clear. Moreover they found the real objects used during the interaction very useful. The lowest score (0.66) corresponding to the question How much the game has been difficult to remember confirms that the game was judged easy to play by the children. On the overall, we can conclude that the UX of PeppeRecycle was positive.

Fig. 6
figure 6

User experience: average values of the considered UX items

Full size image

5.2.3 Emotional and Relational Factors

As concerns the emotional aspects of the interaction with Pepper, the Anova repeated measures \([F(4, 49)= {74.55}; p<{0.030}]\) showed good results. This can be seen from figure 7a that plots the average values of children’s concentration, easiness, challenged, sense of mastery (item able in the figure) and enjoyment.

Fig. 7
figure 7

a Average values of emotional factors. b Average values of relational factors

Full size image
Table 3 Correlations between UX and Relational factors. \(^\bullet \)Significant correlation at level 0.01. \(^\star \)Significant correlation at level 0.05
Full size table

Also the evaluation of the relational factors (Fig. 7b) gave high values for the majority of the children involved. Pepper was perceived as nice, trustful and credible. Moreover, children reported very low levels of negative emotions (\(\hbox {angry}=0.59\)) indicating that they did not feel angry while interacting with Pepper.

We point out that the emotional and relational factors emerging from the interaction with Pepper show that children created a state of social flow [78] with the robot. Indeed, as depicted in Fig.7a, children reported low levels of perceived challenge that generally are considered the trigger of individual flow experience [15]. However, the low perceived challenge is compensated by the positive experience lived in interacting with the robot, thus enhancing the general emotional evaluation. In addition, we observed that the emotional and relational factors are not affected by gender as confirmed by the Anova analysis.

Then we performed a correlation analysis between UX and relational factors based on the Pearson correlation coefficient \(\rho \). Table 3 summarizes the results. It can be seen that the UX factors are highly correlated to the perceived level of trust and credibility of Pepper. The more children felt likeability, clearness and usefulness of both the game and the objects used during the interaction, the higher the trust and credibility in Pepper. To a lesser extent, also the empathy is correlated to the easiness to talk with Pepper. Anger is inversely correlated to the easiness to talk with Pepper, to the understandability and the clearness of the game. Based on these results, we can conclude that the users emotional dimension is mainly associated with the communicative aspects of PeppeRecycle.

Fig. 8
figure 8

Average values of Attitude toward Ecological Sustainability (single item)

Full size image

5.2.4 Attitude Toward Ecological Sustainability

We measured also some cognitive beliefs that can affect the overall attitude toward recycling (Fig. 8). The Anova repeated measures \([F(6, 48)={5.79}; p<{0.00}]\) showed that at the end of the game, children expressed high agreement on the following items: importance of nature (2.88), how much it is useful to recycle (2.88) and ’how much I feel able to recycle’ (2.70). Then, to a lesser extent, but still with high levels, they felt to have acquired new information (2.60) and to affect the environment with their behaviour (2.56).

5.2.5 Game Effect

Finally, we compared the values of some indicators before and after the session game. We considered the following indicators: enjoyment, acquisition of new information and perceived efficacy in recycling. Figure 9 plots the average values of these indicators before and after the game. It can be seen that the level of childrens enjoyment increased significantly after the interaction with Pepper \([F(2, 48)={10.12}; \textit{p}<{0.00}]\). Also there was a significant increase in the acquisition of new information \([F(2, 48)={15.17}; p<{0.00}]\) and in the perceived efficacy in recycling \([F(2, 48)={5.79}; \textit{p}<{0.05}]\). Hence we can conclude that positive emotions, in particular enjoyment, triggered by Pepper together with the new received information about recycling contribute to the improvement of children’s perceived efficacy in sorting waste.

Fig. 9
figure 9

Game effect: Average values of Enjoyment in playing with Robot, Informativeness evaluation and Self-Efficacy in Recycling

Full size image

5.3 Preliminary Model of Children’s Attitude Toward Ecological Sustainability

In order to better explain which factors strongly contributed to the attitude toward ecological sustainability of children when they played with PeppeRecycle, we performed a regression analysis by including the emotional and relational factors. To this purpose, we decided to aggregate in an unique variable the game UX items and other variables, chosen from our research design, that can significantly be related to the Attitude toward Ecological Sustainability (AtES).

Firstly, we performed a Factor analysis with the UX items, namely likability, easiness, achievement, understanding of Pepper speech, easiness in talking, likability of Pepper speech, evaluation of clearness and usefulness of images, easiness to remember, clarity, usefulness of the game, evaluation of the real objects usefulness, evaluation of the difficulty of remembering. The factorial analysis confirmed that the game UX can be considered a unique factor since it explained the \(31\%\) of the factorial variability. Then, we summed the absolute value of the UX items having high factor loadings \((>{0.030}\)) to obtain an unique index. We also tested the reliability of this game UX index by computing the Cronbach’s alpha [7] and we found that it is moderately reliable with \(\alpha ={0.61}\).

With the same procedure, we calculated an unique index of positive emotional flow by summing the absolute values of the items (corresponding to concentration, easiness, able, enjoyment) on the basis of the factorial analysis. Also in this case the factorial analysis confirmed that emotional flow can be considered as a single factor since it is explained by a variance of \(31\%\). This index was moderately reliable with Cronbach \(\alpha ={0.60}\).

Table 4 Linear regression of relational and emotional measures on AtES of the children
Full size table

Moreover, we calculated a general index of AtES by performing a factorial analysis that explained the 55% of the variance in a unique factor. The index was calculated by summing the values of the following six items: the usefulness of recycling, the extent to which the user feels able to recycle, the extent to which the user feels able to recycle even if this requires time, the importance given to nature, the extent to which the users behaviour affects the environment. According to the Cronbach’s alpha, this index is highly reliable with \(\alpha ={0.85}\).

The derived unique indexes were finally used to create a preliminary regression model of childrens AtES. We performed a linear regression based on F-test using the unique AtES index as dependent variable and the unique UX index, the unique index of positive emotional flow and the relational factors implied during the interaction with Pepper (empathy, trust and credibility) as independent variables. The results are shown in table 4. The tested model pointed out a high significance \([F(5, 47)={18.37}; p<{0.00}]\). We found that the variables that strongly contributed to the final AtES of children are the general Positive UX of the game (\(\beta ={0.29}; p<{0.029}\)) and the relational factors, in particular Trust (\(\beta = {0.21}; p<{0.05}\)) and Pepper Credibility (\(\beta ={0.36}; p<{0.008}\)). According to this model, the emotional flow seems to be not significant for the AtES. Thus we can conclude that empathy and the emotional flow during the game did not directly affect the attitude toward ecological sustainability. This can be mainly explained by the fact that the interaction with PeppeRecycle is lived as a social experience with a credible and trustful robot, rather than as a simple joyful companion.

6 Conclusions and Future Works

The environmental sustainability is one of the most discussed challenges facing our society at all levels. Profound cultural changes in people, institutions, business world, associations, and schools are required. The proposed game aims at promoting skills necessary to activate virtuous processes in children. The recycling of waste material is one of the basic skills that people should acquire from early age to make a step forward sustainable behaviour.

In this paper we have presented PeppeRecycle, that is a social robot combined with a serious game on recycling that uses real objects as waste items. Our intended users are mainly primary school students who have to learn to distinguish between different waste materials and how to discard them properly. Results obtained in terms of children emotional and relational factors are encouraging and draw promising possibilities for educational robotics in changing children’s recycling attitude, since both the behavioural and evaluative measures showed virtuous trends. Starting from the actual ability measured at the end of the game session, high success percentages in choosing the right bin for plastic, glass and paper were registered. As to the evaluative measures playing with Pepper, on the overall the game provides a positive emotional experience, since children felt high levels of enjoyment, concentration, easiness and sense of ability, corresponding to an optimal level of engagement with the robot [21]. In particular, children lived a positive experience as users since they evaluated the interaction with Pepper in a highly positive way since it was easy to speak with the robot and also to understand it. Moreover, children evaluated images used in the game, and the game in general, very clear and useful. From the performed correlation analysis emerged that when children have a positive experience as users, their evaluation of the robot in terms of truthfulness and credibility increases. This shows how playing with a robot can promote a persuasive process especially when the game is featured by accurate and clear rules[17, 23]. In addition, the analysis of results showed that the level of enjoyment, and in general the individual state of flow, during the game experience, is related to the clarity of the game rather than being directly associated to the improvement of the general attitude toward ecological sustainability. Nevertheless, the interaction with Pepper promotes a positive state of ’social flow’ thanks to its image of trustfulness and credibility that strongly affect the children attitude toward ecological sustainability.

The obtained results, even if promising from an applicative point of view, can be further enriched by working on two sides: (a) by measuring the self-assessed variables by means of bodily and physiological signals [60] [10] and (b) by planning longitudinal designs that may prove the persistency over time of the acquired positive behaviours. Also, as a future work, we intend to apply some gamification strategies and show a leaderboard to compare children performances. Moreover, a storytelling about environmental sustainability behaviour could be introduced to enhance the learning effectiveness and also to improve the perceived effectiveness of adult subjects. We plan to incorporate all these features into the next version of the game.