1 Introduction

One of the early, and important, steps to the interaction design iterative process is to collect the user requirements of an envisioned solution/product. The importance of this lays in the fact that it enables designers to envision their ideas towards the users’ needs, thus eliminating usability or UX obstacles later in the process. Similarly, when designing technologies for children, their input can be of high value throughout the process [7, 18]. More recently, robotic technologies are increasingly entering children’s daily environments, and they may also enter classrooms environments in the near future as several studies have shown that they have several learning benefits [1, 3, 13]. Indeed, children themselves also tend to portray robots in a familiar context, such as the school [4].

However, there are also some hurdles when involving children, or adults for that matter, in the design of future technologies, such as classroom robots. People find it hard to imagine the use of future technologies since they haven’t experienced them yet and they are not always aware of the state-of-the-art developments in areas such as robotics. Therefore their design ideas may either be very basic, or heavily influenced by pictures in the media, such as films [2]. An additional problem is with imagining the form factors and affordances or the physical attributes of their designs. In this paper we describe two pilot studies to inform the development of an elicitation method focusing on form factors; a first study in which we have asked children to design their own classroom robot using a toolkit, the Robo2Box, and a second study where we have compared the use of the Robo2Box toolkit and clay as elicitation methods. In the remainder of this paper we first discuss related work on design elicitation methods for children, and their involvement in the design of robots in particular. Thereafter we present the design of the Robo2Box and describe the results of the empirical study of children’s robot designs using the Robo2Box. Finally we present the second pilot study where we have compared the use of the Robo2Box with clay as a free-format design material.

2 Related Work

The Human–Robot Interaction (HRI) research community has focused on defining the design requirements and implications for robotic features that include physical and behavioral aspects. Often, investigations are conducted based on laboratory studies using commercially available robots [8], or the focus is mainly on children’s attitudes towards robots rather than how they would envision a robot themselves (for example, [16]). However, some researchers have focused more specifically on children’s own expressions of what is important in the design of robots. Below, we will provide an overview of related research on children’s design methods, and thereafter we will give an overview of related important studies on involving children in robot design.

2.1 Designing with Children

While there is a limited number of studies actively involving children in the design of robots, it has become rather common to involve them in the design of many other technologies. According to Druin [6, p. 18] “Low-tech materials, interviews, design feedback on prototypes, can all be used continually as methods for informants. What is critical, is that the materials and methods are age appropriate for working with children”. However, the choice of materials for prototyping may have quite some influence on children’s output. For example, Scaife and Rogers [14] found that children involved in the design of a multimedia application about ecology using a drawing technique were so focused on the details of their drawings that they did not pay attention to the interaction or how the software would behave. Therefore, the researchers decided to provide the children with “already laminated cut-outs of the organisms which the children manipulate against the background of an empty pond" [14, p. 36]. This new approach was much more productive and helped the children to focus on the behaviour of the ecosystem. On the other hand, Smith et al. [17] showed in three experiments that the introduction of examples can constrain creative generation of imaginative ideas, even after a period of distraction between the exposure to the examples and performance of the creative task, and also when the participants were asked to generate ideas that were different from the given examples. However, Marsh et al. [10] repeated the three experiments of Smith et al. and examined the total output, elaborateness of design, and the noncritical features. They concluded that while the participants indeed included elements of the examples given to them, this did not constrain their creative output in any of the three experiments. Finally, in the context of digital story-telling, Chu and Quek [5] found that contextual digital structures in story authoring systems both act as prompts for task and story engagement for some children, but may also harm the quality of the stories or creative products that the children typically create.

2.2 Children’s Design of Robots

One of the first studies involving children in a design activity to create robots was performed by Bumby and Dauthenhahn [4]. They conducted a series of design sessions with 38 children between 7 and 11 years old in which the children were asked to draw a robot in small groups and then write a story about their robot. Finally, they were observed when interacting with several robots and discussing what they thought of robots. Thereafter Woods et al. [20, 21] investigated children’s views on robot appearance, movement, gender, and personality by letting children between 9 and 11 pick a robot picture and fill out a questionnaire. The pictures displayed different robot attributes: mode of locomotion, body shape, looking like an animal, human or machine, the presence or absence of facial features, and gender. The questionnaire contained questions about the robots’ appearance and personality. The use of this method revealed that children ranked robots on their emotional expression (happy to sad), as well as on their behavioral intention towards humans (friendly to aggressive). However, each of the robot attributes in isolation could not explain why a robot was placed in each of these categories. Woods thus argued that robot designers should “consider a combination of physical characteristics rather than focusing specifically on certain features in isolation” [20]. Furthermore, there was tentative evidence for the Uncanny Valley effect, where children were increasingly positive towards robots that were more human–machine like instead of purely machine-like, but showed a sharp drop in positive attitude towards robots that were very human-like. In general they also concluded that all the robots were rated as being able to understand the children had humanoid features in terms of legs, arms and facial features, except for the Sony AIBO dog. They reasoned that children may have had previous experience with the AIBO and they knew that it has some language communication abilities. The robots that the children thought were unable to understand them were all machine-like in appearance, and did not often have legs, arms, or facial features.

Sciutti et al. [15] asked children to participate in an experiment in which they were asked alone or in groups to order 14 pieces of paper with a robot characteristic on it, from most important to least important. They were asked to imagine building a robot that they could interact and play with. The properties (in no particular order) were: (1) walking, (2) hearing, (3) seeing, (4) having 2 hands and 2 ft, (5) having 2 eyes, (6) having hair, (7) having a mouth, (8) having a head, (9) talking, (10) grasping objects, (11) able to smell, (12) able to feel pinches, (13) able to move, (14) able to stand. Based on this study they found that opinions on what features are considered important in a robot companion change with age: before the age of nine, children pay more attention to a human-like robot appearance; older children and adults are inclined to think more of its skills and functions. They also found that when children have been able to see and interact with a robot they pay more attention to perception and motor abilities in a robot, rather than just its shape. This suggests that actual experience with robots, such as in a robotics class, may influence children’s design requirements.

They also concluded that while robots should have some human-like properties to be easily readable by the users, they should also have the action and perception that go along with these features in humans. A robot should thus balance “humanness” and “robotness”. The humanness enables users to understand the robot’s similarity to humans, while the robotness ensures that users understand that some differences might exist, also in terms of action and perceptual abilities.

Finally, in our previous work [12]Footnote 1 we asked both interaction designers and children to draw a classroom robot, including children with and without robotics knowledge. We found that interaction designers imagined a much smaller and cuter robot than children in general. However, we also found that children’s factual knowledge of robotics affected how they imagined a classroom robot, with children without any robotic knowledge tending to envision a human teacher with some additions and modifications while children with some robotics knowledge imagine a more technically inspired classroom robot. Furthermore, while drawing as an elicitation method provided some freedom, similarly to Scaife and Rogers [14] we felt that children were very focused on creating nice drawings, which took quite some time. We concluded that when involving children in the design of classroom robots it might be useful to enable them to broaden their design views. While it might not be feasible to provide them with a good overview of the state-of-the-art in robotics, it could be possible to offer them design materials that express different and possibly novel ways of thinking about robots that can help them to imagine their preferred classroom robot.

Summarizing the previous work presented above, when involving children in the early design of classroom robots, we need to use methods and techniques that help them to focus on the aspects of interest. The process of involving children in the design of classroom robots can benefit from many different inclusive methods such as sketching, storytelling on paper or verbally, role-playing and design with prototypes. Yet the design problem comes with the need for covering many different properties; form factor, gender, material, characteristics, and behavior. In this paper, we focus on the use of two elicitation techniques, the Robot2Box and clay, for children to focus on form factors of a classroom robot. While the Robo2Box provides children with pre-defined design elements, which might help them to focus on the broad picture, clay offers children the possibility to imagine anything they like, but with the limitation of having to be able to craft it, and without the possibility to get inspired by what is possible. Our main aim is to develop an elicitation method that helps children to easily express what form factors are important to them in a classroom robot. This elicitation method is meant to be combined with other methods, like story-telling, to elicit other aspects of classroom robots.

The following section describes the development of the Robo2Box and presents two pilot studies to investigate the usefulness of the toolkit and inform its further development.

3 Development of the Robo2Box Toolkit

Based on the imperative to involve children in the design process of educational robots, we aimed to develop creative ways to help children imagine their own robots and reflect on their behaviours in the classroom. As a first step, we based our work on outcomes of our previous study [12], and extend our research to develop a robot design toolkit to involve children in the design of classroom robots. We, then, conducted two pilot studies; one using the toolkit only, and a second one comparing the toolkit with clay giving both to the children to design their own classroom robots. Figure 1 illustrates an overview of our approach to realizing the design of the robotic toolkit.

Fig. 1
figure 1

The elicitation method started by learning about the design taxonomy from the literature. A next step was to work with children and professional designers to collect their initial ideas on a classroom robot. Based on these two sources we designed a toolkit, which was then tested in two pilot studies

Full size image

3.1 Exploratory Drawing Study

Our previous work [12] primarily aimed to explore (1) how children’s views on the design of robotic teaching assistants differ from interaction designers’ views, and (2) how children’s views may be influenced by their knowledge of robotics (or lack thereof). We had small groups of children with and without robotics knowledge as well as small groups of international interaction design masters students. The study data consisted of the robotic sketches drawings (Fig. 2), optional writings in the sketches, as well as the audio-recordings of the design sessions. The data analysis revealed a summary of the robotic attributes that we built on to envision the toolkit elements presented in the next section.

Fig. 2
figure 2

The drawings presented in our previous work [12]

Full size image

3.2 The Toolkit Elements

From our previous study [12] we learned that children may have an important role in the design of robots in classroom environments. However, one of the limitations that might have had an impact on this approach is the limited sketching abilities of children, which might have held them back from expressing the designs they liked. For example, Scaife and Rogers [14] claim that the act of drawing might keep children from focusing on other aspects of the interaction, such as behavior. Thus, we propose to not just include attributes envisioned by children, but also the attributes envisioned by the interaction designers’ groups, as it might help children’s imagination when using the toolkit. In addition, to be inclusive of most attributes, we included several attributes that were based on the previous related literature outcomes by Woods [20]. Therefore, we proposed a robot design toolkit named Robo2Box, which consists of 3D printed elementsFootnote 2 as shown in Fig. 3. The Robo2Box includes five main categories: heads, torsos, legs, arms, and materials. Each of the categories has elements that are shaped as human, animal or machine like (and mixes of these categories), which is a similar to how Woods [20] described forms of a robot. The elements of the category Materials are represented by cuts of example materials (e.g. rubber and fur), to allow children to feel and present their preferences. The elements of the Robo2Box can be assembled using double-sided tape.

Fig. 3
figure 3

A list of the proposed elements in the robot-design toolkit with a description on how the element is shaped and where the element comes from. Items marked with ‘*’ are elements that were not present in the designers’ or children’s drawings but reported in Woods’ et al. research [20] as possible physical characteristics for robots

Full size image

4 Pilot Study One: Children Building a Robot Using the Robo2Box

As a first step to the proposed Robo2Box, we conducted a study to explore the use of the toolkit in the hands of children when designing a classroom robot. A first aim of the study was to explore how children would use and design their robot using the Robo2Box and the usefulness of this method to understand children’s form-factor preferences. In addition, we aimed to get an understanding of whether the toolkit could elicit similar or different information from the drawings of our previous work, as shown in Fig. 2. Finally, the study aimed at determining additional features to add to the Robo2Box.

4.1 Participants and Procedure

The study involved 31 school children (16 girls) aged from 8–15 years old (\(\hbox {M}=11\) years, \(\hbox {SD} = 2.3\) years). The majority of children did not have any robotics knowledge (25 children). The others either had little knowledge (4 children) or participated in robotics’ class (2 children).

The main study session was moderated by an administrator, while an observer took notes. The session started by getting informed consent from the parents or guardian of the child. Each session was moderated in Turkish, and was video recorded. During the session, each child was asked to (1) assemble a robot and (2) draw and/or tell a story about what they built. The following subsection describes the details for each task.

Fig. 4
figure 4

Children using the toolkit to build their robots

Full size image

4.1.1 Task 1: Assembling and Placement

In the first task, each child was asked to focus on assembling a robot and, once finished, to place it in a classroom mock-up. The study moderator provided the child with the Robo2Box elements described in Sect. 3.2, and a classroom mock-up. Figure 4 illustrates the Robo2Box elements being used by children. Throughout the session, the observer took notes while the child was assembling their robot. Once they were finished, the moderator asked the child which of the provided materials they would prefer for their robot. The task ended when the child placed their robot in the classroom mock-up as illustrated in Fig. 5 answering the question of where they would think the robot would locate itself in general.

Fig. 5
figure 5

Study items: Robo2Bbox (left) and a model of the classroom with children and an adult teacher (right)

Full size image

While the overview by Mubin et al. [11] shows that robots are generally deployed as assistants to the teacher, we wanted to let the children decide for themselves whether the robot’s role would be an assistant, a tutor or a teacher. The classroom mock-up had a representation of people as a teacher and students to help children understand the relative sizes. The study moderator then asked children about their robot’s size preferences relative to the mock-up, and if they wanted their robot to be larger or smaller. Finally, they were asked to described the role of their robot in the classroom.

4.1.2 Task 2: Storytelling/Drawing

The second task focused on asking the child to write or draw a story about the robot they had assembled. At the end, the study administrator conducted an interview to allow the child to elaborate more on their robotic model and story/drawing. Generally, this task was aimed to explore whether there were form factor details that children imagined that they could not express in the robots they built with the toolbox, such as colors, tools, and attachments.

To conduct the second task, we followed a similar approach to Woods et al. [21]. Thus, we provided children with colored pencils and an A4 sheet that had four sequential frames that children could use to draw or write their story. Once they were done with drawing/writing their story, the study moderator conducted a semi-structured interview asking children to explain their design/story, and to answer several questions that related to how they imagined their robot behaving and why they chose their robot’s physical form. At the end, the moderator asked if they would like to include any further details or elements to their robot design.

4.2 Analysis

The robot construction phase in the first study took 2:25 min on average (longest session 8:54 min, shortest two sessions 1:05 min). While the stories and interviews provided us with further insights into the details regarding the robot behavior in the classroom, these will be presented in another paper. The only part from the interview that we will report on in this paper is the question of whether the children wanted to add or change anything in the robot design toolkit.

For some of these categories (e.g. materials) the children chose more than one part, which was indicated in the coding. Two independent coders first coded the categories for the data for 6 out of 31 children. For the categories where the inter-coder reliability was reasonably high (Cohen’s kappa \(>0.70\)) one of the coders continued the coding of the rest of the data. For the categories “Head"and “Size", which had a lower inter-coder reliability (Cohen’s kappa \(<0.70\)), both coders worked on the whole data set independently and then resolved any conflicts through a discussion. Below, we share the results of the robot construction phase part by part.

4.3 Results

The analysis of the robot design by the children clearly showed that the Robo2Box enabled the elicitation of certain design preferences among the children, especially concerning the head, arms, legs, and size of the classroom robot. For the torso and the materials, the preferences were less clear. Figure 6 illustrates some example of the robot designs by the children using the Robo2Box elements, along with the sketches they made during the storytelling activity.

Fig. 6
figure 6

Examples of the robot designs using the Robo2Box and their corresponding sketches from the the Storytelling task. Note that the sketches were based on the toolkit designs

Full size image

First of all, the children typically chose a human head or a sphere, which is mainly an abstraction of both children’s and designers’ drawings. The difference between the categories was significant, \(X^2 (5, N = 30) = 14.4, p < .05\) (one child used a torso instead of one of the heads and was therefore excluded from the results). Alternatively, the children chose a more human–machine like rectangular head (with a neck). The preferred heads chosen by children resembled similar shapes to the children’s drawings presented in our previous work [12]. In addition, children preferred a head that is not animal-like and separate from the torso. This is in line with Woods’ [20] suggestions that children associate robots with no face to negative behaviors (e.g. aggression).

The difference between the leg categories was also significant, \(X^2 (8, N = 31) = 39.31, p < .0001\). While the children in our previous study (especially those without any robotics knowledge) often drew human-like legs, the children in this study showed a clear preference for two fixed machine-like legs. A possible explanation is that children in the previous study found it difficult to imagine and/or draw machine-like legs, therefore the legs looked rather human-like even though they may not have been intended to. Having other ways of locomotion, for example with four legs (either animal-like or machine-like), or thrust engines was not very popular. Legs looking clearly animal-like (either two or four) were not very popular either. Wheels and tracks were sometimes chosen, which corresponds with some of the drawings of the children in the previous study, in which a robot with two legs was given roller skates or tracks. In this study the children had to choose between legs or wheels/tracks, which probably favored the legs.

Children generally chose to have two machine-like arms. The difference between the arm categories was significant, \(X^2 (3, N = 37) = 12.19, p < .01\). This choice confirmed that children do imagine a classroom robot with two mechanical arms as exhibited in the design drawings of the children in our previous work [12]. This type of arms fitted with the machine-like legs that the children preferred. In addition, the majority of the children indicated the size of the robot to be between a child and an adult. The difference between the size categories was significant, \(X^2 (4, N = 31) = 14.32, p < .001\). In general, children expected their robot to be larger than a child size, certainly not smaller or equal to a child size. This confirms our previous findings that children expect a rather large robot in the classroom [12].

The difference between the torsos was not significant, \(X^2 (5, N = 30) = 10.8, p > .05\) (one child was confused which parts to use as a torso and therefore was excluded). However, it was clear that the two torso parts most often chosen were the sharp-edged and curved-edged distorted rectangular torso, in which both can be mapped as human- and machine-like. Clearly squared, rectangular, or spherical torsos were less popular, which may indicate the preference of a slightly human-like form with shoulders broader than the robot’s middle.

Finally, the children chose more than one material type; using different materials for different parts of the body. None of the distributions of materials used for the separate body parts was significant \(p > .05\). However, when accumulating the number of times each material was used (counting one child’s use of one type of material for several elements as one), it was clear that metal was chosen most often, followed by plastic; this difference was significant \(X^2 (3, N = 59) = 9.68, p < .05\).

5 Pilot Study Two: The Robot2Box Versus Clay

In this section we describe a study to understand the impact the form and materials of the Robo2Box have on the creation of robots. In particular, our exploratory study compares designing a classroom robot with the Robo2Box elements or with clay. We chose to use clay as it is a free-form material for children to build with, which contrasts to the pre-defined Robo2Box elements. The overarching aim to our approach is to define similarities and differences between a predefined prototyping toolkit and a free-form prototyping material, when used in the elicitation of a robotic design.

5.1 Participants

The study involved 29 children (8–15 years of age, \(\hbox {M}=11.7\) years, \(\hbox {SD}=2.17\) years, 16 girls, 13 boys) from Turkey. All children were asked whether they had any previous experience with robots or robotics. Almost all the participants reported that they had robotics courses at school or in recreational activities out of school, or that they had seen robots being made in coding clubs or maker fests at their school prior to this study.

5.2 Materials

Children were presented with the materials, both 40 g of art clay and the Robo2Box toolkit, in a counterbalanced order. To avoid any possible influence of the materials on children’s portrayal of form factors for a classroom robot in their design process we started with a semi-structured oral interview about the classroom robot design before presenting the materials. As discussed in Sect. 2, clay represented the free-form material to create a model, whereas the Robo2Box provided a more limiting but also possibly inspiring elements to construct a robotic model.

5.3 Procedure

Each child participated in the study individually in a quiet room. A written consent form for each child’s participation in the study was taken from their parents. Before starting the experiment one of the researchers informed the child briefly about the experiment and asked to describe any previous knowledge/experience with robots as a warm-up activity. Then the researcher asked the child to imagine a robot in one of her/his classroom settings assisting the teacher. During this semi-structured interview the researcher tried to understand the child’s initial design ideas before being allowed to work with any materials. Some other questions related to other aspects of the robot were asked as well (e.g. how does your robot talk or is it more like a friend or like a teacher), but these will be elaborated on in another paper, as our focus in this paper is on understanding form factors.

Thereafter the researcher presented one of the materials (clay or the Robo2Box) to the child and asked her/him to create or construct an assistant robot with this first material. In order to help children stay focused on the particular material they were using, the two materials were presented one at a time. The construction process with each material stopped when the child informed the researcher that she/he had finished the model. Half of the children first constructed their robot with the Robo2Box and then the clay, while the other half used the materials in the reversed order. All children thus used both materials but in a counterbalanced order. In contrast to the first pilot study, the children were allowed to use all elements of the Robo2Box, so they were not informed about which parts represented heads, arms etc. The whole session for each participant took about 10–15 min in total and was audio recorded or videotaped.

5.4 Comparative Analysis

Since this study focused particularly on describing the form factors that could be elicited using both clay and the Robo2Box as two different materials, we only present the comparative analysis for these two materials here as follows: (1) the duration time spent with each material; (2) similarities and (3) differences between form factors elicited from each child using the two materials. The time spent with both materials was extracted from the time that the child started to construct the robot model with one material and finished when the child informed the researcher that she/he was done with that material. The analysis of the similarities and differences was based on the actual outputs with each of the materials, but also on the answers given to each questions during the semi-structured oral interview. The analysis was initially done by the facilitator of the design sessions, and was checked by one of the other authors to look for inconsistencies or mistakes.

5.5 Results

The average time that the oral interview took was almost 4 min (\(\hbox {M}=232\) s, \(\hbox {SD}=74.7\) s). The average time that children spent on using the toolkit was less than 3 min (\(\hbox {M}=162.5\) s, \(\hbox {SD}=69.8\) s), whereas it was above 3 min with clay (\(\hbox {M}=181\) s, \(\hbox {SD}=70.6\) s). However, one of the children played with the toolkit for around 20 min, making his time for building a robot an outlier data point. In order to avoid an effect on the actual interpretation of the overall data we excluded this subject from the comparative analysis in regards to session time. However, our observation on this outlier data indicate that the unstable fixing parts for the toolkit might be time consuming for some children who are keen on building a robot that can stand upright. Some of the Robo2Box elements were used for different purposes or categories than they were originally labeled (e.g., some of the torso categories were used as head - six children used the sphere as head whereas only one child used the sphere as torso; two children used other torso-labeled parts as heads.

Fig. 7
figure 7

Examples from children’s robot designs through Robo2Box and clay in the order in which they were created

Full size image

Figure 7 shows several examples of children’s robot design using both the Robo2Box and clay. There were several similarities between the robot designs with clay and with the Robo2Box:

  • The type of material did not change child’s decision to design a human-like or machine-like robot.

  • All robots had a head, a body and at least two arms.

  • The shape of the head also showed similarity between the materials regardless of the order in which they were presented.

However, there were also several differences between the robot designs with clay and with the Robo2Box:

  • Nine children used wings in their designs with the Robo2Box elements although they did not mention flying features for their robot in their narrative before seeing the toolkit.

  • One of the children mentioned a flying feature of the robot in his narrative before seeing the materials. However, he did not use the wings in the Robo2Box, and rather created a flat bottomed body without legs in the clay that helped the robot float in the air (see Fig. 7, the 4th set of images from the top).

  • Although children imagined a machine-like classroom robot it was hard to distinguish it from human-like forms in the clay condition due to children’s limited manipulation or construction skills with clay. In that case the think-aloud input that occurred during the design process was helpful in understanding the child’s intentions in his/her choice of the form factor.

  • Children added more details with clay (e.g., buttons, hair, facial features like nose and mouth, longer arms, or an additional drawer hidden inside the body). For example, one of the children made 4 or 5 balls in different scales and explained that the scale of the balls referred to the level of language that the robot spoke - the bigger ball meant that the robot knew that language at an advanced level (see Fig. 7, the second image from the top on the left hand side). Another child thought of a triangular body and could create this with clay, but commented on the lack of triangular shapes in the Robo2Box elements.

  • Machine-like details could not be well-presented in the clay condition. For instance, one of the children imagined a thrust engine in his narrative beforehand, but could not design it with clay. He was excited when seeing it in the Robo2Box afterwards and expressed his relief (see Fig. 7, the 6th image from the top on the right hand side).

  • Several girls included hair in their clay models, something they were unable to do in the Robo2Box models.

  • Some children thought that it was hard to create a Robo2Box model of a robot that could stand unsupported, especially because using removable adhesive pads for attaching objects and joining parts together was not very convenient.

6 Discussion

The aim of the work presented here is to explain the development and validation of a robot design toolkit, Robot2Box, which can be used as part of a human-centered design approach to involve children in the design of classroom robots.

6.1 Pilot Study One

With the first pilot study we aimed to answer the following research questions:

  1. 1.

    Is the Robo2Box able to elicit some clear preferences from children concerning the different design elements?

  2. 2.

    Do children design classroom robots similar to children’s drawings or are they closer to interaction designers’ drawings when using the Robo2Box?

  3. 3.

    What changes should be made to the Robo2Box for children to express their design ideas?

Related to the first question, the first pilot study revealed that the Robo2Box appeared to be easy-to-use for the children. The designs created with the Robo2Box indicated some clear preferences for the type of head, arms, legs and size of a classroom robot.

Concerning the second question, the robots created with the Robo2Box toolkit clearly resembled some of the drawings made by children and interaction designers in our previous study (Fig. 2), especially the drawings made by children without any robotics knowledge. Since the the children in this pilot study did not have much robotics knowledge either, this may indicate that the Robo2Box is indeed able to capture part of what is important to children when designing classroom robots.

Moreover, the additional elements based on interaction designers’ drawings did not change children’s views considerably. This indicates that children’s views are rather stable. Finally, the difference with the interaction designers’ drawings could mean that children really have different requirements for classroom robots than those who may be designing them. Involving children in the design of classroom robots, for example through the use of the robot design toolkit presented here, is thus important to let them express their own views.

However, in order to allow the children to create robots with the Robo2Box that are more similar to the drawings, we might consider separating the arms from the hands, and the legs from the feet. This may allow children to add for example wheels to the legs, or tool-like hands to the arms.

Regarding the third question, twenty-four children indicated that they were satisfied with their design, while six children were not completely satisfied with it. First of all, one child wanted to have a torso that was more clearly human, meaning that the human torso included in the toolkit was not sufficiently human-like. Regarding the head, there were several requests: a human head with a taller neck like an ostrich, a more curved shape in the neck area, a fully spherical head instead of a half one. One child wanted a cylindrical torso instead of a spherical one, and one child wanted thicker rectangular legs. Finally, one child wanted two legs, but with wheels instead of feet, and one wanted one big wheel.

Several children wanted to add additional functionality to their robots. Seven children wanted to add a screen on the torso and one wanted the robot to be able to turn into a television. Five children wanted to add buttons, for example to open and close or stop the robot, and two wanted the robot to have a way to keep pens and erasers, for example in a storage compartment or in the hands. Two children wanted to add guns to the robot’s hands.

Several children wanted their robot to include elements present in existing fictional robots or action figures. Three children wanted an appearance more similar to the popular Baymax figures (body armor, Baymax hands, and a Hulk-like figure with large wings), while one wanted his robot to look like the Optimus Prime transformer figure, and another wanted the robot to have shields like Captain America. This indicates that indeed, experiences with robots in the media influence children’s designs.

Finally, there were some requests to be able to add more expressiveness to the robot such as one child wanting to have the possibility to add stickers to the robot’s head and another wishing the head to have facial expressions, but different from a human being. One child wanted the robot to be able to express emotions, but only in the form of symbols. While all drawings of the children in the previous study were rather colorful, the toolkit only provided the children with white elements. Although children in general did not comment on this negatively, they usually added many details about colors and other elements of the robot in their stories and mentioned them in the interviews. This might be an indication that some more ways for creative expression would have been appreciated.

6.2 Pilot Study Two

With the second pilot study we aimed to answer the following additional research questions:

  1. 1.

    How long does it take children to design a robot using the Robo2Box and using clay?

  2. 2.

    What are the similarities between the robot designs using the Robo2Box and clay?

  3. 3.

    What are the differences between the robot designs using the Robo2Box and clay?

  4. 4.

    What suggestions do the similarities and differences between the Robo2Box and clay provide for the creation of an elicitation methods for a robot’s form factor?

The second pilot study showed that there was no clear difference in time the spent on shaping a robot model with each of the materials, as shown in Fig. 7. There were also several similarities between children’s models using clay and the toolkit, especially regarding the decision to design a human-like or machine-like robot. In both cases the robots had a head, a body and at least two arms, and the head was shaped in a similar way. However, there were also several differences between the designs made with the toolkit and with clay.

First of all, several children only included wings upon seeing them in the toolkit. Wings can be considered to be the most symbolic part among other abstract shapes designed in the toolkit and may attract the child’s attention at first hand. It may therefore be regarded as priming and a limiting effect of the toolkit. However, it can also be considered an inspiration. We thus have to make a conscious decision about the inclusion of wings in further use of the Robo2Box. Furthermore, many children added elements to their clay models that could not be added in the toolkit models, such as buttons, facial expressions and hair. Adding a free manipulable material such as clay to the toolkit to let the children add such details could be beneficial. Finally, several children used the elements of the toolkit in unexpected ways, such as using the sphere torso as a head. It might therefore be useful to include such elements in both categories. Our main conclusion from the second pilot study is that although clay provides a free material to elicit some different form factors and affordances from the children, the toolkit helps designers to code and distinguish the form factors more accurately than clay does. Since the first pilot study also indicated that children want to express some more details in their robot designs, including clay as part of the toolkit might be an appropriate approach to accommodate this, while still allowing for some rigor in the analysis.

6.3 Limitations

There are several limitations to the pilot studies presented in this paper. First of all, the studies were performed in Turkey, with Turkish children only. As Lee et al. [9] indicated, there may be cultural differences in how people imagine social robots, so it is possible that our findings are only representative for Turkish children. However, the drawings on which the elements of the toolkit were based came from Portuguese children and international Interaction Design students. The fact that the children in the present study often chose the Robo2Box elements that were based on other children’s input indicates that findings may be generalised over different nationalities.

Furthermore, while the reliability and detail level of the Robo2Box are important, feasibility and low-cost production of the toolkit are also needed. An additional risk of providing clay to enable children to include many more expressive options is that children may spend a lot of time on dressing up their robot at the expense of eliciting actual design requirements from them. Therefore, we think researchers need to determine whether adding additional options to the toolkit will serve the purpose of using it as a human-centered design tool. Furthermore, while the toolkit may help to elicit some design requirements for classroom robot from children, it should not be treated as the only elicitation method. Listening to the children when they explained their designs was a vital part of our approach. In order to get a full picture of children’s design ideas for classroom robots, the physical design phase should also be complemented with other elicitation methods, such as storytelling or group discussions.

Finally, in this study we chose to ask the children to physically design their classroom robot before discussing the robot’s role in the classroom with them. The rationale for this decision was that it may be easier for children to talk about roles and functionalities when they have a physical model to refer to. However, it is possible that children would more purposefully pick their design elements or model the clay if they had formulated the role of their robot beforehand. The Robo2Box in itself does not prescribe any order, so it may be interesting to study further what the effect of switching the order of those activities could mean for the designs.

6.4 Opportunities for Further Research

As mentioned earlier, Sciutti et al. [15] found out that when children are allowed to interact with a robot, they start paying more attention to its perception and motor abilities rather than its physical attributes. This suggests that robot design studies with children could benefit from a toolkit that also includes some moving parts. So a next step to the approach might be to build a toolkit that has moving parts and means to join them. This might even lead us to a toolkit where we leave the design of some issues such as color, actions, behavior and size to accompanying software synchronized with the physical toolkit. This approach is similar to how Zhu and Zhao [22] work with paper prototypes. With these programmable parts we can create a new experience without forgetting that this is still a human-centered design tool and not a tool for children to effectively build robots.

While children might imagine a classroom robot in a certain way, teachers might have very different ideas of what a classroom robot should be like. Although the Robo2Box in its current form does not include elements based on teachers’ drawings, it could be very informative to let teachers design their classroom robot as well and then involve them in a scenario description activity. By doing so, we can reveal whether there are any disparate views on classroom robots that need to be resolved before developing a classroom robot that is acceptable to all stakeholders.

7 Conclusion

Based on the two pilot studies presented here we argue that the Robo2Box can be a good basis for a human-centered design approach in which children are involved in the design of robots for the classroom. However, it may be improved in several ways, such as using clay and stickers or markers for the details. Even though the Robo2Box focuses on classroom robots, it can probably also be used as a basis to investigate other robot uses. However, while the appearance of a robot is important for its acceptability in the classroom, its tasks, responsibilities and behaviors may play an even bigger role. Therefore, we think the Robo2Box should be used in combination with some other elicitation techniques, like storytelling or interviews. In our case, we have already used the toolkit in combination with storytelling and an interview to investigate whether children envision a robot in the classroom to, for example, set homework, show feelings, or reprimand pupils. These results will be presented in another paper. Another use of the toolkit could be to give it to small groups of children, so that they are encouraged to discuss their robot designs and come to a consensus. However, in that case, the researcher should be aware of the group dynamics that may influence the results [19].