Keywords

1 Introduction

Autism Spectrum Disorder (ASD) is defined by reduced social-communication skills and restricted and repetitive patterns of behavior and/or focused interests [1]. ASD is also associated with a number of strengths, including attention to detail, honesty, and enhanced ability to recognize and create patterns [2,3,4]. Autistic people often (but not always) express their focused interests through a systematic approach to learning and a strong affinity for computers [5]. Computer-based information is organized according to predictable rules that may be well matched to the systematic processing styles of many autistic people [6]. Heightened affinity for computers may make many autistic people well suited to careers in STEM (science, technology, engineering and math) fields. Indeed, while autistic people are less likely to enroll in college overall than non-autistic people, they are more likely to enroll in STEM majors than students with other disabilities and students without disabilities [7]. Although a growing number of autistic college students are graduating from college with marketable STEM skills, young people with ASD remain chronically underemployed as they face pronounced struggles developing the interpersonal skills needed to obtain and maintain competitive employment [8,9,10]. We are developing a video game to help autistic youth improve their social understanding and collaboration skills. By involving autistic college students in the design and evaluation of this game, we hope to help the students who are involved in the project develop employment-readiness skills while using their feedback to ensure that the game is well designed to teach autistic people.

Challenges collaborating with others and difficulties communicating in ways that are well matched to social contexts are core aspects of the diagnostic criteria for ASD [1]. Autistic people often become depressed because they wish to make friends but have difficulty recognizing the subtle social cues one must interpret in order to forge close relationships [11]. In contrast to the difficulties autistic people experience adapting to offline social contexts, many autistic people report that they can communicate effectively through computer-mediated communication [5]. Not only do computers provide new social horizons for autistic people, interventions delivered via computers can be more motivating for autistic people than interventions delivered in-person [12]. Consequently, families have been quick to adopt computer-mediated supports for autistic people and research evaluating computer-mediated interventions for autism is burgeoning [13].

Despite great excitement about computer-mediated interventions for autism, evidence that computer-mediated interventions are effective in helping autistic people develop generalizable social-communicative skills that are apparent outside of the computer-mediated contexts in which they are trained remains very limited [14]. Troublingly, given that generalization has long been recognized as a limitation of many in-person interventions for autistic individuals (e.g., [15]), few assessments of computer-mediated interventions for autistic people include evaluations of whether increased social-communicative skills when engaging with computers generalize to in-person social interactions [16]. In the relatively few studies that do assess generalization, benefits of computer-mediated interventions for autistic people are often apparent when participants are tested with the stimuli they viewed during the intervention but gains are often no longer apparent when participants are asked to generalize trained skills to new stimuli [17]. Nevertheless, a computer-based face processing training program, FaceSay TM, has consistently been associated with improvements in the real-world social skills of autistic children [e.g., 18]. Whyte and colleagues speculated that the paucity of studies demonstrating that autistic people generalize skills from computer-mediated to face-to-face situations may be attributable to flaws in game design [14]. Key elements known to be important motivational aspects of computer games more generally, such as opportunities to play the game with peers, clear and consistent rewards, a storyline, and customizability, are often lacking in games designed by researchers for autistic people.

1.1 Why Is Participatory Research for and with Autistic People Needed?

Interdisciplinary participatory research wherein autistic people are involved as active collaborators in game design and evaluation may be needed to address the disconnect between the apparent potential of computers to help autistic people and the limited benefits of computer game-based interventions for autistic people documented thus far [e.g., 19]. Indeed, Parsons and Cobb [20, p. 427] proposed a “triple-decker sandwich” wherein effective technologies to support autistic people are developed by aligning three principles: (1) Theory (i.e. top-down insights derived from research), (2) Technologies (i.e. the affordances of technologies), and (3) Thoughts (i.e. the situated perspectives of members of the autism community).

Opportunities for autistic people to be meaningfully involved in all aspects of technology design and evaluation can greatly improve the alignment between technologies and the needs and perspectives of autistic people. Involvement of autistic people in game design is also likely to be empowering; prior work with autistic children has revealed that active participation in the process of game design improves collaboration skills and confidence [e.g., 19]. Nevertheless, a recent literature review of participatory research involving adults with autism revealed extraordinarily few empirical studies utilizing a participatory approach [21]. The authors of the review emphasized the importance of involving autistic people as true partners in autism research (e.g., in the development of research questions and analytic strategies) rather than only involving them in lower level aspects of the project (e.g., data coding). The paucity of participatory autism research may arise because opportunities to help design and conduct research are not always structured to include direct benefits for the autistic people who are considering involvement. A key point from an article describing a uniquely effective research collaboration between autistic and non-autistic people, AASPIRE, was that clearly defined and mutually respectful roles are needed in order to support effective collaborations between autistic and non-autistic researchers [22].

Interdisciplinary collaborations may be particularly important for helping autistic people to be meaningfully involved in the design and evaluation of technologies to support them as interdisciplinary collaborations allow insights about how to successfully involve autistic people, and the benefits of such involvement, to be communicated across disciplines [23]. In a recent paper describing an interdisciplinary seminar seriesFootnote 1 examining technological innovations to support autistic people, the authors emphasized both potential benefits (e.g., empowerment, social inclusion, and socially valid technologies) and potential challenges (e.g., variation in the skills of different autistic people and potential disagreements between different groups of stakeholders and/or the researchers) of involving autistic people in the design of technologies to support them. For example, autistic youth and researchers may disagree about what they would like the game to teach people [e.g., 20]. These differences in opinion can elucidate ways in which a proposed technology may lack social validity in the community it is designed for (e.g., an intention to use technology to normalize autistic people may conflict with the desire of some autistic people not to be normalized).

In addition to potential difficulties integrating diverse perspectives, process oriented and outcome oriented assessments do not always align. For example, autistic people may learn collaborative skills and gain self-confidence through the process of helping to design new technologies but the end product may not always have the impact on other autistic people that the researchers were seeking [e.g., 19]. Given potential disjunctions between processes and outcomes, the authors emphasized the importance of carefully documenting the contributions of each member of the team in order to be able to track ideas from the process of development to the final product.

Despite the relative lack of data-driven participatory research with autistic adults [21], a small but growing number of feasibility/usability studies document the process of involving autistic children and adolescents in the design of technologies for autistic people [19, 20, 24,25,26,27,28,29,30,31,32]. Benton and colleagues [19, 24, 25] noted that participatory design can help researchers design technologies to reach the very diverse types of people who share an autism diagnosis. They speculated that participatory design remains infrequently used with autistic people because the social-communicative challenges and rigidity that define autism can make it difficult to involve autistic people in design. They developed a successful approach, Interface Design Experience for the Autistic Spectrum (IDEAS), to include autistic children in the participatory design of a math game. IDEAS merged participatory design strategies developed with non-autistic youth with principles of a widely-used intervention for autism, TEACCH [33]. The IDEAS method involves four steps: (1) introducing each design session with a visual timeline (with tasks to tick off) and a recap of prior meetings, (2) demonstrating/discussing software relevant to the focus of the session, (3) generating design ideas (with examples of technologies that are and are not effective and idea generation templates as needed) and (4) drawing out the idea agreed upon. Although the process was effective in involving autistic children in game design both individually and in groups, the end game was more appealing to the autistic students who had been involved in designing it than it was to other autistic children. Given that autistic people are often attached to their own idiosyncratic preferences, opportunities for customization are likely to be essential when designing games for autistic people [28].

Truly participatory research reframes the traditional power dynamic of “technology user” and “technology creator/expert” by empowering “users” to co-discover the potential of technologies while collaboratively deciding how to transform them [20]. Although most of the research documenting participatory research with autistic youth has focused on usability/feasibility studies, Parsons and Cobb recently wrote a paper documenting insights derived from a highly systematic and data-driven example of participatory technology development and evaluation with autistic youth, COSPATIAL (Communication and Social Participation: Collaborative Technologies for Interaction and Learning) [20]. Their project provides an excellent example of the aforementioned “triple-decker sandwich” wherein effective technologies are developed by aligning theories, the affordances of technologies, and the thoughts of users. The theoretical framework driving their project was constructivist; they viewed learners as actively constructing their own skills through interactions with others. To match this theoretical perspective, they selected technological platforms that promoted collaborative interaction (e.g., collaborative virtual environments). They captured the perspectives of community members by involving them in design and usability testing. In the first year of the project, they conducted observations of autistic children and needs assessments with teachers to identify key learning goals (i.e. collaboration and social conversation). They also obtained feedback on game design from teachers, 5 autistic and 6 non-autistic students. In the second year of the project, they obtained usability data from 6 autistic and 8 non-autistic students. In the third year of the project, they conducted an educational evaluation of the impact of the technology on 22 autistic students. After observing conflicts between the ways teachers wished to develop the technology (i.e. a desire for controlled activities to guide the learners toward specific learning objectives) and the students’ preferences (i.e. a desire to make the game as flexible as possible to allow exploration), the core design team (which included 5 teachers) decided to prioritize the goals of teachers. The researchers highlighted the complications inherent in trying to develop technology in response to conflicting stakeholder perspectives and emphasized the importance of transparency when communicating the aims of a participatory research project, the perspectives of the researchers, and the reasons certain recommendations are and are not taken. When discussing tensions between process oriented and product oriented research, they indicated that careful documentation of both the process of technology development and the impacts of the technology on outcomes may help to alleviate these tensions.

1.2 Research Aims

Our research is both process and product oriented. The aims of our research are: (1) to document the process of including autistic youth in the design of a game to help autistic people understand complex emotions and collaborate and (2) to determine if an emotion matching game that autistic individuals can play with peers and siblings is effective at scaffolding collaboration and recognition of complex facial and bodily emotions. Over the past year and a half, the principal investigators (an autism researcher/psychologist and a professor of computer science) have been developing this game in collaboration with Pavel Asanov, a technology innovator. Autistic college students in Dr. Gillespie-Lynch’s mentorship program, Building Bridges Project REACH, have been providing feedback on the design and evaluation of the game. Meaningful inclusion of autistic students in game design and evaluation allows them to gain valuable employment skills and also ensures that the game is well-matched to the interests of the individuals it is designed to support.

2 Game Design

The game is designed to support the development of generalizable social-communicative skills in two ways: (1) by providing participants with an immersive intervention wherein they can simultaneously engage with a peer or sibling digitally and in-person and (2) by teaching participants how to interpret complex emotions using varied cues (facial cues, bodily cues, and context cues). Evidence suggests that immersive computer-mediated interventions, such as virtual reality interventions, are more effective at promoting generalization among autistic individuals than less dynamic computer-based interventions [16]. Whyte and colleagues recommended that computer-mediated games to help autistic people transfer skills to the real world include multiple players (as most games for autistic people focus on individual instruction) and blended computer and in-person interactions [14]. Our collaborative game is immersive and dynamic. Standing near one another, participants complete collaborative emotion matching puzzles by moving images on a screen using Kinect technology. Because participants are acting together upon a digital world while standing next to one another in the “real world”, they have many opportunities to engage with one another in-person while receiving digital scaffolding to help them solve increasingly complex collaborative emotion matching tasks. Not only is the opportunity to simultaneously engage in-person and online likely to promote generalization of social-communicative skills, our game is designed to address inter-related core challenges associated with ASD. While autistic people often do not struggle with interpreting simple emotions, such as happiness or sorrow, they do struggle with interpreting complex emotions, such as shame or surprise, which provide important but subtle clues about how to adapt to social contexts [34].

Our game is being developed by an interdisciplinary team using the Unity engine with scripts written in C#. The platform is a Windows PC with the Kinect gesture-based user control system. This hands-free control has two major advantages over mouse/keyboard/controller for autistic individuals: (1) it eliminates an additional device that players often have difficulty manipulating and (2) it incorporates exercise into game-play, which is thought to be helpful for autistic players [35].

An introductory visually guided tutorial lets players acclimate to the Kinect environment by using animations to show the player how to move and place game objects. In the core game, two players stand side by side using hand gestures to move the pieces of a puzzle. Each puzzle depicts the outline of a figure in an emotionally valenced context (e.g., a bear in the background to depict fear). After players construct the body of the figure, they must agree on the correct emotion for the face of the figure by selecting from three displayed emotions. When players master a skill, an audio-visual film clip or mini-game appears as a reward. These emotions become increasingly complex over the course of the game. In addition, the depiction of the emotions shifts from cartoons to photorealistic displays in order to build from a widely documented interest in cartoons among autistic people [36] toward photorealistic emotions they will encounter in everyday life. Similarly, puzzles early in the game include overlapping cues to the appropriate emotion to agree upon (e.g., a background image of a group of peers excluding the main character and a body cue of sadness) while later levels contain only one cue (e.g., just the body cue of sadness) to encourage recognition of complex bodily and facial emotions based on limited cues. In order to motivate all players to persist and learn, our approach to game design follows well-regarded principles outlined by Gee [37] such as coding for “Empowered Learners” with levels that are pleasantly frustrating while incorporating ‘just in time’ instruction. Our game design is guided by the principles of Serious Games that Whyte suggested may be particularly useful for autistic players [14] such as including scenarios to elicit emotion recognition and building games with two players playing cooperatively.

The collaborative component of our game was adapted from a less immersive collaborative picture-matching task that was effective at promoting collaboration among minimally verbal young children with autism. Holt and Yuill [38] noted that many of the “collaborative games” developed to help autistic individuals engage with one another do not actually require any collaboration as one participant can dominate the game without any feedback from the other player. To address this, they developed a truly collaborative picture-sorting task that required children to collaborate by indicating agreement at key points. Four minimally verbal autistic preschoolers showed more engagement with and awareness of one another when playing the game that was truly collaborative than when playing a so called “collaborative game” that did not actually require the children to indicate agreement.

Holt and Yuill’s innovative game supported collaboration among children who do not typically collaborate with one another [38]. However, it was delivered to children seated next to one another in front of laptops who had to manipulate a mouse to operate the game. Participants in our game are standing and must use large hand movements to manipulate pieces rather than clicks of the mouse. It is important to encourage autistic individuals to move physically and to provide activities that they can access with a range of motor skills as sedentary behaviors and motor difficulties are common in autism [39, 40]. Indeed, one of the four autistic children in Holt and Yuill’s original study struggled with using the mouse; in a subsequent study, which replicated the original benefits of their collaborative technology, Holt and Yuill evaluated a touch-screen collaborative interface [41]. Our game, which is entirely nonverbal with audio-visual scaffolding and feedback, addresses a core difficulty in autism with successively more difficult emotion matching tasks. Therefore, it has the potential to be useful for autistic people at many different stages in development and across cultures.

2.1 Participatory Feedback from Autistic College Students

We invite autistic college students enrolled in the Project REACH mentorship program to provide feedback on our game at key junctures in game design, compensate them (with gift cards) for their time, and invite them to become more deeply involved in game design or research depending on their level of interest. Two autistic college students and their mentors provided feedback on very early stages of the game last spring. One of the autistic students provided extensive feedback which included: (1) Making the sound effects louder and the images bigger, (2) Providing demonstrations of how to pick up objects, (3) Making clearer whose hand operates each hand on the screen, (4) Motivating players with virtual rewards, and (5) Selecting backgrounds that match the desired emotion. For example, images of fall and sunsets conjured up happiness for him. He could not commit to greater involvement in the project in the spring. However, he returned this fall to provide more feedback and indicated that he is now very interested in becoming more involved in game design.

This fall, five autistic students and two of their mentors were videotaped while playing the game (to allow us to code interactions) and participated in structured interviews about the game. Their overall response to the game was positive. Four of the students mentioned the collaborative aspect of the game. One commented, “Most games where there are two players… make you feel frustrated about the other player playing with you. You rage at missed opportunities and have a scapegoat to blame when you lose. (This game) teaches you to really work together.” Students provided detailed suggestions about how to improve the look, feel and content of the game which included: (1) Providing demos about each level of the game, (2) Reducing the lag between player movements and the Kinect response, (3) Making the faces bigger, (4) Allowing players to customize the characters, (5) Making coins meaningful by pairing them with prizes, (6) Designing images for the background rather than “google images” and improving the quality of images (to “make it seem like it’s 2016 these days and I’m not playing like a 1990’s game”), (7) Making the game more difficult using a timer, more puzzle pieces or by asking participants to recognize blended emotions, (8) Providing opportunities to select expressions that do not match the context and/or each other (“a lot of people get some sort of kick out of seeing situations and people in them with expressions that don’t really match”), (9) Providing situations that young people commonly experience (e.g., getting caught cheating on a test), (10) Allowing players to express negative emotions that they can’t explore in day-to-day life (“like Grand Theft Auto”), and perhaps most importantly, (11) Giving the game a storyline showing how the characters progress.

Two of the authors of this report qualitatively coded the videos of the autistic and non-autistic college students playing the game. They obtained reliability on seven coding categories for the behaviors of the autistic students: (1) looking at the screen, (2) looking at their partner, (3) showing positive emotion, (4) expressing frustration, (5) asking for help, (6) turn taking behavior, and (7) conversation/commenting. All of the autistic students were highly visually engaged with the screen; they typically attended to the screen for the entire duration of each observation period. Although they rarely looked at their partners during game play, three of the five autistic students looked at their partner at least once. Most of the autistic students frequently communicated verbally during game play. However, they had been prompted to comment on their ongoing perceptions of the game. Although the students did not express frustration during game play, they remained fairly taciturn overall while playing. However, all five of the students expressed positive emotion at least once when playing. Only three autistic students exhibited explicit turn taking behaviors. However, we had prompted the non-autistic player to let the autistic player make each move first which may have diminished turn taking.

The ease with which researchers attained reliability on the coding scheme suggests that it is a promising foundation for larger scale assessments. Our difficulties interpreting some of the results of the coding due to prompts we had given the players to interact in specific ways taught us that we should initially allow players to play the game without any verbal directions in the future. Although the fact that the players displayed some positive and no negative emotions is promising, we hope that positive emotions will occur more frequently as we make the game more engaging by incorporating students’ recommendations. Although it is promising that the autistic players were highly engaged with the screen when playing the game, we may wish to embed prompts for players to look at one another in future iterations of the game in order to maximize benefits of interacting via the screen and in-person simultaneously.

Many of the autistic students gave valuable feedback regarding the look and feel and content of the game. They liked the Kinect interface and enjoyed controlling items with their hand. The tutorials helped them understand how to select and release objects. They suggested improving the tutorial by animating the object to grab (instead of only the hand moving). Students suggested scenarios that might be more relevant for other autistic youth (e.g., interactions at school) than the scenarios we had initially developed (e.g., a person in the woods with a bear).

When asked the purpose of the game, autistic students typically realized that emotions were involved. However, two autistic students thought the game was designed to promote motor coordination and one student reported motor difficulties playing it. Autistic students also noted distracting details more frequently than non-autistic students (e.g., coding text that was accidentally visible at the bottom of the screen). Although the autistic students all said that the emotions depicted by the puzzle characters were clear, a few autistic students (but not mentors) misidentified the valence of emotions (e.g., happy instead of ashamed), thus supporting the need for this type of game. All of the autistic students approved of the purpose of the game. Two students suggested that we also use the game to teach literacy and creativity.

Three autistic students indicated that they were interested in becoming involved in game design or research. These students have been invited to attend our first planning meeting of the spring term where they will receive training in the roles they wish to take on. Pronounced variation in the abilities of these students may create challenges when initially developing their roles but will help us develop the game towards the diversity of the spectrum. We will use the IDEAS model to create opportunities for autistic college students with diverse skills sets to help with game design by (1) introducing each session with a visual timeline (with tasks to tick off) and a recap of prior meetings, (2) demonstrating/discussing relevant software, (3) providing examples/idea generation templates to support students in generating and evaluating ideas and (4) exploring ideas we agree upon through multiple modalities (e.g., drawing, speaking, writing and/or demonstrating).

2.2 Changes Based on Feedback

While observing novice players testing our game, we realized that we needed a more extensive tutorial level to introduce them to the Kinect interface. In general, players were not familiar with how to grab, move and release objects. We created three tutorials with animated demonstrations for the user to copy. In the most basic level we show an animation of a hand moving; a second level shows the animated hand opening and closing to demonstrate grabbing; a third level teaches the player to place an emotional face in the puzzle. These tutorials were modified in response to user feedback and from watching the players interact with the game and one another. For example, originally we used an animated hand to show how to move an object. However, some players did not understand that they needed to grab the piece and place it. Figure 1 shows a snapshot of an updated animation that demonstrates the placement of a face puzzle piece. We used a faded (somewhat transparent) puzzle piece to show the desired move/interaction.

Fig. 1.
figure 1

Tutorial to demonstrate placement of faces

Full size image

The most essential change in game design initiated by student feedback was to develop a storyline for the game with a consistent main character whose gender and race will be customizable. In early iterations of the game, each level of the game portrayed a different character in a different situation. The autistic students noted that they would prefer to choose among different main characters who varied in race and gender. They also indicated that the game would be more relevant to players if it more closely mirrored the lived experiences of urban youth yet had a fantastical element. Therefore, we created a story focused on a single customizable protagonist facing a challenge that many autistic youth face, exclusion and bullying, and how friendship (with a friendly purple dragon) can help one overcome bullying (See Fig. 2).

Fig. 2.
figure 2

Two scenes (fear and shame) from the first version of the game are on the left. Two scenes (sadness and anger) from the revised game with a storyline are on the right.

Full size image

Players recommended increasing the sizes of the emotional faces so they could be read more easily and improving the overall quality of the images. Therefore, we increased the head size of the main character and began to use Illustrator rather than Photoshop as the primary illustration tool (as it is more flexible). We are also planning to change the shape of the puzzle pieces of emotional faces so that they are all half circles which are larger than the face itself (aren’t closely cut out) so that the shape of the face doesn’t influence which face is correct (and the player can focus instead on the emotion).

Our original design had a numeric point system with a running score. Students recommended using pictorial feedback instead so we changed the point system to coins that float up when the correct puzzle piece is placed. Players then reported that it was unclear why the coins appeared or what function they served. Students recommended collecting the coins at the top of the screen and allowing players to use them to purchase tools that can be used in the game (e.g., a shield). We are planning to implement these suggestions.

Players were sometime confused about which hand on the screen they were controlling so we placed color-coded hands on each side of the screen so that each player knows which hand is theirs. We subsequently moved these hands to the top of the screen after realizing that the players didn’t notice them when they were at the bottom of the screen.

One very significant change we made was based on watching the way novice Kinect players positioned their hand during play. We noticed that when their hand was not completely closed, Kinect was interpreting any motion as a grabbing motion. This sometimes led to puzzle pieces moving when the player did not intend to move them. We changed the code to recognize only 2 hand position modes - grab and release; anything in between (lasso) is now interpreted as release.

Although not a design issue per se, we are also working intensively on a coding issue that multiple autistic students highlighted as a key “glitch” to address. This coding issue sporadically introduces a lag between players’ movements and the resultant Kinect-mediated motion of puzzle pieces. A number of students indicated that they were distracted by this “glitch” so we have been working intensively on improving it.

3 Planned Game Evaluation

In order to systematically evaluate both the process of game design and the learning outcomes associated with playing our game, we adapted the evaluation strategy developed by Parsons and Cobb [20]. After we have documented the process of developed a polished version of the game guided by our autistic collaborators, we will evaluate the usability of the game in order to refine it further (e.g., planned usability study) and then conduct an assessment of its efficacy (e.g., planned efficacy study). In order to develop meaningful roles for autistic college students who are interested in helping to evaluate the game, we will adapt strategies from participatory action research. For example, we will provide autistic researchers with “potentially wrong” versions of outcome goals and assessments (i.e. stated learning objectives and standardized measures that they may feel are not ideal) and will ask them to edit the outcomes and assessments to make them more effective [42].

3.1 Planned Usability Assessment

In our usability study, we will determine if our Kinect-based game is more engaging than identical in-person emotion-matching tasks while obtaining feedback on game design from a large number of individuals with and without ASD. Dyads, comprised of 60 autistic college and high school students and 60 neurotypical peers, will participate in a single session consisting of a pre-test, computer-based and in-person emotion matching tasks, and a post-test. Students will be paired with a peer of approximately the same age. Modality of collaboration will be a within-subjects variable; dyads will be randomly assigned to complete either an in-person or a Kinect-based series of emotion-matching tasks first followed by the other modality of tasks. The specific emotion matching activities they encounter in the computer-based and in-person versions of the game will be counterbalanced. Type of dyad will be a between-subjects variable: (1) Twenty autistic students paired with twenty autistic peers, (2) twenty autistic students paired with twenty non-autistic peers and (3) twenty non-autistic students paired with twenty non-autistic peers. By comparing engagement across computer-mediated and in-person versions of the game for different dyad types, we can determine if the intervention is uniquely beneficial for autistic people.

All participants will be video-taped and audio-recorded during a 5 min free play activity prior to the pre-test, during the emotion matching tasks, and during a 5 min free play activity after the post-test. Research assistants who attain research reliability on the following codes will qualitatively code the videotapes: (1) fidelity of administration of tasks, (2) looking at the screen, (3) looking at their partner, (4) showing positive emotion, (5) expressing frustration, (6) asking for help, (7) turn taking behavior/collaboration, (8) conversation/commenting, and (9) joint engagement with positive affect. The number and type of tasks that participants successfully complete together will also be assessed. All participants will be asked what they liked and did not like about the game and how to improve it.

We expect that increased collaboration, engagement during and completion of complex emotion recognition tasks will be apparent in the Kinect-based version of the tasks relative to the in-person version. We expect that benefits of the Kinect-based game will be most apparent for dyads containing one autistic and one neurotypical peer. Few prior studies have examined communication between two autistic individuals although it is a topic of high interest given that autistic people often, but not always, report feeling more comfortable engaging with other autistic people [43].

3.2 Planned Assessment of Efficacy

After incorporating feedback from participants in the usability study, we will conduct a more focused and intensive study to examine benefits of repeated engagement with the program for minimally verbal younger autistic individuals and their siblings. Participants will include 40 minimally verbal autistic children (4–12 years of age) who have a sibling and their 40 siblings. Dyads will be randomly assigned to either receive the Kinect-based intervention immediately (playing the game for 5 h a week for 6 weeks) or to a wait list control group. Pre-test and post-test measures will consist of assessments of Theory of Mind, recognition of simple and complex facial and bodily emotions, a parent-report of autism symptoms and the gold standard behavioral measure of autism symptoms (the Autism Diagnostic Observation Schedule). The qualitative behavioral measures described in the usability study will be obtained during pre-test and post-test free play sessions and during game play. Parent and sibling reports of the frequency with which siblings play with their autistic sibling will also be obtained.

We expect that participation in the Kinect-based training will be associated with improvements in complex facial and bodily emotion recognition, Theory of Mind, collaboration and sibling engagement. Although we expect improvements to be more pronounced for autistic siblings, we also expect improvements among non-autistic siblings given that challenges with Theory of Mind are an aspect of the Broader Autism Phenotype [43].

4 Conclusions

Our participatory research project is designed to document both the process of involving autistic college students in the design and evaluation of a game to help other autistic people develop social-communicative skills and the degree to which the game is effective in actually helping other autistic people develop collaboration and emotion recognition skills. So far, our autistic college student collaborators have provided very useful suggestions that have led to extensive improvements in game design. Their suggestions have often (but not always) been compatible with the suggestions of other autistic students and our aims as researchers. In fact, the autistic students reminded us to include key design principles (e.g., a storyline and an interpretable reward system) in our game that are believed to promote generalization of skills but are often lacking in games designed for autistic people [14] and were initially lacking in earlier versions of our game as well. Therefore, the autistic students effectively guided us to make our work more consistent with theoretical principles of learning.

The autistic students also provided some recommendations that we may not be able to incorporate into the storyline we developed without introducing confusion (e.g., providing opportunities to select emotions that don’t match the context). We will need to discuss the process of evaluating recommendations to decide which to use during our game design meetings with our autistic collaborators. Importantly, all of the autistic students agreed that the learning goals of the game would be helpful for other autistic people.

We are attempting to navigate the tension that Parsons and Cobb described between process oriented and outcome oriented research [20] by documenting the empowering process oriented aspects of participatory research with a small group of autistic college students, evaluating if the game is engaging for a broader population of autistic students, revising the game accordingly, and then evaluating if it is effective in promoting social development among autistic people who struggle to express themselves. Unlike other participatory research studies which have involved autistic children and adolescents in game design, we hope to use the insights that accomplished autistic adults have derived from their life experiences to develop a game that is engaging and effective for autistic youth who are not able to express themselves through spoken speech. Autistic people report that they gain understanding of themselves and others with development [e.g., 45]. Therefore, autistic adults may be uniquely capable of melding insights derived from the lived experience of being autistic with a gradually developed understanding that other autistic people may have different perspectives than their own in order to help us adapt the game to the needs of autistic people who can’t express their own preferences easily. Our project is designed to investigate the possibility that autistic adults can provide unique insights into the top level of Parsons and Cobb’s “triple-decker sandwich” of effective technology design by helping us gain greater understanding of the learning processes of minimally verbal autistic people in order to design our game to better serve their needs.