Keywords

1 Introduction

Symptoms of Autism Spectrum Disorder (ASD) are commonly found in children because they tend to appear at a very young age [1]. As a spectrum disorder, each children with ASD has their own unique challenges to overcome. Generally though, autism usually leads to impaired posture and motion, poor explicit (verbal, etc.) and implicit (eye contact, etc.) communication, lack of willingness to interact with others and social awkwardness [1]. In recent years, researchers and clinicians are beginning to focus on the deficits in sensory-motor abilities, which is responsible for the rhythm and synchrony of social interaction [2]. If left without intervention, this will lead to the children eventually facing a lot of challenges living independently [3]. Though the cause of autism is still unclear until this point, early intervention can help children with ASD. However, research findings about the conventional solutions so far mostly rely on improving methods for the caregiver or teacher to interact with the children, such as having better communication or physical interaction.

We hypothesize that a tool specifically designed to aid with physical interaction can further assist these children with their social development and learning experience. Therefore, we propose Furekit, a tactile music toolkit that is modular, wireless, easily deployable and easy to use. The hardware comprises of a wide touch-enabled 3D-printed surface with integrated vibrotactile actuators, speaker, and wireless communication modules. Snap bottoms are used to easily attach the modules onto any parts of the body allowing it to facilitate various physical interactions. We conducted a workshop where Furekit was integrated into the lessons for the children in a special need school. The workshop required them to move their bodies and dance with Furekit by following the music. In summary, our contributions are therefore threefold:

  1. 1.

    We propose a wearable tactile music toolkit interface that can provide a sensory response to each goal-directed movement to augment the learning and social experience of children with ASD.

  2. 2.

    We conducted a workshop at a special needs education school to determine the effectiveness of our prototype.

  3. 3.

    We found that Furekit was well-received, enjoyable, and encourages spontaneous physical movement from the children.

2 Related Works

2.1 Interventions for Children with ASD

There has been several research in the past that looked into understanding the development of Children with ASD [4,5,6]. When interacting or educating children with ASD, it is important to be aware of their non-verbal behavior, create personal interactions among them, and always provide other forms of feedback modality for easier comprehension [6]. This led to the research on interventions for children with ASD. Most interventions should be deployed at an early age due to their increased neuroplasticity [5, 7] and signs of autism typically start emerging between the child’s birth to the third birthday [4].

Past interventions have looked into modifying the interaction with children with ASD as a form of rehabilitation. For example, a free operant procedure via delayed reinforcement was used, and it was found that there was a need for physical contact among them [8]. This was also reinforced by Dohsa-hou, a well-known Japanese rehabilitation method for children with ASD where an intended movement is trained to correlate with the physical movement [9]. This method has been widely used in research and has been proven to support mutual interaction [10, 11]. The results from these findings form the basis of our system which is meant to facilitate spontaneous physical movement and interaction from the children. Fraunberger et al. [12] found that for a tool be designed to assist children with ASD, it needs to be highly engaging, provide visual support, have a structured interaction, can provide means to express themselves, and cannot be complex. We draw from these findings to design both our prototype and the conducted workshop.

2.2 Effects of Tactile and Audio Feedback

Tangible user interfaces (TUIs) have been shown to bring social benefits to children with ASD due to its tactile nature [13, 14] and also suited for therapy [15]. Lego is a common tool due to its playful appearance, yet having infinite possibilities for interaction [16]. However, TUIs simply mean the use of physical interfaces; in addition to that, having an object that reacts to touch with tactility creates new interventions. For example, vibrotactile feedback has been shown to be able to enhance their learning skill [17] as it provides an “act of meaning" to an interaction for children with ASD [18].

On the other hand, music therapy has also been used to support the development of children with ASD. For example, the Observation of Social Motor Synchrony with an Interactive System (OSMOsSIS) is an interactive musical system that transforms body movements into sound [19]. Qi et al. [20] also worked on a prototype that provides audio feedback via an attachable wireless speaker on the children’s body. Though both works showed an overall increase in engagement, OSMOsSIS only allowed the interaction between the researcher/facilitator and the child, whereas both works did not provide any form of tactile feedback. Another similar tool was proposed by Alessandrini et al. [21] that uses an audio-augmented paper as a tool for the therapist instead to engage the child in storytelling. However, the prototype was not designed for children to interact with each other. Le et al. [22] implemented a haptic and audio-based interactive painting activity which showed positive engagement. Yet, to our knowledge, there is little work on an intervention toolkit that is able to provide both tactile and audio feedback to children with ASD, as well as being easily integrated into their educational and social activities.

3 Furekit Development

The goal of Furekit is to design, develop and test a toolkit that can aid the communication and interaction with children with ASD. It will be attached to the children’s body and used as an interactive modality for socializing, learning and basic interaction. To achieve this, we focus on the following features: 1) modular design, 2) usable wirelessly, 3) easily wearable on the body and 4) allows for simple interactions that promotes physical contact. For the design process and workshop integration, we collaborated with the Tokyo Metropolitan Rinkai Aomi School for Special Needs EducationFootnote 1 (the following is abbreviated as Rinkai Aomi School).

3.1 Hardware Prototyping

When designing the appearance of the toolkit, we emphasize the need for it to look fun, simple and playable so as to encourage the children to interact with them. Additionally, the size and shape of each toolkit will depend on which area of the body it is attached to. We adopted an iterative design process, with feedback gathered from Rinkai Aomi School for each design. The TouchMusic form factor was initially used, comprising of a separate black touch surface. However, it was found that a separated touch surface make it hard to understand where to touch. Additionally, different sizes are preferable so that the toolkit can be attached to body parts that are harder to reach, yet can still be interacted with easily, such as the back. We finally opted for two variations; a general rounded design without any sharp edges, and a wider version so as to require lesser precision of touch. The designs are shown in Fig. 1. (left) as Version 2 and 2+. The interior of the toolkit was initially fitted with the ESP32-WROOM-32DFootnote 2 module. However, we found that it tends to overheat inside the enclosure, it lacks an SD card for data logging, as well as a power switch for ease of use. Therefore, we opted to design our own printed circuit board as shown in Fig. 1. (right), based on the aforementioned ESP32 board but in a circular form factor of 64 mm \(\times \) 64 mm to fit the enclosure well and with additional features like a micro SD card slot, touch sensor interface, two amplifiers, an LED light and a switch.

Fig. 1.
figure 1

(left) Various Furekit modules; (middle) attached to a wearable vest; (right) custom board fitted into a Furekit

Full size image

3.2 System Interface Design

We developed a system interface in MaxFootnote 3 that acts as the server machine that controls each toolkit independently and simultaneously. By interfacing with the ArduinoFootnote 4 program on the board, we can control the system to toggle the sound wirelessly and in real-time, modify the sound effects, record time and behavioral data of each toolkit. When any of the modules are touched, a haptic and audio feedback is generated from them. Additionally, the program also supports parallel feedback; when a module is touched, haptic and audio feedback can also be activated on other modules. This allows for solo as well as collaborative use.

4 Workshop in Special Needs Education School

We conducted a workshop for students of Rinkai Aomi School. The workshop obtained ethical approval from Keio University Graduate School of Media Design ethics committee and its contents were notified to the children’s parents in advance. The school grouped the children according to their impairment status. The workshop participants were nine elementary school males in the second grade who are receiving a curriculum corresponding to autism spectrum disorders. Due to the limited number of Furekit, we divided group into team a and team b. There were four children in team a and five in team b.

According to the teachers, the more severe the children’s intellectual disabilities are, the more limited they can express themselves with verbal language. Additionally, children with ASD have their own communication style and culture. In this workshop, we aimed to examine if Furekit is an effective tool for their social expression through physical interaction.

Following the teacher’s suggestion, we designed a session in which participants were allowed to follow the music and touch the Furekit freely without instruction. To make it easier for children with ASD to get on with the activity, we chose a songFootnote 5 with a precise rhythm and moderate speed, and intercepted 90 s from the very beginning of the song for the session. We attached Furekit on a wearable vest as shown in Fig. 1. (middle), and made the modules on the shoulders and stomach to produce a hand-clapping sound, and the back module to produce a ride cymbal sound.

4.1 Procedure

This Workshop was held in a 30-min format. In the first five minutes, the teacher introduced the workshop, the Furekit researchers, and the Furekit itself. After the introductory session, children were asked to dance by following the video of a simplified version “Haptic Exercise", a touch-based gymnastic that aims to rediscover the sense of touch in body movement and environment, which is produced by Haptic Design ProjectFootnote 6. Then, the children of team a were asked to stand up, put on the vests with Furekit modules and dance freely with the song (shown in Fig. 2). Next, the team a and b exchanged the position, the team b children stood up, and completed the same activities as the team a. In this session, each team had 3 min to prepare and 90 s to dance. Before putting the vests on the children in team b, the staff disinfected each Furekit thoroughly. Lastly, the teacher ended the workshop with a 5-min summary of the today’s content.

Fig. 2.
figure 2

The children are dancing with Furekit

Full size image

4.2 Results and Discussion

Observations. Three of the nine children started out without touching, but their proactive actions increased in the middle of the song. Six of them tapped the Furekit to the rhythm, and all of them touched the Furekit modules on their shoulders, stomach, and back. Participant 1(abbreviated as P1), P6, and P8 tapped Furekit following the order of shoulders, stomach, and back. P2 and P9 mainly looked at the next child and touched Furekit less frequently; however, when it was the other team’s turn, P2 swayed his body from side to side with the music and tapped his body to the rhythm. P3 and P4 were swaying left and right with the music while tapping Furekit. P5 sat on the floor with his hands waving along to the rhythm when the other team danced. When it came to his turn, he first put the shoulders and stomach modules against his ears to hear the sound; when the song was halfway through, he began to tap Furekit frequently as well as the floor. P7 walked around the room while tapping Furekit on his body.

Feedback from School Teachers. Overall, the children performed actively with Furekit. The interactive experience of Furekit was very entertaining for them. Several children who are sensitive to things on their bodies did not show antipathy to wearing the Furekit, and they even tried hard to interact with the Furekit. Though some children were confused at the beginning of the song, most of them became active by imitating the children next to them. The school rarely gives free expression classes to children with ASD, and this workshop allowed the teachers to observe each child during this free activity and gain a deeper understanding of the child’s unique communication style. Moreover, teachers believe that Furekit can also be used in other forms of teaching, such as placing Furekit modules throughout the classroom and allowing children to walk around and move their bodies to explore the environment.

Discussion. Through the workshop, we found that Furekit, which integrates tactile and sound stimuli, can serve as an effective tool to encourage touch behavior in children with ASD. It acts as a reward-based system giving sensory responses to each goal-directed movement, which shows its potential in developing the intentional movement of autism. By wearing the Furekit, children with ASD can be motivated to express themselves through physical movement. We also expect Furekit to be used as an educational toolkit to accommodate assistive learning for children with ASD, as well as a tool that can increase empathy between the teacher and children. However, in this workshop, we attached the Furekit to the vest to make it easier to put on and take off, which limited the behavior pattern of the children. Furthermore, due to the COVID-19, the school does not allow physical contact between children, which affected the observation of the effectiveness of Furekit in promoting interaction among children.

5 Conclusion

We propose Furekit, a wearable tactile music toolkit to assist children with ASD in social development and learning. We found that overall, Furekit was well-received and was able to encourage spontaneous physical movement in children with ASD. In the future, we plan to design more wearable forms of Furekit and continue to explore the applications as an educational and communication tool with its collaborative use. We will further discuss how Furekit can be more integrated into their social lives, as well as how it can also potentially be used for other forms of neurodevelopmental disorders.