Guidelines for research and design of software for children with ASD in e-health

Abstract

Autism spectrum disorder (ASD) is a neuro-developmental disorder that results in narrow interest and impairments in communication and social relationships. Usually, unknown contexts generate anxiety and frustration in persons with ASD, and great impediment in accessing services, including health ones. This seriously compromises prevention and treatment interventions in different medical areas. Technology offers many opportunities for persons with ASD and can be used to act as a mediator, helping to manage communication and interpersonal relationships and to familiarize with the environment, especially in new contexts. Specifically, we used digital tools to reduce stress and anxiety while teaching adequate dental care to children with ASD (MyDentist project). To generalize the R&D approach, we followed in the context of the MyDentist project for its adoption for research and development of inclusive software in e-health. We analyze our design and development process using the three-cycle model proposed by Hevner: the relevance cycle, which connects design with the environment; the rigor cycle, which connects design with the grounding theory; and the design cycle, which builds and evaluates artifacts. We present the results obtained in the project and the lessons learned. Lessons are formalized as guidelines that are further validated by a focus group including experts and stakeholders. A set of guidelines which can drive software development and research in similar contexts. Our research will benefit e-health researchers to perform rigorous and relevant Design Science Research. It will also inform software developers of e-health solutions and healthcare professionals.

1 Introduction

Autism spectrum disorder (ASD) is a severe condition that seriously compromises the way in which people communicate with and relate to other people and, in general, their ability to make sense of the world around them. ASD is also characterized by a remarkable heterogeneity at the behavioral level, with substantial inter-individual differences [1]. The overall prevalence of ASD, measured by the network of Centers for Disease Control and Prevention in the USA, was of 23.0 per 1000 (one in 44) in 2018, the last report published in December 2021 [2], while another recent study [3] estimated the prevalence of ASD, in Italy, in one in 87 in school-aged children considering, as a health system catchment area, the province of Pisa. To the best of current knowledge, the only possible intervention for ASD lies in early diagnosis (in Italy around the age of 1.5/2 years) and subsequent personalized and multi-directional intervention. On the basis of group-level data, research suggests that behavioral programs early implemented and through intensive planning can be effective in improving cognitive, adaptive and social–communicative competences [4, 5]. As a matter of a fact, persons with ASD can acquire many skills, become fairly autonomous, work and even live alone depending on symptom severity that may be extremely variable. Nevertheless, the ability to effectively cope with day-to-day demands is seriously compromised.

In recent years, aiming at improving the quality of life of persons with ASD, the use of digital tools to support persons in everyday contexts like at school, at home, during therapy, etc. [6,7,8,9] has received more and more attention. This interest is motivated by the positive effects of technology-enhanced learning programs providing comfortable and predictable environments for persons with ASD [10,11,12]. Digital tools can also have a role for stress and anxiety management in persons with ASD [13, 14]. In the work of Carlier et al. [15], the authors investigated the feasibility of empowering children with ASD and their parents through the use of a serious game to reduce stress and anxiety and a supportive parenting app. In addition to testing their software tools with positive results, the authors emphasized the urgent need for further research on defining domain-specific guidelines and how to practically incorporate them into e-health applications for ASD. They also confirmed that designing for this target population is an iterative process that requires the active involvement of all parties.

According to Murray [11], the potentiality of computer-mediated educational programs had to be attributed to the prevalence of visual stimulation of the screen, that interferes with the autistic world, breaking the individual’s attention tunnel. The positive impact on attention level improves learning effectiveness potentially increasing individual self-esteem and self-awareness. Global benefits have been found to (a) increase focused attention; (b) increase overall attention span; (c) increase in-seat behavior; (d) increase fine motor skills; (e) increase generalization skills (from computer to related non-computer activities) (f) decrease agitation; and (g) decrease self-stimulation [16].

It is with the support of these experiences that we conduct the MyDentist project. This project aims at developing digital tools and a clinical protocol to support dental care in children with ASD. Using MyDentist, dental professionals can produce and organize personalized video material, photos and serious games to set up customized training paths for each patient, making them accessible also from home. The goal is to help children with ASD reduce their anxiety and to train them on how to take care of their own oral hygiene accepting the dental clinic environment and procedures. A preliminary experience of dental care with MyDentist, involving 59 children with ASD in an Italian public health service, showed high acceptance of digital tools by children and families and a positive impact in oral care attitude both in clinic and at home [17].

The design of the MyDentist app and the definition of the clinical protocol based on these digital tools is a process exposed to many of the challenges that arise when designing artifacts in the e-health domain for persons with special needs. Functional and non-functional requirements collection involves a variety of stakeholders having different mental models and different communication languages. As end users (children with ASD) often cannot directly express their needs, it is necessary to enroll mediators, domain experts and caregivers who know the users and that can convey their voice both during requirements collections as well as in the validation phases [18]. Furthermore, as software development and the clinical protocol are tightly bonded together. They have to be co-designed in order to meet the needs and expectations of users with disabilities and to respect the medical standards without interfering with procedures and processes [19].

This paper reports the experience of the design of the MyDentist App framing it within the three-cycle model proposed by Hevner [20]. In particular, an important byproduct of the design process is the set of guidelines that drove our work. This set of guidelines is suitable for a software development process centered on an iterative designing and testing cycle with the active participation of all stakeholders, including children with ASD and their caregivers, and they originate as a consequence of the concrete problem to solve and from the lessons learned. We believe that, beyond software development in applications for persons with ASD, they may be useful also in other health domains as well as in similarly challenging contexts. In summary, the main objectives of the current work are to:

• 1. Formalize our experience by clearly distinguishing its components and relationships;

• 2. Provide a contribution to the knowledge base (domain and process) through a set of guidelines to guarantee the reproducibility of the approach in similar contexts and generalizability to other domains.

The paper is organized as follows. Section 2 discusses the background and the relevant literature. In Sect. 3, we describe the MyDentist project and analyze its development and testing using Hevner’s three-cycle model. In Sect. 4, we present and discuss the lessons learned synthesizing them in a set of preliminary guidelines. In Sect. 5, we present the validated final guidelines grouped by four themes. Section 6 draws the conclusions.

2 Background and state of the art

As discussed in the Introduction, the use of digital technology may improve the quality of life of persons with ASD, especially children. At the same time, however, the development of such tools incurs a number of issues and difficulties that could occur at any stage of the design, from the requirements collection to the specification and the field experimentation. In this section, we review preliminarily the Hevner model [20] that we use as a reference to report the experience of the MyDentist project and then we report the state of the art in the guidelines for graphical user interfaces and software design for persons with ASD.

2.1 The Hevner model

The Hevner model (shown in Fig. 1) defines three cycles (namely the relevance, design and rigor cycles) and three contexts (Environment, Design Science Research and Knowledge Base) [20]. The relevance cycle connects the contextual environment that defines the application domain with the design activities that produce the artifact (in our case the MyDentist prototype) by means of progressive iterations. The rigor cycle, instead, links the design activities with the knowledge base, including the scientific foundations, experiences and expertise over which the project builds. The core of the system development is the design cycle, which progressively improves the artifact prototype. This improvement is the consequence of the field tests conducted in an increasingly richer scenario, part of the relevance cycle, which, in turn, is made possible by the refined functionality and reliability of the prototype itself.

Fig. 1

The design science research cycles of the Hevner model

2.2 User interface design guidelines for persons with ASD

Several studies addressed the development of effective user interfaces for persons with ASD. These works differ in the process used to extract the final guidelines. Some of them synthesized main results of existing studies on software design for persons with ASD [21,22,23,24,25] focusing on specific groups of categories and associated graphical interfaces elements providing guidelines and practical implications of them; others analyzed current applications and web sites [21, 23, 26] also providing a rating score of each one for assessing its usability by users with ASD.

From these works, we can highlight some recurring elements such as the use of soft colors, the exploitation of images for redundant representation of content, a simple general design, the need to avoid background on images as well as the preference for realistic images and the necessity to avoid multiple stimuli on the screen. The crucial aspects, extremely recurrent in the literature, are personalization and customization since each user is unique and nothing can fit for all.

Finally, a recent document of W3C [27] provides advice on how to make web content and web applications usable for persons with cognitive and learning disabilities. The document identifies eight high-level objectives outlining the key design goals, each one associated to four elements: (i) user needs and user stories that show the user perspective; (ii) design patterns; (iii) personas with user scenarios, that explain the users’ experiences and challenges; and (iv) test questions for each objective of a user testing. In our work, we leveraged part of these guidelines.

2.3 Software design guidelines for persons with ASD

The use of technology to improve educational interventions destined for persons with ASD or intellectual disability has increased drastically in the past years. Designing for this specific domain presents peculiarities that have an impact on the research setup, methodology and interaction with the stakeholders [19, 28]. Regarding research methodology, since the design process must ensure artifact accessibility and usability other than adequate user experience, there is a general accordance on the importance of adopting participatory design since it is the best approach to favor the emerging of different needs and expectations from a multidisciplinary work group using methods such as questionnaires, vis-à-vis interviews, mockups, prototypes, sketches, etc. [29]. Among others, caregivers are crucial figures to involve in the working group both as mediators of end-users (especially in medium/high severity levels of ASD) as well as being users themselves: in fact, a huge part of the digital tools are destined to caregivers with the aim to improve care practice. Moreover, their knowledge of persons with ASD can enrich communication and help to better set up participatory design procedures. However, traditional participatory software design techniques need to be adapted in order to include persons with ASD in the process along with the rest of the multidisciplinary team [30]. End users with ASD may be included in the design group at different levels depending on opportunities and problems in the actual application environment and on the severity level of ASD of the target group. In case of high or medium severity of ASD, often associated with some difficulty to recognize and express personal needs and preferences, users are observed in natural condition while using software artifacts to spot any difficulty and to inform future iterations of the design [31]. Otherwise (low severity of ASD), they can be asked for direct feedback and partially involved in design phases. In any case, direct user inclusion has to pay attention to: (i) make persons with ASD familiar with the physical environment; (ii) provide opportunities to bond with other members of the work group; (iii) favor communication through appropriate facilities such as Augmentative and Alternative Communication (AAC); (iv) set up environments avoiding excessive sensory stimuli; (v) adapt activities to individual pace, also providing time for relax; (vi) involve caregivers to mediate communication when necessary (especially for non-verbal users) [32, 33].

Related literature offers scattered sets of guidelines and best practices regarding the design and implementation of software tools for children with ASD [34,35,36,37]. Those sets are each one extremely focused on a specific type of SW tool and a particular learning objective, such as social interaction improvement, emotion recognition or language skills [35, 38]. However, to the best of our knowledge, there are no specific guidelines to address the whole process of participatory software design for persons with ASD, especially in challenging e-health environments. The main challenges in e-health environments that triggered our work are: (i) the presence of multiple users that must be considered interdependent actors; (ii) the global mission of the SW artifact that must serve all different types of users and the whole health system respecting and integrating the current medical protocol. The main contribution of the current work is to fill this gap, providing a comprehensive set of guidelines that includes the clinical protocol and not just the software artifact construction. In our experience, this requires addressing a complex co-design of the software and the clinical protocol with the active participation of experts and users. This is what we have done in MyDentist project as described in the following sections of the paper.

3 MyDentist: overview and development model

MyDentist is an ongoing project supporting dental care of children with ASD. As mentioned, it is motivated by the fact that children with ASD perceive sensory experiences differently than neuro-typical children, which makes it extremely difficult for them to accept unfamiliar environments and contexts. This is particularly problematic in dental care settings, where there are many strong sound-visual stimuli, which are not found in known environments such as home, school or therapy session rooms. This usually upsets children with ASD, often forcing dentists to administer complete sedation even to provide basic dental hygiene [39, 40].

Herein, we present the research and development process of MyDentist by using the terminology and notation of the Hevner model (Fig. 1). In practice, the design and development of MyDentist is a progressive refinement of the requirements (in the Environment context), the prototype (in the Design Science Research context) and the knowledge base in terms of production of enhanced design guidelines. In the following subsections, we present an overview of the MyDentist functionalities and a rundown of the three Hevner cycles as parts of its global development cycle.

3.1 MyDentist overview

The main goal of MyDentist was to build a path of prevention and treatment for children with ASD using digital resources to: (a) reduce anxiety during visits, (b) teach correct behaviors both at the clinic and at home, (c) motivating them to maintain adequate oral health [17]. We focused mainly on two aspects: promoting children cooperation when sitting on the dental chair (to reduce the use of complete sedation) and teaching healthy routines for oral hygiene at home. Both required: (1) a substantial revision of the clinical protocol to tailor each procedure; (2) the design from scratch of a digital tool (the MyDentist app) to manage personalized digital resources accessible from the clinic, home and anywhere; (3) the definition of the digital resources (games, videos, pictures, social stories, etc.) and rewards to better motivate each child. Using these tools, we had been able to approach the simplest medical procedures: sitting in the dental chair, opening the mouth, exploring oral cavity without and with tools such as mirror and explorer. Only at a later stage, we integrated increasingly complex tools and procedures. During the study we also explored the use of wearable sensors to assess the subject’s physiological responses before and after the dental procedure by applying commercially available devices. These allowed us to recognize any modification in the Autonomic Nervous System activity via signal processing related to electrocardiogram (ECG) and Galvanic Skin Response (GSR), which are two significant and well-known physiological indicators of psychological stress [17, 41].

The MyDentist app is designed as a web application with the aim of facilitating the interaction between clinicians and children with ASD. Specifically, it supports the clinicians in creating/managing users’ personalized profiles and digital material while allowing users and caregivers to access these personalized materials from both clinic and home. In this sense, MyDentist can be seen as a customizable “toolbox” through which dentists can provide personalized digital anxiety reduction interventions for their patients.

The application provides two different access points, one for the dentist and one for the child with ASD. The dentist can create new user profiles and assign them personalized activities such as cognitive games, videos, social stories, etc. Figure 2 shows two screenshots of the dentist GUIs (the material presented does not correspond to real users): on the left the list of patients and on the right the organization of the multimedia material for a specific patient. The activities are accessible by the users whenever they have (or want) to retrace their previous experiences at the clinic, to consolidate the adaptation to procedures, to people and environments and to learn new skills functional to future steps. To build personalized pathways, MyDentist provides generic material that the dentist can select and customize according to the users’ needs. The digital material for each dental visit is created by the dentist personalizing a set of recommended activities to familiarize the user (patient) with the visit procedures. The patient can access the digital activities from home, to prepare for the next visit, possibly with the help of the caregivers and browse photos and videos of past visits (Fig. 3). All multimedia resources and activities are private and personalized taking into account the user’s needs.

Fig. 2

MyDentist: GUI (for the dentist) management of patients and multimedia material

Fig. 3

MyDentist: GUI (for the users): Accessing activities

The MyDentist project received a very good acceptance by the dental professionals as well as by the patients and their families and has achieved a rather good impact. It started with a small scale pilot with 10 users in 2016 and just one year later it became an official health service delivered by the University Hospital AOUP (Azienda Ospedaliera Universitaria Pisana) to around 100 users; 59 of them have been involved in the in-field tests to validate the effectiveness of both the protocol and the digital tools [17]. At the moment, MyDentist is being offered to more than 300 patients and we are negotiating its replication in other University Hospitals in Italy.

3.2 MyDentist relevance cycle

Common limitations found in research works regarding assistive technology for persons with intellectual disabilities include the small number of people involved, high dropout rates and the little impact on the people life [42, 43]. Consequently, a significant effort was made in MyDentist to recruit stakeholders and to build the transdisciplinary group (TD group) [44] including clinicians, therapists and caregivers, and to assign them a role in the design process at an early stage. The effort of recruiting stakeholders also lead to a very large number of patients being involved in the project (currently, more than 300 children with ASD participate in the MyDentist project in the Dental Clinic in Pisa and the project is spawning to other University Clinics in Italy) with a very low drop rate (less than 1%).

The TD group has the role of implementing the requirement elicitation and analysis by integrating and amalgamating knowledge from different backgrounds with a common language. Its work starts from an in-depth study of the targeted literature and continues with the analysis of the problems under all the different perspectives provided by the group members [44], which in our case include clinical protocols, technological aspects, users involvement, economic, organizational and safety aspects. The TD group also analyzes all the aspects of the environment and the protocol (sensory stimulation, sequencing of activities, customization, involvement of siblings or caregivers, room organization, etc.) aiming at removing obstacles and exploiting opportunities. The group has thus identified the set of skills to be taught, the digital activities most likely to be useful, the way in which the level of ability reached in each activity can be measured, and the testing protocol to be carried on. All group members contributed from different points of view: clinicians on the validity of the study, educators on teaching and communicative aspects, caregivers on the practical daily use of digital resources and acceptance, and so on.

The participants in the TD group during the initial phase are shown in Table 1. The ASD therapists and the neuropsychiatrist are members of UFSMIA (Unitá Funzionale Infanzia Adolescenza), a health service providing support for children with disabilities (up to 18 years of age) as a part of the Italian Public Health System (ASL—Azienda Sanitaria Locale). In Tuscany, there are three ASL entities handling the center (C), the north-west (NO), and the south-east (SE) of the Region. Pisa and Livorno are the two largest cities in the ASL-NO. The Professor in the Dental Clinic works also a clinician in the local University Hospital (AOUP – Azienda Ospedaliera Universitaria Pisana) and has a long-lasting experience on oral treatments to children and adults with ASD. The student in the Dental Clinic had outlined the problems of oral health for children with ASD in her master’s thesis and has been treating children with ASD in the Dental Clinic since 2016. The caregivers are members of Autismo Pisa, the largest local association, which brings together the families with children with ASD in the area of the Pisa province in Tuscany. Finally, the Researcher and the senior Technologists from the CNR-IIT in Pisa (Istituto di Informatica e Telematica of the Italian National Research Council) are experts in designing and developing assistive technologies and serious games platforms for persons with intellectual disabilities and ASD.

Table 1 TD group in the first iteration of the relevance cycle

Then, we performed a set of interviews involving the participants in the TD group. The interviews were semi-structured, in order to allow instant feedback by the participants if something was unclear. The questions were written to facilitate the respondents in providing a personal elaboration of their point of view. The questions were designed to bring out from the participants a description of all the oral hygiene problems experienced by children with ASD both at home and in the clinic and to be informed about the personal attitude of each child with ASD toward technology. Questions were divided into five main categories:

• Questions about problems related to dental care for children with ASD (e.g.: “What are the main problems encountered when performing oral care at home and in the clinical setting?”);

• Questions about the user’s motivation for dental care (e.g.: “What motivates your child with ASD to carry on generic activities under request at home or at school? Can this “motivator” be used also for dental care?”);

• Questions about digital tools (e.g.: “In your experience, are digital tools useful to motivate your child with ASD? How can we use digital material to support dental care?”);

• Questions about designing serious games and graphical user interfaces for persons with ASD(e.g.: “How can we convey rewards and positive feedback from the digital tool?”);

• Questions about the hospital’s internal organization and protocols (e.g. “Which are the main problems encountered in dental sessions applying the current protocols? How they could be improved?”).

The output of this first-round interviews was the identification of the following critical aspects to address:

• Organizational setup: the hospital regulations impose strict privacy rules and require complex procedures to authorize the presence of external people during dental sessions. Furthermore, clinical staff is not trained on ASD (how to communicate, react to behavioral problems, etc.) and cannot directly take photos/videos or manage digital tools during clinical activities;

• Environmental and technological requirements: the room used for the dentist’s activities and the waiting room must be quiet, avoiding the presence of other patients and deprived of disturbing visual and auditory stimuli. The waiting time before the session must be minimized. The dentist’s room must be equipped with a robust WiFi connection and tablets;

• Users’ needs: the dentist needs to define personalized patients profiles including all the digital material ensuring security and privacy; caregivers and patients need to access the digital material from home with a simple and easy-to-use graphical user interface;

• Ethical requirements: the digital material is extremely sensible and personal including images, videos and health information; thus, it must be managed to meet all the ethical requirements imposed by laws and hospital regulations.

According to the requirements collection performed during the interviews, we selected a first set of digital activities as candidates to reduce anxiety: galleries of photos and videos of sessions in the dental clinic, video models, social stories, serious games such as puzzles, sequences and memory games. Then, we recruited 10 children to perform a first pilot study which concluded the first iteration of the relevance cycle. The pilot lasted for three months, from April to June 2016 [45]. In Table  3, we summarize the main features of the two rounds of tests of MyDentist. In both rounds, we had to face problems related to a very bad WiFi connection. The clinic is housed in a medieval building with very thick walls that obstruct the hospital provider’s signal. This problem was solved by a dedicated data line donated by the Autismo Pisa association.

In the pilot, we used a small subset of the activities personalized for the 10 children. The clinical protocol was modified to take into account the children’s needs and the use of digital tools and to make it extremely flexible in order to explore different strategies. The pilot provided: (a) a first assessment effectiveness of the clinical protocol integrating digital tools; (b) some preliminary results in terms of user interfaces usability (considering dentists, children and caregivers); (c) an initial set of requirements for the MyDentist app integrating all the digital material.

In the pilot, we evaluated the children’s response mainly with on-field observations (stimulus–response approach) and with interviews to the caregivers. This choice was due to children’s difficulty in expressing their preferences and needs, and hence to the difficulty to obtain valuable indications directly from the users, especially regarding design options that would work best for them. For that reason, a digital humanities PhD student was included in the clinical staff and acted as a digital mediator during the visits. The mediator was in charge of proposing the digital material and observing and annotating users’ reactions. Although this “embedded approach” may be considered not ideal, it gave us the opportunity to identify precisely where and how to improve medical procedures using digital materials, thus contributing to the definition of the final clinical protocol. Summarizing, the pilot showed extremely positive results both regarding the feasibility and acceptability of this approach [45, 46] providing useful requirements that we used in the second iteration of the relevance cycle.

The second iteration of the relevance cycle started from: (a) a more solid clinical protocol and (b) a first prototype of the MyDentist app (as the output of the design cycle, Sect. 3.3) integrating the resources and the activities tested in the pilot. Using the app, the patient could access the digital materials both at the clinic and at home. The transdisciplinary group was enriched with dental hygienists from AOUP, ASD experienced psychologists of IRCCS Stella Maris, two biomedical engineers from CNR-IFC in Pisa (Istituto di Fisiologia Clinica of the National Research Council), and a freelance software developer (see Table 2). The TD group planned a larger test, including biomedical parameters which took place continuously from October 2017 to February 2020 (when it stopped due to COVID-19 restrictions).

Table 2 TD group in the second iteration of the relevance cycle

In this test, we enrolled 59 children with ASD aged 4 to 18 years from October 2017 to February 2020. The goals of the test were to evaluate the usability and the effectiveness of the app as well as the overall effectiveness of the refined clinical protocol using digital tools. In the test, we continued to profit from on-the-field observations but we also collected another type of data through [47]: (i) a Likert scale validated questionnaire to measure the changes in dental habits and protocol effectiveness from time T0 (before starting the intervention supported by MyDentist) to T1 (after 6 months of intervention) [17]; (ii) a standard SUS (System Usability Scale) test administered to caregivers and medical personnel; (iii) the response of the Autonomic Nervous System before and after the dental visit by using wearable devices to obtain an objective measure of anxiety reduction (Table 3).

Table 3 Summary of the two iterations of MyDentist relevance cycle

3.3 MyDentist design cycle

The development of MyDentist is the result of two iterations of the design cycle, which leverage on the requirements elicitation activity of the relevance cycle, which identified the following key points:

• Usability and accessibility: the MyDentist application should be accessible for persons with ASD and their caregivers as well as for the clinical staff. Usability should be high to allow simple and straightforward access without special digital skills both at home and in the clinic;

• Customization: persons with ASD are all different with extremely variable needs, thus the ability to customize the graphical interface and the contents of the application according to the needs of each person with ASD needs is fundamental for the effectiveness of the application;

• Security and Privacy: the app should be robust to failures and ensure careful management of users’ sensitive data;

• Mobile access: access from mobile devices is fundamental to engage caregivers and clinical staff and to make possible to involve children in activities at the clinic.

To meet the requirements on usability and accessibility, the development of MyDentist had to respect also the basic criteria valid for all users, which means that the graphical user interfaces must comply with the standard usability criteria [48]. In our case, the application must work well with the established clinical protocols and be usable by the medical staff, the caregivers and by users with ASD.

For what concerns the customization requirement, in MyDentist we used different digital activities and simple games to attract the attention of users with ASD and to consolidate their learning. We used audio files, video models, social stories, cognitive serious games [49] and collections of photos and videos. The activities used in MyDentist in the different phases are:

• Preparatory digital activities at home: video-modeling exercises, cognitive learning games (puzzles, memories, matching), digital social stories;

• Distracting digital activities for the most stressful moment: pictures and videos taken during sessions, watching distressing familiar videos or playing loved games to reassure and increase the collaboration;

• Reward activities for the relaxing moment, upon the completion of a task: funny videos, easy cognitive learning games already known and digital coloring books.

Furthermore, still to meet the requirements, the MyDentist application also includes the following elements:

• Account management system with a separate accesses for dentists and users with ASD. Dentists can set up users’ profiles, assign them activities personalized according to the characteristics of the user and organize the clinical path through a shared calendar. Users can access personalized activities through a simple and accessible graphical interface;

• Databases to manage the repertoire of activities to be customized, user profiles, appointments for dental sessions and various multimedia contents;

• Data collection to gather a set of data that are the results of the evaluation of standard metrics. For example, we collect data about usage patterns of digital activities; furthermore, on a rather limited number of cases, we gathered data also through sensors embedded in wearable devices. For each metric, the relevant data are collected and then analyzed to assess the effectiveness of MyDentist.

The first iteration of the design cycle focused on the development and validation of: (i) each digital tool such as photos and videos, customized social stories, cognitive serious games, etc.; (ii) requirements elicitation for the app as a container of customizable tools including the graphical user interfaces, workflows and interaction mechanisms. The digital tools we used in this first iteration of the design cycle are better defined as a pretotype [50] rather than a prototype. According to this definition, a pretotype is a set of tools, techniques and tactics designed to validate an idea for a new product quickly, objectively and accurately. The need for a pretotype was motivated by the fact that, at that stage, our aim was to evaluate the feasibility and effectiveness of a totally new approach, avoiding the costs and the effort time requested for a fully working prototype. In concrete, the pretotype gave us the possibility to set up a controlled environment to test materials, digital activities and specific personalized educational paths for a preliminary on-the-field study. The feasibility and the clinical efficacy of the pretotype have been evaluated by on-field observations as described in the relevance cycle (Sect. 3.2).

The second iteration of the design cycle focused on the integration of each component into a prototype web application (the MyDentist app) and on its test. We analyzed both its performance using stand-alone software tests and its impact on the clinical protocol. Observations during the tests spotted many problems which were progressively solved in multiple refinement steps involving both the application and the protocol. Our tests addressed different levels:

• System reliability: we performed extensive unitary and functional testing using standard techniques during all stages of development to assess robustness before going for field tests. Furthermore, We use two separate instances of MyDentist, one is in production (public), and one is for testing new improvements. Only when robustness requirements are fully met, an update is performed on the stable public version of the app;

• System security: security leverages on the use of standard software platforms for the front-end and the back-end of the application and in their proper setting and configuration, which also include the definition of the different roles for the users and of the respective permissions;

• Usability and accessibility, by means of standard tests administered to caregivers, and clinicians, automatic tools and on-the-field observations. For what concerns the testing of the GUI, we relied on observational study steps, both at the clinic and at home. Due to the extreme heterogeneity of the spectrum, in future we plan to find a reasonable division of users with ASD into macro groups, characterized by well-defined interaction ability and set up different protocols for each group;

• Effectiveness of the whole protocol including digital tools, in terms of anxiety reduction and dental care abilities learned, by means of Likert scale questionnaires administered to caregivers at specific points in the protocol. The questionnaires had been used to assess changes in the specific abilities of the children in dental care at home, by monitoring the evolution of children’s behavior in the course of the familiarization path;

• Anxiety reduction: we addressed these aspects only with a group of participants, by carrying quantitative measures of test screening measuring objective physical parameters (blood pressure and heartbeats) pre-visit and post-visit by using wearable sensors.

The results were extremely encouraging and are detailed in [17].

3.4 MyDentist rigor cycle

Following Hevner’s model, the rigor cycle performed in MyDentist project proceeded through two iterations. In both of them, we carried out an extensive analysis of the relevant scientific literature and we worked on adding knowledge to the Design Science Research field.

In the first iteration of the rigor cycle, we made a first reflection on the relevance and design cycles in the development of MyDentist, to produce a first set of guidelines. These are reported in Sect. 4.2 and classified according to the different aspects of development in the relevance and design cycles.

In the second iteration of the rigor cycle, we reconsidered the first set of guidelines and formalized them in a new set of refined and validated guidelines which are the main output of our work and that are presented in Sect. 5. This time, however, the guidelines are organized in a more general way according to thematic areas, to make them more usable by other researchers and practitioners. To this purpose, the second iteration runs a focus group including members of the TD group and additional stakeholders. Participants in the focus group are described in Table 4. Three of them had never been involved in the project before.

Table 4 Participants in the guidelines focus group

The focus group was run online using Miro, Google Drive and Zoom as cooperative platforms and was organized according to the following steps:

• 1. Organization of the guidelines presented in Sect. 4 into three themes: (1) participatory design; (2) motivate and engage stakeholders; and (3) make the clinical environment accessible;

• 2. Sharing of the initial set of guidelines (in Sect. 4) the week before the focus group;

• 3. Addressing ethical and privacy issues: consent form signed by all participants;

• 4. Performing the focus group on Zoom platform with session recording;

• 5. Carrying out a discussion organized as follows:

  • (a) Presentation of the objectives, tools and procedure. In particular, a brief presentation of the three themes of guidelines identified;

  • (b) Each guideline has been presented and discussed to shape its final form and its placement in the right theme;

  • (c) A final discussion that has led to the creation of a forth theme specific to the digital materials and tools.

We provide additional details on steps 4 and 5. The Miro collaborative environment allowed all participants to see and edit a shared virtual whiteboard. Each guideline was shown in a post-it. Logically related guidelines were organized in a group, by themes according to the best practices in Thematic Analysis [51]. We refined one group of guidelines at a time, and, for each group, we discussed every single guideline positioning its post-it at the canvas center, as shown in Fig. 4. Before the discussion, we provided a short period for autonomous reasoning of about ten minutes that allowed each participant to gain familiarity with the tool used and to reflect on the current task to perform. During this time all participants were invited to take notes on doubts, suggestions, general feedback, etc. writing them on a personal box visible on the whiteboard. The discussion began with a moderator reading the text of the guideline aloud and detailing the implicit or explicit meanings and contexts. All participants were asked to provide their feedback. The main questions raised during the discussion focused on the real opportunity for the guideline, its semantics, its potential, its comprehension and re-use by other researchers, its potential to be generalizable in a different context (different from MyDentist), any redundancy and potential overlapping with another guideline. This allowed us to refine the initial set of 36 guidelines, reducing it to a set of 23 elements since a better formalization of concepts expressed drove us to merge some of them. The results and final groups of guidelines are presented and discussed in Sect. 5.

Fig. 4

The Miro whiteboard used in the focus group

4 Lessons learned

This section presents the initial set of guidelines extracted in the first iteration of the rigor cycle. These guidelines reflect the analysis conducted over the relevance and design cycles in the development of MyDentist. In the rest of this section the guidelines extracted from the relevance cycle are identified with an ’R’ and reported in Table 5 and those extracted from the design cycle are identified with a ’D’ and reported in Tables 6 and 7.

Table 5 Initial guidelines factorizing the lessons learned in the relevance cycle

Table 6 Guidelines for user interface design emerging from the experience of MyDentist

Table 7 Guidelines for activities emerging from the experience with MyDentist

4.1 Relevance cycle

The guidelines extracted in the relevance cycle refer to the main phases operated in this cycle, namely: recruiting stakeholders, building the transdisciplinary group, requirement analysis, field testing and management of ethics issues.

4.1.1 Recruiting stakeholders

In general, the recruitment of stakeholders is a complex task. For caregivers, the time limitations associated with their activities of support and care of persons with ASD are the main obstacle to engage them in further activities unless they see the possibility of solving a distressing problem and/or improving their quality of life in short time. The same considerations apply to medical and nursing staff of the dental clinic. Changing clinical protocols, deepening their knowledge of ASD and acquiring digital skills require a considerable amount of time on their part, which is added to a very demanding work scheduling, also worsened by the COVID-19 pandemic. These considerations led us to formulate guidelines R1–R4.

R1

The problem addressed by the research group needs to be perceived as motivating by the stakeholders. In the MyDentist project, being able to provide proper dental care for persons with ASD is a well-known problem in the literature. From a questionnaire administered to 44 caregivers of the local association (Autismo Pisa), 95.5% consider it extremely important to be able to carry out proper dental care and prevention at home and in the clinic (5 on a scale from 1 to 5). The caregivers also described their past experiences highlighting their frustration with the enormous difficulties of following a traditional protocol for dental visits.

On the other hand, the clinicians involved in the Dental Clinic were very frustrated by the overall dental hygiene level of the patients with ASD and by having to resort to complete sedation very often. Their first need was to be able to decrease the use of the operating room and to improve hygiene at home and collaboration in the clinic. They had tried to use videos and photos during visits noticing a beneficial effect in reducing anxiety, so they were very motivated to explore the potential of such an approach in a more systematic way.

R2

Local ASD associations members are a valid help for recruiting stakeholders and need to be involved from the beginning. A very important aspect to avoid dropout is the creation of a relationship of trust and reciprocity between researchers and stakeholders. It is absolutely necessary to let them be fully aware of the meaning of the experiment. Caregivers usually have a trusting relationship with local associations which can be a valid way to explain the benefits of the project and involve them. Representatives of the local association were involved in the transdisciplinary group and helped us to recruit children for the testing, to solve logistic problems for their participation, overcome hospital limitations (e.g. providing a WiFi connection) and to devise a suitable medical and testing protocol.

R3

Stakeholders must be aware of the project and be able to participate with their experience. Resources must be allocated to explain the project and its objectives and to give stakeholders the opportunity to contribute with their observations and suggestions. Another important point is to provide a return on investment in terms of participation benefits.

R4

Stakeholders must have a clear return of their investments in time and efforts in terms of participation benefits. In MyDentist, we analyzed the reasons that had motivated most caregivers to participate through the same questionnaire that we mentioned in the discussion of R1. The most common reasons were: (1) the possibility of participating in the prevention program assisted by dental clinic staff specially trained on ASD and using digital material, (2) the hope of being able to avoid total sedation, (3) having ascertained the effectiveness of the protocol from the first sessions (in compliance with the adaptation times of each patient and the personalization of the path), (4) the short waiting times for appointments and access free of charge, (5) the feeling of being supported in a problem that generates a lot of anxiety in the caregivers. The clinical staff, on the other hand, felt supported in the use of digital tools and by the presence of the technological mediator who helped during the sessions in the clinic.

4.1.2 Building a permanent transdisciplinary workgroup

In the activity of establishing the transdisciplinary group for MyDentist we extracted guidelines R5 and R6.

R5

A transdisciplinary group needs to be selected to represent all the areas of expertise addressed by the project. In MyDentist, we implemented transdisciplinarity involving a number of experts as described in Tables 1 and 2. All the participants in the transdisciplinary group had a strong motivation to work on the problem and to spend effort in finding a common language for working together on a solution. The group worked together throughout the project sharing information by using a Google Group and meeting usually online once a week to discuss problems and solutions related to the project. We actually built a network of interaction at the very beginning of the project to let the group work together, share information and to continue and sustain the design phase after the first pretotype release, quite following the philosophy of design after design [52]. This is because envisioned artifact use is rarely the same as actual use, no matter how much participation has occurred in the design process and increasing the number of users always brings new challenges and opportunities [53].

R6

A common language and interaction network needs to be designed to make the transdisciplinary group work together for the whole life of the project. In our experience, the continuous work of the transdisciplinary group shed light on problems not spotted at the beginning and also addressed issues not directly related to the use of digital tools such as interacting with the hospital administrative offices to solve problems related to accesses, regulations and spaces organization. In MyDentist, we used semi-structured interviews, mockups, drawings and brainstorming to find a common language for requirements collection during the relevance cycle iterations.

4.1.3 Requirement analysis

Concerning the requirement analysis carried by the TD group, we summarize the strategy followed and lessons learned in guidelines R7–R9.

R7

The clinical protocol needs to be modified to remove possible obstacles and to include the use of digital material. In MyDentist, we carefully redesigned the clinical protocol to make it more likely to generate less anxiety for persons with ASD using all the opportunities offered by the characteristics of the spaces and the hospital organizational setup. We set appointments for visits in times of low turnout, sensitized nurses for shift changes, planned a number of visits to get acquainted with the environment and to build a relationship between the children and the clinical staff.

R8

All the clinical staff need to be informed about ASD and on how to deal with patients with ASD. In MyDentist, we have provided both initial training for dentists and nurses on ASD characteristics and issues as well as case support from experienced neuropsychiatrists and therapists during all the project.

R9

Requirements must be elicited in an iterative process using progressively more complex and complete digital tools. Following the Hevner model, we considered the entire prototype development process as a part of the requirement analysis itself. Thus, we started with a first set of experiments with a pretotype including only some of the digital activities devised. This first pilot test made possible the refinement of the clinical protocol including digital tools and the assessment of the activities for the design and implementation of the prototype to be used in the second iteration. In the second iteration, we continued to refine the protocol on the basis of feedback and observations.

4.1.4 Field testing

What to test and how to test are two aspects that need to be analyzed by the TD group referring to standard tests and scales in the relevant literature. The aspects to be assessed are many: system reliability and correctness; usability and accessibility of the graphical user interface for all the categories of users; efficacy of clinical protocol which includes the digital activities. Our experience in field testing is reflected in the guidelines R10–R14.

R10

Use cases/scenarios need to be devised to represent all the user types. Since traditional scales generally do not take into consideration the performance of persons with ASD in specific real-life activities, usually a careful adjustment of them needs to be designed by the experts involved in the transdisciplinary group. In MyDentist, we used an informal set of user-profiles inspired at the Personas model [54] and Empathy maps [55], a questionnaire Likert scale for assessing the improvement in dental care abilities and fairly standard SUS to measure usability.

R11

Activity of caregivers and educators need to be included in the use scenarios. Caregivers and educators were included in the user profiles since they sustain persons with ASD in using technology and learning new abilities.

R12

Direct observation of use in the environment is fundamental to complement interviews with persons with ASD and needs to be embedded in the clinical protocol from the beginning to avoid increasing anxiety during the testing phase. Assessing usability for children with ASD was a main point of discussion since most of them presented important limitations in understanding spoken and written language. This is fairly typical for persons with intellectual disability [56], as several persons with ASD are. To overcome the problem and avoid introducing stressing activity in the protocol, we included a digital mediator in the protocol. The mediator followed the sessions at the dental clinic and observed directly children’s behavior. In this way, during the first phase of knowledge of the environment and personnel, the child records the presence of the mediator as part of the medical protocol, avoiding the stress of getting used to a new person or a new procedure only for a few testing sessions. After the first pilot test, the participation in the activity at the clinic and observation were carried out also with the help of students engaged in internships.

R13

Interviews with caregivers and therapists can help to understand the correct meaning of persons with ASD answers, reactions and behaviors. We interviewed caregivers and educators on the interaction with the app at home and on the correct interpretation of the child’s reactions and behaviors at the clinic.

R14

Inclusion of persons with ASD in the design phase requires to adopt each participant communication style and avoid inaccessible questions. Specifically, Augmentative and Alternative Communication (AAC) can be exploited, attention could be stimulated with finger pointing, a visual Likert scale can be set and a specific training could be activated to make the participant with ASD able to effectively communicate their opinions carrying out an accessible evaluation.

4.1.5 Ethics

All research involving clinical testing needs to be approved by an official ethical committee and, in our experience, interacting with the relevant committee right from the beginning of research design is fundamental to make the correct choices and avoiding wasting time and energies.

R15

Address ethical issues right from the beginning. There are established guidelines, policies and considerations that highlight the ethical implications of involving users with intellectual disabilities as subjects in empirical studies [57, 58]. One important problem to be solved is how to explain the nature of the research and to obtain valid consent from a person with ASD. In MyDentist, all users were either minors or had a support administrator who could provide consent on their behalf. Questionnaires were always filled by caregivers. However, an effort must be done to explain the research and its implications using a communication level that is accessible to persons with ASD (for instance using social stories). Among the ethical issues, there is also the need for proper dissemination of the research work and results, in such a way also the stakeholders could benefit from the results of the research and the introduction of new intervention strategies, training methodologies, software and hardware [58].

4.2 Design cycle

The design cycle provides a second set of initial guidelines that refer to the following aspects: system reliability, graphical user interface, providing a flexible repertoire of digital activities and games, collecting and analyzing measures, and field testing. We discuss all these aspects in the following subsections.

4.2.1 System reliability

The system must provide robustness, being able to cope with errors and erroneous usage, and security [59]. Since some of the users belong to the ASD spectrum, the system needs to be particularly robust. In particular, it should react well to unexpected events, not suffer sudden crashes or slow down and not perform unexpected activities. This is because one of these events may make the user with ASD lose focus, make him/her nervous, and trigger anxiety, aggression or self-harm. Scarce robustness can also demotivate the clinical staff to the point of completely giving up the activity.

The issue of security is particularly important as the application processes sensitive data (very often of minors), in clinical settings producing large quantities of multimedia material and considering that family contexts are often extremely attentive to the protection of their children’s data.

4.2.2 Graphical user interface

Graphical user interfaces need to be developed to make the user’s interaction as simple and efficient as possible, to facilitate the execution of a set of fixed tasks minimizing the user’s cognitive load [21, 60].

All materials have to be comprehensible, operable and predictable for the children. Materials include both digital as well real objects. For digital objects the W3C accessibility guidelines are the main reference. Specifically, the 2021 W3C Working group Note: “Making Content Usable for persons with Cognitive and Learning Disabilities” (W3Cb) provides a useful summary [27]. Persons with ASD are often visual learners, so it is recommended to exploit the user’s visual channel to convey information. Examples of possible material are images, pictures, videos, audios, serious games and video modeling. Digital materials should be accessible, adhering to the W3C WCAG (Web Content Accessibility Guidelines, W3Ca) [27].

Regarding users with ASD, in Table 6, we summarize the guidelines that have proved to be more valuable for our project regarding the graphical user interface (D stands for design). Many persons with ASD tend to rely on caregivers even for activities they are able to do on their own. Thus, it is very important to organize the graphical user interfaces in a way that is completely understandable without the caregiver’s help (D1). Persons with ASD get confused by too many details and sensory inputs and lose focus when they have to decode complex pictures. For this reason, it is important to keep the interface simple without unnecessary embellishments (D2), make easy to predict page content (D3) using similar layouts in each page (D4), provide clear information on how to navigate within the activities (D5) and allow many degrees of customization (D6). Persons with ASD usually find very difficult to follow complex sentences or abstract reasoning thus the language used should be simple and concrete (D7), making possible to include Augmentative and Alternative Communication for users that need it (D11). For persons with ASD struggling with reading, we can use fonts that accentuate the differences between the letters (D8), not too small and with contrasting colors (D9). Finally, text to audio and images can further help to grab the more complex concepts (D10).

4.2.3 Providing a flexible repertoire of digital activities and games

Video modeling is an imitation-based prompting strategy that can be used to increase positive behaviors and aid in skill acquisition in individuals with ASD [61]. After watching over and over someone completing an activity, persons with ASD tend to imitate the activity themselves. Social stories are individualized short stories that depict a social situation that are used to teach communal skills through the use of precise and sequential information about everyday events that a person with ASD find difficult or confusing [62]. Serious games are digital games, created specifically for educational purposes. The review in [49] provides a broad overview of a large body of research in designing serious games to improve the autonomy and cognition of vulnerable people.

Table 7 summarizes the guidelines specific to activity design we found useful in MyDentist. A fundamental point is customization. Each person with ASD is different and can be motivated by different pictures, videos, cartoon characters, colors that can be motivating and that should be used to personalize activities and games. Moreover, some persons with ASD may be upset by particular colors, sounds or images, which need to be excluded from the set. Thus, we need a large selection of activities and digital material (D12) to be able to meet different needs. However, customization should be straightforward and easy to use for the clinical staff. In our case, we designed a unified graphical user interface for activity customization which pilots all the activities used in a given user profile. Clearly, clinical users need clear instructions to drive the customization process (D13).

Choosing a combination of the most appropriate activities for each child is a difficult task that should be accomplished with the information provided by therapists, caregivers and on field observations (D14). The choice depends on the specific abilities and personal tastes of the user and on the goals defined at a given point of the clinical and educational path.

Enjoyment of an activity or a learning game has an impact on the motivation to continue engaging with the subject matter being taught; thus, the activity designed must be fun for the users (D15). A reward, medal or positive feedback should be given, since positive rewards are very effective in consolidating good habits and are widely used in most approaches to ASD treatment (D16). Many persons with ASD have difficulty interacting with mouse and keyboard: the use of a touchscreen is important to make the user more autonomous and keep attention for a longer time (D17). Motivation needs to be given throughout the path, since persons with ASD tend to loose focus on the final goal. In MyDentist, we offered an updated set of activities at each step adjusting the level of difficulty to be stimulating but not frustrating (D18). Parents, therapists, siblings, pets or favorite toys and cartoon characters are a very good source of motivation and should be used to personalize all digital material (D19). As happens in any visual interface (Table 6) we should avoid cluttering and visual burden. The background should be kept as neutral as possible (D20) and the objects in the activity should stand out and have clear edges, possibly cut out (D21).

4.2.4 Collecting and analyzing measures

There are mainly two ways to organize data collection: transparently, (inside the application) or with the cooperation of the user by means of sensors/wearable devices and sensors in the environment. Recording data transparently has the advantage of being completely hidden from the user, thus avoiding distracting his/her attention from the activity he/she is carrying on in the application. On the other hand, transparent data collection does not provide information on clinical parameters which can measure the user reaction to the activity proposed.

4.2.5 Testing

A main issue that emerged in this phase was the testing of the GUIs with users with ASD, which is largely an unexplored task due to the intrinsic complexity of interacting with such persons to perform the task usually required by the user tests.

Another important aspect to test is effectiveness, that is, how well the application and the new clinical protocol teaches the abilities they are supposed to teach. Existing scales need to be adjusted and refined to be able to capture effectiveness in real-world activities. Moreover, the limitations specific to the testing environment and ethical issues need to be taken into account in designing the test phase in the TD group.

5 Guidelines and reflections

This section provides the final set of guidelines that are the result of the focus group run in the second iteration of the rigor cycle. Differently than the first set of guidelines presented in Sect. 4.2, here the guidelines are organized according to thematic areas and thus do not bond to the development cycles of relevance and design. We believe that this makes this second set of guidelines something more general, which may benefit other researchers and practitioners regardless of their design methodology.

The four main themes group and the proposed guidelines are summarized in Table 8.

Table 8 Final themes and guidelines

In the following subsections, one for each theme, we reflect on the relation between the proposed guidelines, the MyDentist project and the related work.

5.1 Theme: make the participatory design drive the whole design process

In MyDentist, we had a transdisciplinary research and development group active from the very beginning of the project (1.1). The group involves all the stakeholders, such as clinicians, psychologists, speech therapists, computer scientists, caregivers, and if/where/how possible, final users. All participants in the group had strong backgrounds (except final users of course) and high motivations to collaborate. The group worked together for the whole project sharing information by using a Google Group and a common working space, with weekly meetings to coordinate the work and manage issues related to the project. The continuous work of the TD group sheds light on problems not spotted at the beginning and also helps to address issues not directly related to the digital material, such as interacting with the hospital administrative offices to solve problems related to accesses, regulations and space organization.

It has been of paramount importance to recruit persons with ASD, their caregivers and dental clinic staff in the whole research and design process to meet each participant’s particular communication style and to avoid accessibility issues (1.2). We used semi-structured interviews, questionnaires, mockups and drawings to share a common language for requirements collection during project iterations. Specifically, we exploited AAC [63,64,65], finger pointing, visual Likert scales, observation and a specific training to make participants with ASD able to effectively provide personal assessment in an accessible way.

The requirements were collected in an iterative process (1.3). We started using stand-alone digital activities (such as games, videos, and social stories) and then used progressively more complete apps which were proposed to the users and used in the clinic and at home.

Regarding uses cases/scenarios (1.4), we used an informal set of user-profiles inspired by the Personas model [54] and Empathy maps [55], a questionnaire Likert scale for assessing the improvements in dental care abilities and fairly standard SUS to measure usability. Caregivers and therapists have an essential role in the definition of use and user scenarios since most persons with ASD them presented important limitations in stimuli receptiveness and in understanding spoken and written language, as generally happens for persons with intellectual disability [66]. Moreover, they were included in the user profiles (in some cases in place or on behalf of their cares) since they sustain persons with ASD in the usage of technology and in learning new abilities.

Scales for assessing user performance were designed by modifying the traditional scales which do not take into consideration specific improvements in dental care [17].

Caregivers and therapists helped also in interpret answers and behaviors of their cares observed during the testing phase (1.5).

To prevent introducing stressing activity in the testing protocol, we included a digital mediator in the clinical team (and in the protocol) right from the beginning (1.6). Observations added fundamental information to the process of defining the most suitable set of digital tools and for identifying how the protocol could be adapted to the needs of each patient. The digital mediator attended all the sessions at the dental clinic and directly observed the children’s behavior. In this way, the patient perceived the digital mediator as part of the medical staff and get used to his/her presence right from the beginning, avoiding the stress derived from introducing new unknown people in the testing phase. The role of the digital mediator was covered by PhD students or undergraduate students engaged in internships. In order to collect exhaustive data about the app usage in all the different contexts, we asked the caregivers to observe the app usage at home and to release their feedback through an online questionnaire.

5.2 Theme: motivate and engage stakeholders from the early phases

This theme includes knowledge, awareness and motivation and in general requires applying persuasive techniques [67]. However, in MyDentist, we had no need to persuade clinicians, therapists and caregivers since dental care was already perceived as a very important problem by all the stakeholders [17].

Involving local ASD associations from the beginning of the project is crucial to be able to recruit stakeholders and set up the transdisciplinary team (2.1). Stakeholders must include caregivers and end users with ASD. They must be aware of the project and they should have the opportunity to share their personal experience. Local ASD associations representatives usually have a trusted-based relationship and can give a fundamental help to make caregivers and users understand the goals and benefits of the project and to reduce dropouts.

There must be allocated time resources to explain the project and its goals and to give the stakeholder the opportunity to contribute with their observations and suggestions (2.2). In MyDentist, representatives of the local associations were involved in the transdisciplinary group and helped us to recruit children for the testing, to solve logistic problems for their participation and to devise a suitable testing protocol (2.3).

It is crucial to apply communication and persuasive strategies to make research objectives clear, appealing and useful, in order all the stakeholders, including clinicians, caregivers and end users to perceive these targets as fundamental (2.4). Research projects that are based on a participatory design and test model require voluntary membership of many people with different roles who provide their time and effort for something that does not yet exist but is being tested. Voluntariness must not be taken for granted and in any case, it is necessary to give something in return to each figure according to their role. In MyDentist, the benefit was tangible from the beginning because there was no transition phase from the research laboratory to real life. The most appreciated benefit was being able to participate directly in the experimental pathway of prevention and cure, then having their children with ASD cured with professionalism and attention to their safety as well (e.g., trying to avoid complete sedation).

5.3 Theme: make the clinical environment suitable to children’s needs and to the use of digital tools

Adapting the clinical environment was a fundamental building block for the success of MyDentist (3.1). To adapt the physical environment, we choose the quietest hours of the day and we removed too distracting sensory stimulation from the rooms in order to reduce possible sources of anxiety of the patients.

All the clinical staff was trained on how to deal with patients with ASD (3.2). This was crucial to provide coherent feedback to patients from all the staff members, to make easier patient-staff communication and to increase the predictability of stimuli during interactions.

Also, some of the staff were not very familiar with digital material production and use. Thus, we provided personalized training sessions for all those who needed it (3.3).

We carefully redesigned the clinical protocol to generate less anxiety for the users and to include the use of digital material (3.4,3.5). We used all the possibilities offered by the physical characteristics of the rooms and by the hospital’s organizational structure. We set appointments for visits in times of low turnout, sensitized nurses for shift changes and planned a large number of short visits at the beginning to make the patients with ASD get acquainted with the environment and to build relations between the children and the clinical staff. We added a digital mediator to the medical staff normally present during sessions (3.6). This had different goals. First, taking pictures and videos or proposing digital activities during sessions may be very difficult for the clinical staff since it is engaged in actual dental care activities. Thus the digital mediator takes care of proposing and producing digital material without interrupting the dental care procedures. Second, we need to observe the reactions of children while using the digital material to elicit requirements in the iterative process and to test the material and the app. Adding a digital mediator allows to carry on observations without changing the environment perceived by the patient and without the need of building new relations during the testing phase. Third, the digital mediator can be trained for interacting with children with ASD and to cope with their behavioral problems and help the clinical staff to deal with any loss of regulation or behavior problem that might emerge during sessions.

5.4 Theme: make the children familiar with clinical and digital materials

Being generally well-accepted by children with ASD, digital tools can mediate the familiarization process. There is no specific digital material suitable for everyone or every situation, but a constellation of digital solutions that can cover a wide spectrum of end-users’ needs to help them perform each specific goal-directed activity.

In MyDentist, we created a repertoire of digital material accessible to persons with ASD (4.1). In preparing the material, we followed the guidelines in Sect. 4.2.3  (Tables 6 and 7).

The material is available in the MyDentist app and can be personalized according to the needs of each child (4.2).

In particular, we used photos and videos of the clinic and of the staff to make the child familiar with the environment before starting the program. Photos and videos are available from the app and each time the child comes to the clinic new photos are taken including the child and his/her caregivers to help remembering and reinforcing (4.3). Photos are taken directly from the MyDentist app and are automatically collected in the user’s repository.

During the program, the child needs to learn a set of skills related to dental care at home and at the clinic and some social skills such as asking for a break or asking to go to the bathroom. For instance, he/she needs to learn to sit in the dentist’s chair, to keep his/her mouth open for several seconds, to brush his/her teeth properly at home. For each skill, we developed templates of social stories, video models, serious games that can be personalized and that help the child in the learning effort (4.4).

Rewards are very important for motivation. We have selected a set of tangible and digital rewards that are generally very welcome; furthermore, we have provided the possibility to expand the settings for taking into account the specific preferences of a particular child (4.4). Rewards are used during the sessions in the clinic and in the digital activities.

Digital material can also be of great help to reassure and relax during activities at the dental office (4.5). For this, it is important to identify with the help of caregivers and therapists a set of pleasant and relaxing digital activities to be proposed to overcome moments of anxiety and lack of collaboration.

The digital material proposed at home and in the clinic needs to be adjusted often according to the progress of the patient in learning habits and behaviors (4.7). The activities must be chosen with great care, adding a little difficulty and novelty each time but avoiding too difficult activities that can frustrate the child’s curiosity and involvement. In MyDentist, the dentist may review digital activities after each session to keep them in line with the child’s progresses.

6 Conclusion

MyDentist is a project that addresses dental care in children with ASD. It offers digital tools and a clinical protocol with the objective of reducing anxiety and teaching proper dental care to children with ASD, while avoiding the typical problems that may often occur in this context.

The project has already achieved the maturity and robustness sufficient to be adopted as an official service of the university hospital AOUP in Pisa, serving now more than 300 users. The users visit the clinic every week for the initial part of the prevention program or are actively monitored once every two months in the follow-up program.

The development process of MyDentist has been contextualized in several iterations of the three cycles (relevance, design and rigor) of the Hevner model. The main contribution of this work is a set of guidelines for the development of e-health software for children with ASD, which had been presented in Sect. 5.

The study explains the priorities of software development for e-health and the specific need for managing the relationship between different stakeholders around children with ASD. We provide practitioners with a better understanding and awareness of their own context, helping them make technical and health-related decisions of a sustainable character.

This study provides validated empirical evidence for development practices in software development for e-health. We encourage researchers to explore the long-term effects of development of digital tools and new clinical procedures and their inter-relations.

There are, however, several limitations to this study. Having based our study on qualitative measures, the results and implications are subject to bias.

Another limitation concerns the involvement of only a small number of interviewees with specific knowledge on clinical protocols and everyday medical activities who are also aware of the challenges of the use of technology.

Finally, another shortcoming to the study is the diversity of the investigated practices, as the selection constituted data from one hospital. The study would profit from a wider collection of data, both to discover more relevant themes and to ensure credible conclusions (i.e., generalization of the results). Further investigations of software development for e-health in different contexts, age of children and various geographical locations can improve the reliability of the research results.

The adoption of MyDentist is now being considered by other hospitals in Tuscany and Sardinia (Italy). Moreover, future work consists of planning and execution of future iterations of the relevance, design and rigor cycle with the objective of improving the robustness and scalability of the MyDentist app, and of adding functionalities for the automatic analysis of the data collected. Moreover, considering the increasing number of end users involved, the field tests of further iterations will also result in a further refinement of the guidelines. Future work should verify the results in other e-health contexts to find its applicability in other environments, enabling generalization to a larger audience. More investigations should be undertaken to understand the role of scope in the development activities of e-health support.