1 Introduction

People with Autism Spectrum Disorder (ASD) present persistent limitations in their verbal and non-verbal communication (American Psychological Associationn, 2022). According to the DSM-5™, these students may present phonological-syntactical Language Disorders (LD) which affect phonology and morphosyntax in their expression and reception. This may be accompanied by the use of short phrases, omissions, or the poor use of linkers; impaired pronunciation; and delayed echolalia, associated with repetition of phrases or expressions some time after hearing them. There may also be other, lexical-syntactic disorders, associated with lexical, morphosyntactic, or word-evocation difficulties (paraphrasing, circumlocution, pseudo stuttering, etc.) as well as semantic-pragmatic disorders, reducing the meaning and use of language (Cordón & Torrijos, 2021; Marzo & Belda, 2021).

As indicated by O'Keeffe and McNally's research review (2023), the linguistic domain has always generated significant interest within studies related to individuals with Autism Spectrum Disorder (ASD), given the consequences of these limitations in daily life and other areas such as social, familial, or professional settings. Specifically, Communicative Competence (CC) is understood as a speaker's ability to communicate effectively in different contexts (Gumperz & Hymes, 1972). Celce-Murcia (2007) defines Communicative Competence based on six dimensions: sociocultural, formulation, interactional, linguistic, strategic, and discursive. On the other hand, the European Parliament and Council (2006) conceive it as being related to socio-emotional skills and empathic capacity to understand different viewpoints and be tolerant, among other aspects.

The Council of Europe (2001) indicates that Communicative Competence (CC) —in neurotypical people— includes linguistic competencies (phonological, lexical, and grammatical), socio-linguistic competencies (body language, sounds, and interjections), and pragmatic competencies (discursive and functional). However, students with ASD do not develop these skills in the same way as neurotypical individuals. It is even common for them to use the Total Communication System (Schaeffer et al., 1980), which is based on the simultaneous use of speech and sign language to establish communication with the individual, as well as verbal, tactile, and visual supports. They may also utilize the Picture Exchange Communication System (PECS) (Bondy & Frost, 1994), which enables individuals without linguistic abilities to interact and communicate using images; or Augmentative and Alternative Systems of Communication (AASC), image-based instruments of expression to reduce the communication deficit, supporting speech or using images as an alternative to spoken language (Syriopoulou-Delli & Eleni, 2021). Therefore, the stimulation of CC in these students’ early years needs specific interventions that strengthen both their linguistic skills for internalizing lexical, phonetic, and semantic rules (Marzo & Belda, 2021) and socio-emotional skills related to the use of verbal and non-verbal language depending on the context (American Psychiatric Association, 2022).

Interventions go from using pictograms (Martin et al., 2019; Torrado et al., 2017) to incorporating digital resources, characterized by their versatility, flexibility, and adaptability (Durán, 2021). Specifically, there are various studies focused on the opportunities of digital applications for students with Autism Spectrum Disorder (ASD) (Gallardo et al., 2021; Del Moral and López-Bouzas, 2020), with some specifically designed for this purpose. There are applications designed to activate language precursors, communicative intention, and social behaviour (Allen et al., 2016; Jiménez et al., 2017), as well as to stimulate interpersonal relationships (Alharbi & Huang, 2020; Papoutsi et al., 2018). Augmented Reality (AR) apps are also used which activate linguistic abilities in everyday communication contexts (Taryadi & Kurniawan, 2018), apps related to reading skills (Kolomoiets & Kassim, 2018), and apps related to literary skills (Arief & Efendi, 2018). Other apps stimulate socio-emotional skills through recognizing and responding to facial expressions of emotion (Chung & Chen, 2017), while others improve social interaction (Lee et al., 2018).

Nowadays, the mechanics, dynamics, and aesthetics of games are adopted (Zichermann & Cunningham, 2011) in gamified interventions with successful outcomes, immersing the student in enjoyable activities around social initiation (Malinverni et al., 2017). Gamification applied to stimulating communication increases students’ motivation, producing long-term changes in behaviour (Van Dooren et al., 2019). In addition, incorporating stories and characters in Serious Games is a source of stimulus and models for the development of interpersonal relationships (Griffin et al., 2021).

Most interventions have been aimed at high-functioning people with ASD or Asperger’s (Fridenson et al., 2017; Terlouw et al., 2021). However, this research stems from an intervention aimed at low-functioning students with ASD —belonging to the same socio-cultural context— supported by the design of an Augmented Gamified Environment, in order to analyse its contribution to stimulating their language skills for naming and associating objects with their names, as well as socio-emotional skills to identify emotions and relate them to their causes. Unlike other studies resulting from innovative interventions targeting individuals with ASD using AR (Lee et al., 2020; Mota et al., 2020; Politis et al., 2017), this research specifically focuses on low-functioning individuals with ASD in Special Education centres. Therefore, it represents a novel intervention that provides data not only for high-functioning individuals with ASD. Additionally, it utilizes the playful narrative of a highly popular animated series among child audiences, which combines gamification, Augmented Reality, and cinema to facilitate learning and the development of communicative skills.

2 Description of the Gamified Augmented Environment

A Gamified Augmented Environment (GAE) is “an immersive digital space that combines enjoyable learning with AR-supported activities, promoting student immersion in their learning process through interaction with digital devices” (López-Bouzas & Del Moral, 2023). For this study, we created the GAE From Cabin-boy to Captain: in search of the lost treasure (https://bit.ly/3VGKxWR) to stimulate CC in students via a tailored gamified intervention. The environment is framed by a pirate story adopting the metaphor of a voyage. The missions are presented through activities which combine digital and AR resources (iOS and Android versions) complemented by YouTube videos to enhance immersion in the story, encouraging the multi-sensorial involvement of the student (Fig. 1).

Fig. 1
figure 1

Source: author’s own work

GAE screens.

Full size image

The missions are organized in three game levels, and there are adaptations for different developmental levels (Table 1).

Table 1 Adaptations included for the interaction in each game level
Full size table

The environment includes a feedback system based on Cabrera (2020): prescriptive, relying on the characters to produce feedback through speech challenging the student to guide how they do the task; evaluative, telling them if they have done tasks correctly; descriptive, explaining how to do the task correctly; and interrogative, inviting them to think about the activities. Each mission has prizes and rewards—gold doubloons in this case. Amassing rewards allows the player to move forward in the story, win prizes at the end of each mission (spyglass, hook, pirate hat), and ultimately achieve the rank of captain.

The GAE was designed in line with the criteria of the Universal Design for Learning framework (Carrington et al., 2020). The content is presented using various codes (text, speech, information windows, images, videos, etc.) and links to visual and audio-visual resources, posters, sheets, and cards, etc. giving variety to how information is presented to encourage student interest and motivation, and to help the student to immerse themselves in a pirate adventure in which they are the protagonist. The present study aimed to analyse how much an intervention based on this GAE would contribute to stimulating Communicative Competence in students with ASD.

3 Methodology

This was empirical, non-experimental research, with non-probabilistic sampling. The study was correlational, and based on the types from Cohen et al. (2011), it was exploratory and analytical. Due to non-normal distributions, non-parametric tests were used for comparing the means between the study variables.

3.1 Objectives

The study objectives were to: 1) analyse the extent to which this gamified augmented environment stimulated students’ Communicative Competence —linguistic and socio-emotional skills; 2) examine the relationship of that competence with students’ levels of immersion in the environment and interaction with the tablet; and 3) determine the relationships between competence levels and students’ gender, age, severity of ASD, comorbidities, and type of language.

3.2 Sample

The intervention took place between February and June 2022 with students who had been previously diagnosed as having Autism Spectrum Disorder by the Asturian Department of Education and who attended special education schools. The sample was limited to schools whose families agreed to take part in the intervention. A total of 54 students participated from three special schools: C.P.E.E Castiello de Bernueces (N = 26), C.P.E.E Latores (N = 18), and C.P.E.E Juan Luis Prada (N = 10). This gave a representativeness index of 91.5%. The final sample was 68.5% boys and 31.5% girls with ages ranging from 3 to 17 years old (3–6 years: 9.3%; 7–14 years: 50.1%; 13–17 years: 40.9%).

Over a third (35.2%) presented comorbidity, as 22.2% had attention deficit or hyperactivity and another 13% had, generally, motor disabilities. In terms of language, the majority (68.5%) presented spoken language, although only 38.9% had functional spoken language because 29.6% had Language Disorders (LD) commonly associated with ASD: phonological-syntactic disorder (12.96%), delayed echolalia (12.96%), lexical-syntactic disorder (3.7%), and semantic-pragmatic disorder (3.7%). In addition, 25.9% of the students did not have spoken language and 5.6% used Augmentative and Alternative Systems of Communication (AASC).

The severity of students’ ASD was determined by the Inter and Interpersonal Measuring Scale for the Level of ASD in Infancy and Adolescence (EMIGTEA) (α = 0.832), adapted from the DSM-5™, and based on the Childhood Autism Rating Scale (CARS) (Schopler et al., 1988). The severity level was based on the results in eight aspects: Consistency of the intellectual response; Level of activity; Inflexibility of behaviour; Use of objects; Visual and auditory response; Social interaction; Social communication; and Emotional response. We set three levels: mild (values between 1.00 and 0.67), moderate (0.66–0.34) and severe (0.33–0.00). Almost half of the participants (46.3%) presented a severe level of ASD, 29.6% moderate, and 24.1% mild. Figure 2 shows the distribution of the students by characteristics and severity of ASD.

Fig. 2
figure 2

Source: author’s own work

Sample distribution by classification values and severity of ASD.

Full size image

3.3 Procedure

  1. 1.

    Phase I. Informed consent was obtained from the children’s legal guardians, in line with the recommendations for research with children (Shaw et al., 2011) and the standards in the Declaration of Helsinki (World Medical Association, 2008).

  2. 2.

    Phase II. Development and validation of the instrument for evaluating students’ Communicative Competence —defined from their linguistic and socio-emotional skills exhibited during the intervention using the GAE along with the level of interaction with the tablet and immersion in the environment (its validation is described in the following section). This was subsequently validated via confirmatory factor analysis.

  3. 3.

    Phase III. Individualized intervention with the children, lasting a total of 120 h 28 min, a mean of 2 h 20 min per student. They were invited to play in an everyday, fun, non-disruptive setting, ensuring the intervention’s ecological validity.

  4. 4.

    Phase IV. Once the individual sessions were finished, a systematic record was made of observations using designed instrument, as suggested by Lokesh (1984). The level of interaction with the tablet and the level of immersion in the environment were assessed. Separately, the students’ linguistic and socio-emotional levels as reflected in their journey through the GAE were evaluated. The Communicative Competence (CC) variable was created from the mean values of these scores.

  5. 5.

    Phase V. Data analysis. In addition to descriptive statistics based on frequencies, percentages, and means, the correlations between variables were analyzed using the Spearman's Rho coefficient test. Significant differences were found between variables using the Mann–Whitney U test for those with two categories of grouping, and the Kruskal–Wallis test for variables with three or more categories of grouping. Additionally, the effect size produced by these grouping variables was calculated using Hedges' g.

3.4 Instrument

The instrument for evaluating Communicative Competence was defined based on the theoretical constructs of the DSM-5™ (American Psychological Associationn, 2022). It is made up of 12 variables referring to students’ linguistic and socio-emotional skills as demonstrated during the tasks set for them in the various GAE screens. These variables are measured through 48 items using Likert-type scales (0 = Not at all, 1 = A little, 2 = Somewhat, and 3 = a lot) (see the instrument: http://bit.ly/3k3CTrg). It also includes 4 additional items for determining the level of interaction with the Tablet and 5 for measuring the level of immersion in the environment, based on the classification from Haggis-Burridge (2020). While more indicators could be included, given the specific characteristics of these individuals and their lack of or limitation in oral language or associated comorbidity, they have been limited following the recommendations of the DSM-5™.

Validation was via exploratory factor analysis due to the sample size. The method of maximum verisimilitude was chosen—based on Lloret-Segura et al. (2014)—with an eigenvalue of > 1 chosen as the criterion, producing values of each variable which explains the total variance (Table 2). Bartlett’s sphericity test was significant (p = 0.000) and they Kaiser-Meyer Olkin (KMO) test for suitability gave a high value (KMO = 0.919).

Table 2 Total variance explained by each variable and matrix of components
Full size table

Table 2 shows that a single variable was able to explain more than 75% of the variance in the results. In addition, the matrix of components shows the variables grouping around a single factor. This interpretation is confirmed by the sedimentation graph (Fig. 3) and the high level of goodness of fit of the data to the model (χ2 = 0.000).

Fig. 3
figure 3

Source: author’s own work

Sedimentation graph.

Full size image

The factorial analysis confirmed the validity of the unidimensional instrument for measuring communicative competence.

4 Results

During the intervention with the GAE, as the difficulty of the missions increased, fewer students passed the levels (Table 3), as their individual characteristics affected their performance.

Table 3 Completed levels and missions in the GAE
Full size table

Students’ engagement with the challenges was assessed from their levels of interaction with the tablet and immersion in the environment. Just over half (51.9%) interacted with the tablet autonomously, whereas 27.8% needed prompting and 20.4% only interacted with it when told to. In general, students’ level of interaction was high (x̄ = 2.31), particularly for students with mild levels of ASD (Mild: x̄ = 3.00; Moderate: x̄ = 2.69; Severe: x̄ = 1.72; p = 0.000), and those with functional spoken language or who used AACS (without spoken language: x̄ = 1.79; AACS: x̄ = 3.00; with LD: x̄ = 2.19; functional spoken language: x̄ = 2.67; p = 0.003). In terms of levels of immersion in the environment, 31.5% of the students managed to immerse themselves in the story and sequencing in the GAE. About a fifth (20.4%) immersed themselves in the system with a similar proportion managing to empathize at a social level with the characters. A smaller proportion (14.8%) only managed to immerse themselves on a spatial level, and 13% failed to achieve any immersion. In general, the immersion was moderate (x̄ = 2.37), and greater for students with mild ASD (Mild: x̄ = 3.62; Moderate: x̄ = 3.06; Severe: x̄ = 1.28; p = 0.000) and those with functional spoken language or who used AACS (without spoken language: x̄ = 0.71; AACS: x̄ = 3.33; with LD: x̄ = 2.44; functional spoken language: x̄ = 3.29; p = 0.000).

4.1 Correlations between variables

Table 4 shows the highly significant correlations between almost all the variables (at the 0.01 level).

Table 4 Correlations between variables
Full size table

4.2 Linguistic and socio-emotional skills

The missions sought to stimulate Communicative Competence through challenges involving linguistic and socio-emotional skills. The results for each mission are given below.

4.2.1 Mission one: Stimulation of linguistic skills

In the first level, students had to name and match characters with the colour of their clothes. Half of the students (50%) named and matched more than three characters, demonstrating moderate skills (x̄ = 2.09). Students with mild and moderate ASD demonstrated better skills (Mild: x̄ = 2.77; Moderate: x̄ = 2.69; Severe: x̄ = 1.36; p = 0.000), as did those with functional spoken language or who used AACS (without spoken language: x̄ = 0.79; AASC: x̄ = 3.00; with LD: x̄ = 2.13; functional spoken language: x̄ = 2.81; p = 0.000). In the second level, students had to name and match characters with their accessories. Over a third (37%) named and matched more than 3, demonstrating moderate skills (x̄ = 1.83). Older children had higher levels of skills (3–6 years: x̄ = 1.00; 7–12: x̄ = 1.67; 13–17: x̄ = 2.23; p = 0.037), as did those with mild ASD (Mild: x̄ = 2.62; Moderate: x̄ = 2.44; Severe: x̄ = 1.04; p = 0.000) and those who used AASC or had functional spoken language (without spoken language x̄ = 0.57; AASC: x̄ = 3.00; with LD: x̄ = 1.81; functional spoken language: x̄ = 2.52; p = 0.000). In the third level, students had to name marine elements from a story with AR. The complexity of this challenge produced lower scores for skills (x̄ = 1.46). A third (33.3%) named between 3 and 5 elements, another 31.5% failed to name any. Older children had higher scores (3–6 years: x̄ = 0.60; 7–12: x̄ = 1.30; 13–17: x̄ = 1.86; p = 0.000), as did those with mild ASD (Mild: x̄ = 2,38; Moderate: x̄ = 2,00; Severe: x̄ = 0,64; p = 0,000) and those with functional spoken language or who used AASC (without spoken language x̄ = 0,00; AASC: x̄ = 2,67; with LD: x̄ = 1,50; functional spoken language: x̄ = 2,24; p = 0,000).

As the difficulty of the challenges and levels increased, the scores for the skills fell. The success rate was 58.9%. Twelve subjects, mostly boys, aged between 11 and 17 with mild ASD, functional spoken language, and without other associated disabilities achieved the maximum score in the three levels.

4.3 Mission two: Socio-emotional skills

In the first level, students had to name emotions represented by icons and match them with facial expressions. Only 35.2% correctly matched all the emotions. The performance was moderate to low (x̄ = 1,70). Older children scored higher (3–6 years: x̄ = 0.80; 7–12: x̄ = 1.52; 13–17: x̄ = 2.08; p = 0.037), as did those with mild or moderate ASD (Mild: x̄ = 2.46; Moderate: x̄ = 2.44; Severe: x̄ = 0.84; p = 0.000) and those with functional spoken language or who used AASC (without spoken language: x̄ = 0.00; AASC: x̄ = 3.00; with LD: x̄ = 1.94; functional spoken language: x̄ = 2.48; p = 0.000). In the second level, they had to match emotions to their causes. The students exhibited moderate skills (x̄ = 2.09). Almost half (44.4%) matched happiness, sadness, and anger, while another 37% also matched fear and surprise. Girls demonstrated better skills (Boys: x̄ = 1.92 vs. Girls: x̄ = 2.47; p = 0.033), as did older children (3–6 years: x̄ = 1.80; 7–12: x̄ = 1.85; 13–17: x̄ = 2.45; p = 0–038), those with mild ASD (Mild: x̄ = 2.69; Moderate: x̄ = 2.50; Severe: x̄ = 1.52; p = 0.000), and those who had functional spoken language or used AASC (without spoken language: x̄ = 1.29; AASC: x̄ = 2.67; with LD: x̄ = 1.63; functional spoken language: x̄ = 2.00; p = 0.000). To pass the third level, students had to recognize their own emotion matched with a colour, used previously to paint a boat. The complexity of this task is indicated by the low levels of skill demonstrated (x̄ = 1.50). Over a quarter (27.8%) of the students did not recognize their own emotion, while a similar proportion recognized it and painted the boat correctly. Older children demonstrated better skill (3–6 years: x̄ = 0.60; 7–12: x̄ = 1.22; 13–17: x̄ = 2.05; p = 0.011), as did those with mild and moderate levels of ASD (Mild: x̄ = 2.15; Moderate: x̄ = 2.19; Severe: x̄ = 0.72; p = 0.000), and those with functional spoken language or who used AASC (without spoken language: x̄ = 0.36; AASC: x̄ = 2.67; with LD: x̄ = 1.63; functional spoken language: x̄ = 2.00; p = 0.000).

The students exhibited better abilities in the second level because the activity was presented with the Merge Cube, making manipulation and exploration of virtual elements easier. The success rate was 58.8%. Nine subjects achieved maximum scores in the three levels, most were boys, aged 10 to 17, with light and moderate levels of ASD, functional spoken language, and without comorbidities.

4.3.1 Mission three: Linguistic skills

In the first level, students had to name and match animals based on Quiver sheets they had to colour in. They exhibited moderate skills (x̄ = 2,17). Just over half (51.8%) matched and named 4 to 6 animals. The highest levels of skills were demonstrated by those with mild ASD (Mild: x̄ = 2.85; Moderate: x̄ = 2.75; Severe: x̄ = 1.44; p = 0.000), and those with functional spoken language or who used AASC (without spoken language: x̄ = 0.79; AASC: x̄ = 3.00; with LD: x̄ = 2.31; functional spoken language: x̄ = 2.86; p = 0.000). In the second level, students demonstrated low skill levels (x̄ = 1.54), identifying and verbally describing animals (size, colour, habitat, etc.) using the Merge Object Viewer. A third (33.3%) identified the animals but did not describe them, 25.1% identified and described 1–6 animals, whereas 24.1% identified between 4 and 6 animals. Girls exhibited higher levels of skills (Boys: x̄ = 1.38 vs. Girls: x̄ = 1.88; p = 0.099), as did those with mild ASD (Mild: x̄ = 2.46; Moderate: x̄ = 2.00; Severe: x̄ = 0.76; p = 0.000) and those with functional spoken language or who used AASC (without spoken language x̄ = 0.43; AASC: x̄ = 2.67; with LD: x̄ = 1.38; functional spoken language: x̄ = 2.24; p = 0.000). The third level involved comparing animals using AR flashcards, meaning an added difficulty, reflected in the low levels of skills (x̄ = 1.59). Less than a third (31.5%) compared colour and size, while 27.8% also identified habitats. Another 27.8% did not manage to compare animals. The highest levels of skills were shown by those with mild ASD (Mild: x̄ = 2.54; Moderate: x̄ = 2.06; Severe: x̄ = 0.80; p = 0.000) and those with functional spoken language (without spoken language: x̄ = 0.36; AASC: x̄ = 2–33; with LD: x̄ = 1.44; functional spoken language: x̄ = 2.43; p = 0.000).

As the linguistic difficulty of the challenges and levels increased, fewer students completed them. The success rate was 58.8%. Nine students achieved maximum scores in the three levels, most of whom were boys, aged 10–17, with mild levels of ASD, functional spoken language, and without comorbidities.

4.3.2 Mission four: Socio-emotional skills

In this mission, students had to help the characters find treasure using a geolocation AR app. The task was simple, but there were a few details to consider to do it properly. First level: simple route, accompanied by an adult, with visual and oral explanation of the route; second level: map-based search following indications; third level: autonomous search with a map. In the first level, the students demonstrated high skill levels (x̄ = 2.52). Almost three-quarters (74%) found the treasure and identified the emotional state of the characters after succeeding. The highest levels of skill were demonstrated by those with mild or moderate ASD (Mild: x̄ = 2.85; Moderate: x̄ = 2.94; Severe: x̄ = 2.08; p = 0.002) and those with functional spoken language or who used AASC (without spoken language: x̄ = 1.79; AASC: x̄ = 3.00; with LD: x̄ = 2.69; functional spoken language: x̄ = 2.81; p = 0.006). In the second level, there was a moderate level of skill (x̄ = 1.83). Just under half (48.1%) found the treasure and identified the characters’ emotions. Older children performed better (3–6 years: x̄ = 1.00; 7–12: x̄ = 1.56; 13–17: x̄ = 2.36; p = 0.031), as did those with mild ASD (Mild:x̄ = 2.62; Moderate: x̄ = 2.31; Severe: x̄ = 1.12; p = 0.000) and those with functional spoken language or who used AASC (without spoken language: x̄ = 0,71; AASC: x̄ = 3,00; with LD: x̄ = 1,75; functional spoken language: x̄ = 2,48; p = 0,000). There were low levels of skill in the third level (x̄ = 1,11). Only a quarter (24.1%) found the treasure and identified the characters’ emotional states. The highest levels of skills were exhibited by those with mild ASD (Mild: x̄ = 2.15; Moderate: x̄ = 1.38; Severe: x̄ = 0.40; p = 0.000) and by those with functional spoken language or who used AASC (without spoken language: x̄ = 0.14; AASC: x̄ = 2.67; with LD: x̄ = 0.88; functional spoken language: x̄ = 1.71; p = 0.001).

As the tasks became more difficult, fewer students managed to complete them. The success rate was 60.7%. Thirteen subjects achieved maximum scores in all three levels, most of whom were aged 10–17, with mild or moderate levels of ASD, functional spoken language, and without comorbidities.

Only four students managed to achieve the maximum score in all twelve levels: three boys and one girl. Three of them had mild ASD, the other had moderate ASD. They had good mastery of language—two with functional spoken language and two with mild delayed echolalia—and two had other syndromes or physical disabilities that did not affect cognitive function. Two of the participating students did not complete any of the missions: two boys (aged 3 and 7) with severe ASD, comorbid ADHD, and without spoken language. It is also worth noting that students who also presented with ADHD had significantly different results from those without this comorbidity (p = 0.000) due to their difficulties in maintaining attention, hyperactivity, impulsivity, etc.

4.4 Communicative competence

To determine the students’ overall competence level during the intervention, the variable Communicative Competence (CC) was created from the mean scores in the indicators related to linguistic and socio-emotional skills; overall, students had medium–high levels of CC (x̄ = 1.94). The distribution of the subjects -according to the level of achievement- was measured in four categories: very low (0.00–0.25), low (0.26–0.50), medium (0.51–0.75), and high (0.76–1.00). Just under half of the students (48.1%) had a high level of CC, 20.4% had a low level, 16.7% very low, and 14.8% medium (Fig. 4). The highest levels of CC were in older children (3–6 years: x̄ = 1.20; 7–12: x̄ = 1.70; 13–17: x̄ = 2.41; p = 0.033), those with mild or moderate ASD (Mild: x̄ = 2.69; Moderate: x̄ = 2.69; Severe: x̄ = 1.94; p = 0.000), and those with functional spoken language or who used AASC (without spoken language: x̄ = 0.50; AASC: x̄ = 3.00; with LD: x̄ = 1.94; functional spoken language: x̄ = 2.76; p = 0.000).

Fig. 4
figure 4

Percentage distribution by CC level, considering the level of ASD. Source: author’s own work

Full size image

Comparing the level of interaction with the tablet during the intervention with CC scores, those with high levels of competence had more autonomous interaction with the tablet (Fig. 5). In contrast, students with low levels of CC tended to only interact with the tablet when told to, and those with medium levels usually needed prompting of some sort to interact with it.

Fig. 5
figure 5

Source: author’s own work

Percentage distribution of the sample by CC level and level of interaction with the tablet.

Full size image

Comparing the CC level with the level of immersion in the environment, those with very low levels of CC managed basic immersion in the system, those with low levels of CC managed spatial immersion, those with medium levels of CC achieved empathic-social immersion, and those with high levels of CC had the highest, narrative-sequential, level of immersion which allowed them to feel part of the story (Fig. 6). Students’ CC levels were directly related to their levels of immersion in the environment.

Fig. 6
figure 6

Source: author’s own work

Percentage distribution of the sample by CC level and level of immersion in the GAE.

Full size image

Lastly, the effect size for each variable on the CC score was determined, with < 0.2 = null; between 0.2 and 0.5 = small; between 0.5 and 0.8 = moderate; > 0.8 = large (Table 5).

Table 5 Effect size of the classification variables (V) on CC scores
Full size table

The effect size on Communicative Competence varies depending on the classification variables. Specifically, the degree of Autism Spectrum Disorder (ASD) has a significant effect on CC, as observed in students with severe ASD (26 out of the comparisons), while this effect is barely noticeable in those with mild to moderate ASD. Similarly, language type has an impact on CC, as 58 out of the 78 comparisons conducted show values above 0.8. In these cases, the group most affected in their competence is the one without language, followed by those using Augmentative and Alternative Communication (AAC). There is also a moderate effect size when comparing subjects with functional language with other typologies (11 comparisons).

Comorbidity has a moderate or large effect on CC, especially when comparing subjects with Attention Deficit Hyperactivity Disorder (ADHD) with others (30 comparisons = 14 moderate + 16 large). The effect is small or null between students with no comorbidity and those with multiple disabilities (9 out of 39 comparisons). The effect of age on CC varies. It is large between younger and older subjects (11 comparisons). It is moderate when comparing the results of the 3–6-year-old group with those of the 7–12 and 13–17-year-old groups, respectively (13 comparisons). The effect on CC is small or null in the 7–12-year-old group, as it shows little difference with both the 3–6-year-old and the 13–17-year-old groups (15 comparisons). The effect of gender on CC is small (7 comparisons) or null (5 comparisons). However, it is worth noting the moderate effect in relation to the ability to associate emotions with their causes (V5 with a value of 0.61).

5 Discussion and conclusions

The results show that this Gamified Augmented Environment can promote Communicative Competence in students with ASD, although it would need systematic implementation. Both the level of interaction with the tablet and the immersion in the environment were positively and significantly correlated with competence level. Both the level of interaction with the tablet and immersion in the environment are positively and significantly correlated with competency levels, as noted by Paquet et al. (2022). This correlation is logical since tasks are integrated into a digital environment that allows individuals to interact independently, explore the environment freely, establish their own pace for activity execution, and allocate time accordingly. Furthermore, the narrative allows for a direct connection with characters, promoting immersion and interaction with them, as highlighted by Griffin et al. (2021).

Engagement and socio-emotional skills increase when characters are presented in audiovisual settings, as demonstrated by Alkinj et al. (2022). In this case, the augmented reality effect further enhances interaction with the tablet and immersion in the environment. On the other hand, visualizing interpersonal and intrapersonal relationships of the characters along with contextualizing their emotions and directly appealing to the player facilitates feedback that influences their response, providing an opportunity to acquire and develop their language skills (Hussain et al., 2021) and socio-emotional abilities, consequently stimulating their CC.

Thus, the higher the degree of interaction and immersion with the challenges presented in the game, the greater the students' cognitive competence, and vice versa as observed in similar studies supported by the use of apps for CC stimulation (Del Moral et al., 2022). Overall, students with better linguistic and socio-emotional skills engaged more with the tasks. It's important to note that older students with less severe ASD, functional oral language, and no comorbidity exhibited higher levels of CC. Clearly, the type of interaction and immersion facilitated by the environment can stimulate CC through interactions with characters and various screens, contributing to achieving game objectives, solving presented challenges, gathering information through audiovisual elements that enrich the playful narrative, and so on.

In this research, it has been observed that the use of Augmented Reality (AR) promotes an increase in attention, facilitating the stimulation of CC in this student population, as evidenced by the study of Kolomoiets and Kassim (2018), who noted an improvement in linguistic skills in a similar sample. Similarly, socio-emotional skills have been activated through interaction with augmented characters and the completion of tasks to help them achieve their goals in the game, in a manner similar to the findings of Taryadi and Kurniawan (2018) when utilizing augmented reality resources. In addition, incorporating game mechanics and dynamics (missions, rewards, feedback, etc.) and the typical short-term objectives of gamification have succeeded in engaging students in the game, fostering motivation that supports the learning of vocabulary related to the marine and pirate world, as well as the identification and association of emotions with their causes, as reported by Lee et al. (2020).

Girls exhibited better skills in challenges requiring students to identify and describe animals, and to recognize and match emotions. Older children had greater linguistic and socio-emotional skills. In the present study, the level of ASD was directly related to performance in the tasks, with students who had severe ASD having the greatest difficulties, as also noted by Silva et al. (2021). Students with functional spoken language or who used AASC demonstrated higher levels of competence, as they already have linguistic skills allowing them to communicate and engage with the environments’ challenges.

It should be emphasized that students who also had ADHD demonstrated greater limitations in doing the activities—in line with Kokol et al. (2020). The level of CC, and particularly the level of linguistic ability, was directly related to the ability to complete the challenges in each mission, many of the students suffered from language disorders, or lacked language, which affects how they perform in vocabulary tasks, as noted by McMahon et al. (2016). The complexity of the tasks calls for a base cognitive level to be able to do them, and two students whose abilities were very limited did not manage to do any of the missions. In contrast, it might be interesting to add a fourth difficulty level for those who easily completed the other three.

There were positive results following the implementation of the GAE, which allowed expression of the subjects’ linguistic and socio-emotional skills during the intervention. More specifically, the competency level of the students was evident when carrying out various missions. It was observed that a greater immersion in the challenges corresponded to higher levels of CC in students. Specifically, older students, those with less severe ASD, functional oral language, and no comorbidity exhibited higher levels of CC. In conclusion, to achieve optimal results, the intervention should be tailored to individual characteristics, present engaging narratives, and integrate playful activities that require communicative strategies.

Furthermore, among the pedagogical implications arising from this research, it is worth noting that the playful context attracts these students and promotes their learning.

The use of tablets and/or digital resources in conjunction with immersive playful scenarios with augmented reality enhances their engagement with educational tasks. Attractive environments should be designed that require interaction with 3D characters whom they must assist in achieving set objectives, establishing feedback that generates linguistic and/or emotional responses, while promoting their engagement with playful missions. The narrative used in the designed environment has captured the students' attention, so that other similar narratives could be designed, expanding areas of interest and associated vocabulary, eliciting responses and/or communicative intentions. It's important that they feel like protagonists in the story, facilitating their emotional involvement and identification with the characters.

Interventions could combine traditional resources with emerging technologies. Other Augmented Reality applications with simpler or more complex elements could also be integrated. Undoubtedly, the creation of such resources requires training teachers in digital competence so they can design and combine active methodologies like gamification with emerging technologies like Augmented Reality. Regarding the study's limitations, it would be advisable to have a pre- and post-intervention measure to facilitate the study of the students' progression, to identify individual competency levels, and to determine the specific issues of each subject.