1 Introduction

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by deficits in social communication and repetitive patterns of behavior American Psychological Association, [1]. Children on this spectrum face challenges in maintaining attention on externally imposed stimuli [7, 24]. An imposed stimulus is simply targeted objects amidst other irrelevant information. For example, children with ASD in a classroom might focus on classmates rather than the board. The ability to pay attention to relevant information is critical for intellectual and adaptive functioning for individuals with ASD [24]. More specifically, children with ASD require life-long support and have little chance to make sense of the surrounding environment independently. Therefore, careful consideration of learning environments for children with ASD is essential for developing their learning skills.

Consequently, children with ASD are often enrolled in a special school with a one-to-one method of teaching. Children in special schools have fewer classroom distractions and peer interruption as compared to mainstream schools. Unlike special schools, mainstream schools provide children with ASD the opportunity of social skills development [39]. Recently, the inclusion of children with ASD in mainstream schools has gained momentum due to its potential benefits and governmental policies on equality of opportunity [35, 42]. The paradigm of inclusive education has lead to several studies on the effect of mainstream schools and special schools on children with ASD. Findings from these studies reported mixed results on social interaction skills development [14] and academic performance [30].

Studies on social skills of children with ASD showed that participants in mainstream schools exhibited more positive social interaction skills than those in special schools [5, 6]. Other studies have reported that children with ASD face challenges with social interaction in mainstream schools [20, 21]. For instance, children with ASD find it difficult to make eye contact, thus interacting with peers becomes challenging. A recent review of the academic performance of children with ASD has reported that there is no significant difference between participants from mainstream and special schools [39]. The review study analyzed the performance score on a national curriculum test in 5 subjects: English, Reading, Writing, Mathematics, and Science. Another study on inclusive education showed that the academic performance of children with ASD is mutually associated with attention [18]. It is worth noting that attention interruption in a classroom at the time of a curriculum test might vary across participants. Thus, research on factors affecting attention in a classroom scenario that is uniform across all participants might provide useful information about how to support academic success in children with ASD.

Attention assessment using neuropsychological tests and virtual reality (VR) can provide a uniform testing scenario across all participants. This assessment is also referred to as a VR classroom [12, 22]. A VR classroom is implemented either with a continuous performance test (CPT) [9, 16] or Stroop task [15]. More specifically, in CPT tasks, users respond to target letters within a set of random letters [8, 37] or digits [36] displayed on the board. Meanwhile, the Stroop task uses the effect of mismatching color and text: the text ‘white’ is displayed in black ink, instead of white. Consequently, participants earn points for correct responses or matches. These neuropsychological tests are often used to evaluate attention to relevant information [27]. VR supports the design of ecological validity by mimicking natural learning environments using 3D objects.

Ecological validity refers to the relation between the scenario in a real-world setting, and the experimentation of this scenario in a setting created by the researchers [29]. In a VR classroom, studies use distractions which might happen in a real classroom to simulate an ecological validity of attention assessment. The first VR-CPT was implemented with head-mounted devices (HMD) to create full classroom immersion for participants [26]. The VRCPT simulates a typical classroom setting with a teacher in front of the classroom, students and blackboard that displays the random letters. VR-CPT provides effective content for testing cognitive inhibition with several distractors in individuals with ASD [9, 15, 40]. Similarly, the performance score of individuals with ASD in a VR-Stroop test showed that distractions affect participants' attention [23]. The authors suggested that classroom distractions can be a contributing factor to academic or attention assessment in children with ASD.

Aside from performance scores, the behavioral pattern of attention can enhance rich attention assessment [11]. An eye-tracking analysis consists of objective techniques that are being used to assess the behavioral patterns of attention in children with ASD. Some of the eye-tracking parameters used for assessing attentional behaviors include gaze position and pupil diameter for joint attention [2], and fixation duration for sustained attention [3]. Eye-tracking devices observe participants’ gaze patterns without the need for body-mounted technologies. Other objective methods such as monitoring brain signals or heart rate, could present challenges as they require on-body attachment.

The reviewed studies on performance of children with ASD in classroom environments show that attention distractions can hinder academic performance and engagement during learning. A simulated learning environment can provide the same scenario for all participants and can therefore give a fair attention assessment. These studies show that the effect of the learning environment can be assessed through performance scores. However, detailed information on how classroom distractions affect attention has not yet been gathered. For example, interrupting students in the classroom may have more of an impact on their attention than furniture distraction. Therefore, further research is needed on how classroom distractions affect attention of children with ASD.

Thus, this study employs eye-tracking measures along with performance scores to investigate the effects of different classroom distractions on the attention of children with ASD. The objectives of this study are: (a) to identify how children with ASD react to externally imposed stimuli in the presence of other distractions in a real classroom environment; and (b) to compare and investigate attentional behavior of children with ASD and typically developing (TD) children. Findings from this study will support the design of classrooms and provide knowledge of what attentional support is required by children with ASD.

2 Method

2.1 Participants

A total of 45 children (20 with ASD and 25 TD) between the ages of 7 and 11 years participated in the study. The ASD group consisted of 17 boys and 3 girls with mild to moderate ASD, while the TD group consisted of 17 boys and 8 girls. The ASD participants were recruited through a special school for children with ASD and the Qatar Autism Society, formed by parents of children diagnosed with ASD. The inclusion criteria for children with ASD were: (a) that they were within the age range of 7 and 11 years; and (b) that they had been diagnosed with mild or moderate ASD. The inclusion criteria for TD children included children who were not diagnosed with ASD or any developmental disability.

We obtained informed consent from each participant’s parent. All the participants met the inclusion criteria for our study; the participants from ASD groups presented a medical report indicating their ASD diagnosis. Additionally, parents of the participants completed the childhood autism spectrum test (CAST) questionnaire. This questionnaire was developed by the autism research center at Cambridge University, UK to identify the severity and possibility of ASD [41]. We also asked further questions to identify any form of visual impairment or other physical issue that might have hindered a participant’s ability to take the test.

This study divided classroom settings into three categories: mainstream, special schools, and home. In this study, ‘children in mainstream schools’, refers to those children who attend a typical school and receive one-to-many instructions. The phrase ‘children in special schools’, refers to those children who attend a special school and receive one-to-one tutoring. Lastly, the phrase ‘children in home schools’ refers to those who receive one-to-one tutoring by their caregivers at home. Information about the participants is provided in Table 1.

Table 1 Demographics of participants with ASD
Full size table

2.2 Set up

2.2.1 Stimuli: desktop VR-CPT

We developed a desktop VR-based attention test that simulates a CPT in a mainstream classroom (Fig. 1). The decision to use a desktop option was based on a study about VR applications for individuals with ASD [4]. That study stressed that the use of head-mounted devices (HMD) can induce dizziness and other uncomfortable sensations. Moreover, individuals with ASD tend to experience unusual sensory reactions to stimuli [33, 34], which include tactile senses such as touch, in relation to objects. Thus, HMD may not be suitable and can interfere with the outcome measurement [31]. Unlike HMD, desktop VR-CPT does not fully immerse participants in a classroom environment. However, previous studies have shown that desktop VR simulations can ensure ecological validity. For instance, pilot studies with teachers on assessing the usability of desktop VR applications found that they were effective in simulating social skills [38] and street-crossing skills [13] in children with ASD.

Fig. 1
figure 1

Desktop VR-CPT

Full size image

2.2.2 Materials

The testing room consisted of two monitors, one each for the participants and the researcher, sized 24 inches and 34 inches, respectively. For the participants, we attached the screen-based Tobii X2-60 eye-tracking device to the base of the 24-inch monitor. The adoption of screen-based eye-tracking is less likely to interfere with the research outcomes [28]. Four eye-gaze measures were analyzed with iMotions, a commercial software tool. This software provides a real-time evaluation of eye-gaze and other biometric measures [10]. We used the keypad in place of a keyboard as seen in Fig. 2 to avoid possible distractions.

Fig. 2
figure 2

Real time view of eye tracking Analysis

Full size image

3 Procedure

We conducted the attention task experiment in an isolated and dimly-lit room to prevent external distractions. All participants were provided with a manual explaining the experiment, and each participant received the same instructions. The researcher explained the experiments to the participants, paying particular attention to how they would perform the attention task and eye-gaze calibration. This introductory session also ensured that each participant was able to settle in to the environment. The instructions for the attention test were in the form of visuals and text that were given in a hardcopy format. Each participant was seated in turn in front of the 24-inch monitor, which had a screen-based Tobii X2-60 eye-tracking device attached. We conducted the eye calibrations with the screen-based eye-tracking device. The eye calibration ensures the tracking device captures the gaze of the participants accurately.

During the eye calibration, participants were asked to gaze at an animated white circular shape displayed at five different points on the screen. The iMotion software evaluated the calibration as poor, good or excellent. For participants with poor eye calibration, the eye calibration exercise was repeated at least two times. The participants in the ASD group were allowed to perform a trial run of the calibration and VR-CPT before the main experiment, to ensure they understood the task correctly. The main attention test experiment lasted for five (5) minutes. Participants earned points based on their performance of the attention tasks in the VR classroom. The maximum possible score was 40, and each participant was awarded points for accurate clicks. These scores were later used to evaluate attention levels. Meanwhile, as the participants took the test, the eye-tracking device gathered attention data.

4 Data analysis

In total, 18 participants from the ASD group and 25 TD children took the VR-CPT attention test. We tracked the performance scores and gathered the eye-tracking data of all the participants for analysis. Fourteen areas of interest (AOIs) were manually created, using objects and students in the VR classroom. The 14 AOIs are shown in Fig. 3 as: (1) board; (2) a teacher; (3) a front desk with stationery; (4) ‘Student 1’; (5) ‘Student 2’; (6) ‘Ceiling Light 3’; (7) ‘Ceiling Light 2’; (8) ‘Window 2’; (9) notice board; (10) ‘Ceiling Light 1’; (11) ‘Student 3’; (12) ‘Student 4’; (13) ‘Student 5’; (14) and ‘Window 1’. We analyzed the AOIs generated using four eye-tracking metrics which are commonly used to measure attention. These metrics include:

Fig. 3
figure 3

Area of Interests (AOIs)

Full size image

(a) Time to first fixate (TTFF): This indicates which object was the first to be of interest to the participants when the stimuli were initially presented. The least duration time is recorded for the first AOI the participants look at in the stimuli. A sequence of fixations is indicated in the orange circles shown in Fig. 4 where fixation number 1 indicates that the board was the first object (AOI-1) viewed, while the third fixation was on the student in the front (AOI-4).

Fig. 4
figure 4

Sequence of fixation in VR-Classroom

Full size image

(b) First fixation duration (FFD): This measures the length of time the participant spends focusing on the object in the presented stimuli. The size of the orange circles shown in Fig. 4 depicts the duration of fixation; the bigger the circle, the higher the fixation duration. It can be seen in the same figure that the fixation duration in sequence 2 is more than 3.

(c) Average fixation duration (AFD): This metric gives the mean values of the total fixation duration for each AOI.

(d) Sum of fixation count (SFC): This metric calculates the number of fixations made on each AOI for viewing time.

(e) Sum of fixation count (SFC): This metric calculates the number of fixations made on each AOI for the entire task.

5 Results

This section presents the performance scores and eye-gaze patterns for all the participants as recorded in the VR-CPT attention test. The first section compares the performance scores of the two groups of participants. The subsequent section presents the four eye-gaze measures.

5.1 Performance scores

The performance scores of participants from the ASD and TD groups provide the answer to the question expressed by the first research objective: how do children with ASD and TD attend to externally imposed stimuli? Two participants from the ASD group could not click or call out the letters, because they were unable to sit calmly enough to take the test. Additionally, four other participants from the ASD group were unable to focus on the screen at the same time as clicking. Consequently, when they focused on the board they called out the letters randomly; thus, no score was assigned for these participants.

The overall performance of the participants from both groups showed that 96% (24 out of 25) of the participants from the TD group scored between 30 and 40 points, whereas only 55% (11 out of 20) from the ASD group scored points in that range. Poor performance (that is, scoring below 20 points) was recorded by 35% (7 out of 20) of the ASD group, while 10% (2 out of 20) of that group were recorded as being in the average range (see Fig. 5). The average score of participants from the ASD group was 22.5 points and that of the TD group was 37.5 points. This difference in scores was assessed by a nonparametric Mann–Whitney test, and the result showed that the difference in scores was statistically significant (z = 3.55187, P < 0.05).

Fig. 5
figure 5

Comparison of VR-CPT scores between ASD and Typical groups

Full size image

5.2 Eye-tracking metrics

The analysis of eye-tracking metrics from 33 participants out of 45 from both groups (ASD = 10 and TD = 23) was used to address the second research objective: to compare and investigate attentional behavior of children with ASD, and of TD children. We excluded twelve participants from the eye-tracking metrics analysis due to invalid eye calibration results (ASD = 10 and TD = 2). This study used four eye-tracking metrics commonly used in attention assessment studies. These metrics include TTFF, FFD, AFD and SFC, all of which are related to fixation patterns. Fixation patterns are illustrated in Figs. 6 and 7 by way of samples of such patterns, taken from one participant in each group.

Fig. 6
figure 6

Fixation Pattern of a Participant with TD

Full size image
Fig. 7
figure 7

Fixation Pattern of a Participant with ASD

Full size image

Subsequently, the raw data generated by the Tobii eye-tracking device was exported to excel using the iMotions biometric research software. Using the generated raw data we conducted the analysis of the four identified eye-tracking metrics. The analysis is discussed in subsequent sections.

5.2.1 Time to first fixate on AOIs

The ASD and TD groups exhibited different durations for the 14 AOIs in the VR classroom as shown in Figs. 8a, b. The first and second objects of interest in the ASD group were the board and the teacher, respectively. However, it took ASD group longer time to fixate on the teacher than the TD group. In addition, the difference in time to first fixate patterns between the two groups indicates that the ASD group fixated on other objects outside the defined AOIs. Of the two groups, the ASD group showed more interest in irrelevant AOIs such as the desk with stationery, the notice board, and the ceiling lights. The two groups also differed in relation to the fixation order for the last three AOIs. The TD group showed sequential fixation on the ceiling lights, while the ASD group showed haphazard fixation. The ASD group fixated on the student, then the ceiling light, and lastly, the window. However, the differences in the TTFF between the two groups are not statistically significant (z = − 0.299, P > 0.05).

Fig. 8
figure 8

a TTFF on 14 AOIs for the ASD group. b TTFF on 14 AOIs for the TD group

Full size image

5.2.2 First fixation duration on AOIs

The FFD is an essential metric because it provides details of the intensity of participants’ interest in the first AOI. The result of the FFD (see Fig. 9) showed that the two groups’ FFD was almost similar except for the board (AOI-1). The average time spent by the ASD group on the board was less (172.87 ms) than the TD groups (440.22 ms). This can be interpreted to mean that the ASD group was distracted. The FFD records for the ASD group also show that the participants’ attention to the board was divided between the teacher (AOI-2) and Student 1 (AOI-4). The difference in attention to the board between the groups is not statistically significant (z = 1.527, P > 0.05).

Fig. 9
figure 9

FFD on 14 AOIs for both groups

Full size image

5.2.3 Average fixation duration on AOIs

The relationship between the performance scores and AFD in the ASD group showed linear correlation that is similar with the TD group. The scores for all the participants in the TD group are almost equal. The two groups showed similar AFD on all AOIs except on AOI-1 (board). The TD group showed higher AFD than the ASD on AOI-13 (Student 5) as shown in Fig. 10. However, this difference is not statistically significant (z = − 0.058, P > 0.05).

Fig. 10
figure 10

Average Fixation Duration (AFD) on 14 AOIs for both groups

Full size image

5.2.4 Sum of fixation count on AOIs

A comparison between the ASD and TD groups for the SFC on AOIs shows that there was a linear relationship between SFC on the board and performance score. However, the scores recorded for the SFC on the board were higher in the TD group (70) than in the ASD group (21) as shown in Fig. 11. The difference was statistically significant (z = 2.958, P < 0.05). The SFC differences between the ASD and TD groups were reflected in the performance scores, where the TD group’s scores were higher.

Fig. 11
figure 11

SFC 14 AOIs for both groups

Full size image

6 Discussion

This study focused on the effects of a classroom setting on children with ASD. We investigated how classroom distractions affect the attention of children with ASD and TD when reacting to externally imposed stimuli. The study contributes to the understanding of eye-gaze attentional behavior of children with ASD in a mainstream classroom setting.

6.1 Attention distribution of children with ASD in a VR classroom

Our findings showed that children with ASD are quick to give attention to externally imposed stimuli, such as an instruction to identify the letter X in a VR-CPT attention test. Among the 14 AOIs, children with ASD fixated first on the board from among the 14 AOIs, and did so for a minimal time of 1.531 ms. Similarly, the TD group also fixated on the board as the primary object of interest among the 14 AOIs but took longer than the ASD group did to start, with a minimal time of 3.413 ms. Thus, the two groups took glances at the externally imposed object—the board—but the amount of time spent on it at first glance was minimal in children with ASD as compared to the TD group. The ASD group spent less time on the board because their attention was drawn to other objects in the classroom.

This finding is supported by a similar study conducted in a virtual classroom involving individuals with ASD whose attention was affected by other distractions in the classroom [23]. In addition, other studies have stated that children with ASD tend to dwell on objects which are of interest to them and spend less time on relevant objects [7, 25, 32]. This distraction of attention exhibited in the ASD group may be linked to the theory of repetitive technique used in regaining the attention of children with ASD to desired learning content [19].

6.2 VR-CPT score assessment

More than half of the ASD group (n = 11/20) scored above 30 in the VR-CPT score assessment, and therefore performed excellently in comparison with participants from the TD group (n = 25). The eleven participants from the ASD group who scored above 30 had all previously been exposed to using computers. Six out of the eleven participants from the ASD group attend mainstream school, four are from special schools and one is homeschooled. However, two out of the ASD group (n = 2/20) who attend special schools and are used to learning with a computer achieved average CPT scores. The rest of the participants from the ASD group (n = 7/20) had below-average scores. Meanwhile, two of the twenty participants could not seat still to take the attention task. One of the participant’s parents requested that, instead of a computer, drawing materials could be used to motivate their child; the child remained on the drawing activity throughout the experiment. The other participant walked around the experiment room throughout. The remaining five, who are from special schools or are homeschooled, were able to call out random letters from the board but were not able to click on the mouse. The difference in performance scores among the ASD group may be due to the fact that some of them are schooled in mainstream schools and some of them in special schools, as well as the fact that not all of them had been exposed to the use of a computer.

Furthermore, the findings showed that some children with ASD were able to look at the board and click on the mouse when the letter X appeared, while others were unable to do the two actions together but could still call out all the letters as they appeared on the board. This experiment showed that a test requiring children with ASD to pay attention while looking and responding might not be an effective way to assess their attention, because combining these two actions at the same time as paying attention might be difficult for some of them. Moreover, looking and clicking as a form of response are factors used for assessing attention in conventional CPT tests, yet it might not be effective when assessing attention in children with ASD. Thus, further investigation should be done on simultaneous assessment of actions of children with ASD to assess their attention levels.

6.3 Mainstream school implications

Transferring children with ASD to mainstream schools is usually challenging for parents due to the fear that their child may get bullied by other students [17]. Some of the fears that the parents nurture are centered on behavioral and attention management. While our findings have shown that children with ASD were quick to focus their attention on the target stimuli—the board, as was seen in the TD group—they spent less time and fixated less when revisiting the board in comparison with the TD group. This implies that children with ASD may need a learning support assistant who can consistently reorient their attention to required stimuli to support their learning experience. Also, for both groups, the first object of interest in the VR-Classroom was the board. This may imply that both groups follow instructions in a similar way, but that the stimuli required sustaining their attention differs. Participants in the ASD group showed less sustained attention on the board as compared with participants in the TD group. This finding is supported by previous literature on the challenges facing children with ASD in terms of sustained attention.

Furthermore, children with ASD who are exposed to the mainstream school environment performed better in the VR-CPT exercise compared with those in other schools. The differences were significant (z = − 2.33, P < 0.05), although the sample size is small to be generalized.

7 Limitations and future work

There are some limitations to this study despite the critical findings on the attentional behaviors of children with ASD in a VR-simulated classroom. First, while the simulated classroom had classroom distractions, the students in the virtual classroom looked more like high school students than grade-level students. Grade level classrooms do have more of wall pictures aside a notice board. Thus, this may affect the level of distractions presented to the participants.

Second, the number of participants in each class setting may not generalize to other children with ASD. Therefore, a greater number of participants from the three different learning environments would give a better understanding of how the learning environment affects attention. Third, the attention stimuli used in our experiment were based on letters, so it may not be possible to generalize this outcome for other attentional behaviors such as reading. Fourth, this study has only considered eye-gaze patterns and not the child’s mood. Information about participants’ mood states would provide greater insight into the inattention to externally imposed stimuli. To validate the findings of this study, further studies using a VR classroom are needed to examine the attention distribution in children with ASD on other subjects, such as vocabulary learning and reading.

8 Conclusion

Learning environments in mainstream schools involve more distractions than those in special schools. Nevertheless, inclusive learning for children with ASD is becoming popular due to its potential benefits and due to government policies. Knowledge about attentional distribution on learning content in such an environment is essential to achieving optimal learning outcomes. In this context, this study shows that children with ASD experience challenges in paying attention to relevant stimuli in a VR classroom setting. However, the study suggests that constant reorientation support can enhance the sustained attention of children with ASD.