Illiteracy is still an unjust social reality of many societies. Despite the advances in education technologies, literacy learning outcomes remain poor in low- and middle-income countries. Current statistics indicate that an average of 70% of children cannot read and comprehend a basic text by the age of 10 (World Bank, 2022). In India, the context of this study, the average reading-with-comprehension level stands at 62% in Grade 3 and 52% in Grade 5 (Ministry of Education, 2021). While India is inching towards its goal of universal access to education, enrolment and schooling have not yet translated into high-quality learning for all. Literacy attainment is confronted with challenges such as inadequacies in teaching practices, variance in home learning environments, disadvantageous teacher–student ratio, and lack of support for students transitioning from their first language to using English as the medium of instruction in schools (Muralidharan et al., 2019). At home, some parents lack fluent reading skills in their first language, thus hampering development of supportive learning environment (Central Square Foundation, 2023). In school, classroom instruction predominantly focuses on choral reading, spelling drills, and handwriting practice (Menon et al., 2019). It has been estimated that a substantial proportion of teachers have low preparedness for facilitating effective English reading acquisition for their students (Shenoy et al., 2020).

To overcome these challenges and support learning outcomes, novel digital tools have been called for to enable greater equity, effectiveness, and efficiency in education (UNESCO, 2023). Education technology (EdTech) is recognised for its potential to support learners facing learning difficulties and without access to teaching and learning resources. Moreover, India’s National Education Policy 2020 emphasises the use of EdTech for improving learning experiences and outcomes, classroom instruction, and teacher professional development (Ministry of Human Resource Development, 2020). Supported by national mandates, over 24 state governments in India have committed to integrating EdTech across ~ 98,000 public schools by 2023 (Goyal et al., 2022). However, it is important acknowledge that EdTech alone is not as effective as when integrated into classroom instruction and requires more evidence on when and for whom EdTech works (see Rodriguez-Segura, 2022 for review).

The current study investigates the effectiveness of classroom phonics instruction and a computer-assisted reading intervention (CARI) tool—GraphoLearn Rime (GL) versus classroom phonics instruction alone for foundational English literacy skills development. To understand the predictors of learning gains, the study also explores the relationship between students’ attendance to classroom phonics instruction and English literacy outcomes. This study adds to previous findings on GL’s effectiveness for supporting struggling readers in India (Patel et al., 2018, 2022) and to the indication that integration of teacher instruction and GL may be more effective than GL alone (McTigue et al., 2019). The study, thus, aims to contribute to knowledge on EdTech efficacy and inform EdTech-enabled reading intervention programmes in India.

Phonics instruction: a road ahead in India?

Defined in the national curriculum for Grade 1 and 2, curricular goals for literacy and language development include phonological awareness and ability to read and write simple words in students’ L2 (National Council of Educational Research & Training, 2022). In most English-medium public schools in India, English is taught as an L2 in Grade 1 and onwards. Further, even though the National Education Policy 2020 emphasises using students’ mother tongue or first language for primary instruction till Grade 5, many states, viewing English as a language of upward social and economic mobility, are committed to increasing the number of English-medium schools (Lightfoot et al., 2022). However, the growing demand has not translated into high-quality English literacy instruction in schools. The Annual Education Survey Report (ASER Centre, 2022), a large-scale, countrywide survey, reported that only about 55% of Grade 3 children in India can read and describe the meaning of simple words in English such as sun, bus, and cat, and only 30% of Grade 3 children can read and comprehend Grade 1 level texts. English literacy instruction in India predominantly utilises the alphabet-spelling method in which letter names are taught along with pictures of object names starting with the target letter and students chant letter names with the help of alphabet charts (Gupta, 2014). Students are taught frequently used words and sentence construction through rote memorisation (Dixon et al., 2011). Also, recommendations in the curriculum for literacy instruction do not explicitly indicate utilisation of phonics instruction. Clearly, these practices are not aligned with the evidence-based foundational English literacy development approaches of which systematic phonics instruction is at the centre (Castles et al., 2018) and appear to fall short for attaining the literacy goals laid out in India’s education policy (Ministry of Human Resource Development, 2020; National Council of Educational Research & Training, 2022).

Systematic phonics instruction is the method with strongest empirical support for developing English decoding skills and reading comprehension (Castles et al., 2018; Ehri et al., 2001). Systematic instruction is characterised by a specific scope and sequence, building on prior knowledge, and moving from simple to complex skills (Ehri, 2020). In addition, phonics instruction is considered most effective when it is explicit; explicit instruction entails direct instruction and explanation of a concept, modelling of the concept’s application, and guided practice combined with feedback (Piasta & Hudson, 2022). In English, the 26 letters (graphemes) represent 44 sounds (phonemes) and understanding the relationship between a single letter/letter combinations and sounds aids in decoding words. Thus, a goal of phonics instruction is to understand patterns and generalisations of letter-sound connections for accurate decoding (Ehri, 2020).

Evidence on classroom phonics instruction for struggling readers in India is currently limited. In a study targeting low-income private schools in India, Dixon et al. (2011) utilised a phonics programme called Jolly Phonics including lesson plans for instruction. The experimental group showed significant improvement in reading and spelling skills and classroom instruction had a strong effect (d = 1.20) on students’ ability to blend three-letter sounds and pronounce words. In a recent study (2022), Shenoy and colleagues examined gains in English literacy skills of students exposed to phonics instruction in a private mid-income pre-primary school in India. The study included three cohorts of students–cohorts 0, 1, and 2 with mean age 5 years 7 months, 5 years 1 month, and 5 years 8 months respectively. Cohort 0 (n = 165) did not receive phonics instruction and was exposed to the traditional alphabet-spelling method; cohort 1 (n = 234) received phonics instruction for 1 year; and cohort 2 (n = 228) received phonics instruction for 2 years. Their results indicated that students receiving one to 2 years of phonics instruction made statistically significant gains in phoneme segmentation, first sound fluency, and nonsense word fluency (d = 1.19, p < 0.01; d = 1.42, p < 0.01). Their scores were one standard deviation higher than the students who did not receive phonics instruction. In fact, the effect size of the programme was two–three times larger than the effect size (d = 0.41) reported by the US National Reading Panel (Ehri et al., 2001). Results of these studies provide evidence on the effectiveness of phonics instruction as opposed to the alphabet-spelling based instruction for English as a second or third language learners in India.

A CARI approach: GraphoLearn

Another way forward for developing effective ways of supporting foundational English literacy skills in India is through a CARI. In this study, we utilised GraphoLearn Rime (GL), also known as GraphoGame, along with classroom instruction on phonics to build students’ English literacy skills. GL is a research-based, computer-assisted tool designed to support children struggling with basic decoding skills by providing explicit and systematic instruction on connections between graphemes and phonemes (Richardson & Lyytinen, 2014). GL is based on intra-syllabic rime units where single grapheme-phoneme correspondences are first blended into rimes and then CVC words. Later, CCVC and CVCC words are introduced. Through visual and auditory stimuli, the game provides repeated exposure and practice which reinforces sound recognition and connecting sounds to written units. Kyle et al. (2013) showed GL to be effective as a supplement to classroom phonics instruction for developing rime- and phoneme-level awareness in native English Grade 2 learners. Further, a review of 28 empirical GL studies indicates GL supporting reading subskills such as syllable reading and phonological awareness; most of these studies are conducted with beginning readers between kindergarten and Grade 2 (see McTigue et al., 2019 for review). This development in phonics learning has been found to be maximum when students are able to access more in-game levels (Ahmed et al., 2020; Richter et al., 2022). Studies conducted in UK and the US have indicated that GL is especially useful for students with better pre-literacy and word recognition skills as it improved their English decoding skills (Ahmed et al., 2020; Worth et al., 2018).

Evidence on GL’s efficacy for supporting struggling readers’ English literacy as an L2 in India have been shown in two studies. In the first randomised control trial (RCT) study, Grade 3 students (ages 7–8) exposed to GL made greater and faster development in the in-game assessment of letter-sound knowledge as compared to the control group even though the students’ baseline scores were very low (Patel et al., 2018). In another RCT study, the authors examined the efficacy of GL and the relationship between in-game progress and students’ literacy level pre-and-post intervention (Patel et al., 2022). Children (ages 5–7) exposed to GL showed faster development in all three in-game measures—letter sound knowledge, rime units, and word recognition. Also, students with more in-game progress showed more gains at post-test. In both the RCT studies, the control group was exposed to a math game, ‘Math Kids’, to ensure students in GL and control group are exposed to EdTech-based games and receive similar amount of classroom instruction time. In addition, in both the studies, GL was not integrated into classroom instruction, and students’ learning gains were limited to the game learning environment, i.e., students did not show gains in oral- and paper-based measures.

The gains in the Patel et al., (2018) and Patel et al., (2022) studies may have been limited to in-game context because of lack of alignment between classroom instruction and the CARI—an important component of effective interventions (see Rodriguez-Segura, 2022 for review). A rapid evaluation of 12 EdTech tools through pilots in India and a survey with 44 schools also reported that poor integration of EdTech and classroom curriculum may make it difficult for teachers to connect classroom and computer-assisted instruction (Sampson et al., 2019). Importance of complementarity between technology and teacher instruction was also underlined in a meta-analysis of GL studies (McTigue et al., 2019): the strongest effects (g = 0.48) were reported for interventions including with adult interaction with students. These findings highlight the imperative need to further examine the benefits of adult intervention, integrated classroom phonics instruction, and the added value of GL for supporting literacy acquisition. It is important to note that in GL studies (McTigue et al., 2019), adult interactions primarily motivated children for focusing on in-game instruction. So far, no study in India has combined GL with classroom phonics teaching for English literacy skills development in practice or research design.

The present study

The study is built on prior research on including classroom phonics instruction along with GL for a comprehensive reading instruction (McTigue et al., 2019; Patel et al., 2018, 2022). The novel focus is on developing English phonological awareness and reading skills in learners with low literacy skills through parallel lesson plans for phonics instruction. We aimed to examine whether classroom phonics instruction combined with GL can better support the development of foundational English literacy skills of Grade 2 struggling readers compared to the classroom phonics instruction alone. In addition, the study aimed to understand better the intervention by determining association between students’ attendance to phonics instruction and their developed literacy skills. Our research questions were:

  1. 1.

    How do children who receive GL along with classroom phonics instruction perform in foundational English literacy skills compared to students who only receive classroom phonics instruction?

  2. 2.

    How does students’ attendance to classroom phonics instruction lessons influence their gains in foundational English literacy skills?

Methods

Participants

In this study, 60 Grade 2 students, age 6–7 years, participated after parents’ written consent. All students attended the same classroom. We included Grade 2 students to ensure that they have received enough exposure to English language in school to benefit from GL. In English-medium schools, English language is taught Grade 1 onwards. Students came from low-income families and were native speakers of Hindi and three students also had additional languages (Table 1). Since Hindi was parents’ primary language of communication and they were not literate in English, the privacy and research notification were provided in Hindi language. Of the 80 parents invited to the study, 63 provided written informed consent. Eventually, 54 students (GL group = 28 and control group = 26) were included in the analysis as three students dropped out before the start of the intervention and six were excluded from the analysis because of data recording failure to the online game server. Consent was also received from the students’ two classroom teachers. Both were qualified to teach in primary classes.

Table 1 Group Characteristics
Full size table

All students were matched (Table 1) for age (t(52) = 0.45, p = 0.65) and gender and then randomly assigned to either the GL group which played GL or the control group which played a math game–Math Kids. According to the intervention plan, both groups received classroom phonics instruction for 30 min thrice a week from the two teachers who were provided training and continuous support during the intervention by the primary researcher. However, some of the students did not participate in each lesson or each play session because of their absence from the school. The mean playtime for the whole group was 10.7 h (SD = 1.88) and classroom instruction time was 9.95 h (SD = 1.78) (refer Table 1 for group differences in playtime). The playtime (t(52) = –1.84, p = 0.07) and phonics classroom instruction time (t(40.09) = –1.93, p = 0.06) did not statistically differ between the groups. Further, the effect size was 0.50 and 0.53 respectively).

Procedure

Prior to the start of the intervention, all participating students were individually pre-tested on measures of phonological awareness, oral vocabulary, and reading in English. In addition, teachers were provided two 2-h workshops on literacy development and phonics instruction to support their pedagogical skills and positively influence classroom instruction. To ensure coherence of content and consistency in classroom phonics instruction, teachers were given detailed lesson plans and audio clips of target sounds. All students received 26 lessons of phonics instruction thrice a week for 30 min a day, over a 7-week period. In addition, both groups played their respective games during school hours for 20 min, 5–6 days a week, over a period of 7 weeks, which yielded a playtime that is towards the higher durations of GL playtime that has varied from 57 to 900 min in previous studies (see McTigue et al., 2019 for review). Students played under their respective profile name on a mobile device equipped with headphones set up in a spare room. During each play session, the primary researcher and a research assistant were present (see Table 1 for the game playing times and instruction attendance times).

Recording of the implementation fidelity

Fidelity was tracked by keeping a manual log of students’ attendance in classroom phonics instruction and the playtime was recorded using a stopwatch. For the control group, the play days and play time was recorded manually. For the GL-group, the play days and time were also recorded in the in-game logs. Both groups were exposed to their respective games at the same time of the day to ensure that they received same play conditions and the primary researcher ensured students played under their own profile. The start and end time of phonics instruction were logged during classroom observations.

Classroom phonics instruction

Classroom phonics instruction content was aligned to the progression of sound units based on the sequence used in GL to ensure alignment between in-game and classroom instruction. The lesson plans included session objectives, core concepts, target sound units, resources needed, and a set of suggested activities to introduce sound units. To ensure that phonics instruction is relevant to students, the pre-planned lesson plans were modified when needed by the primary researcher during the intervention. Modifications were made after carefully observing instruction delivery and students’ progress in the classroom and based on feedback from teachers. The lesson plans were modified to include more examples of words with target letter sounds as teachers were unfamiliar with long and short vowels and sometimes confused letter sounds such as /I/ in fish and /aI/ in kite. Also, more pedagogical explanations of the aims of phonics instruction were added to the lessons to further aid teachers. As some words taken up as examples of target sounds were found to be unfamiliar to the students, teachers were encouraged to explain their meanings by making contextual links to students’ lived experiences. In addition, pictures of mouth positions for pronouncing the phonemes were included in the lessons so that teachers could show them during instruction. Before a lesson delivery, teachers practiced GL sound units using audio clips and verbalised target sound units while receiving primary researcher’s feedback. In the feedback sessions with teachers, any misconceptions such as associating graphemes < m > and < t > in English with sounds म—/mə/ and ट—/tə/ in Hindi or mixing of vowel sounds were resolved. Difficulties in correctly pronouncing phonemes /y/ and /t/ or in distinguishing between phonemes /a/, /e/, and /r/, /l/ were also addressed.

Each lesson plan consisted of 6–10 sounds and/or rime units and lasted for 30 min. While the lessons were scripted in English, the teachers delivered classroom instruction in students’ first language–Hindi. During the instruction, teachers demonstrated sound units using lip and tongue movement and by presenting the written graphemes on the blackboard. Teachers also played audio clips of the target sound units to ensure correct instruction and sounded out letters to form rime units and words. Post this demonstration, students repeated the target sounds along with the accompanying actions to ensure accurate verbal production of the sound units. To further support instruction on grapheme-phoneme correspondences (GPCs), familiar and common CVC words containing the target sounds were used to demonstrate word formation by sounding out letters. Both for the target sound demonstration and playing audio clips, the teachers used a microphone, making sure the sounds were audible to all the students in the classroom which often comprised of 60–75 students.

GraphoLearn Rime (GL)

GL trains players on the connections between spoken and written language. There are 25 play streams and 7 assessment streams in which the players are provided adaptive practice on letter sounds, rime units, and word recognition through various practice levels and assessments. Players are presented with the target sound with its corresponding written form and then expected to correctly match both. A correct choice takes them to the next set of options and incorrect choice provides them with immediate feedback after which they can correct their response. Players should be able to score a minimum of 80% on each level within a stream to move to the next stream. Introduction of letter sounds, rime units, and words progresses from the most orthographically consistent and frequent units to less consistent and frequent units. This sequence of teaching is based on the role of orthographic rime units in English reading development (Kyle et al., 2013). The game provides practice in rime units across 11 streams and word recognition across 15 streams, supporting phonological awareness and spelling skills (Patel et al., 2018, 2022).

Math game

Math Kids, a math game, was provided to the control group to control for the Hawthorne effect. Students practiced basic mathematical operations–addition, subtraction, and multiplication. To ensure that the control group did not receive any visual or auditory input in English, the in-game content language was set to Hindi. The content is given in the increasing order of difficulty, for instance in addition, beginning from number counting using objects, to puzzles on addition and addition tasks with numbers. Like GL, Math Kids provides visual stimuli in case of an incorrect response.

Measures

Students’ foundational literacy skills were assessed at the pre- and post-test using in-game and oral-and-paper based measures.

In-game measures

GL constitutes three in-game skill assessment tasks. In the letter-sound knowledge task, children played 24 trials of letter sounds. In each trial, they picked the correct written letter out of six to seven on-screen options that matched with the auditorily presented letter sound. In the rime-unit recognition task, children played 24 trials where they matched the target sound unit such as /en/ with the correct two-to-four letter string from a set of options such as /im/, /em/, /in/, /en/. Finally, in the word recognition task, children played 47 trials wherein target auditory input such as /from/ must be matched to the correct on-screen options such as < from > , < worm > , < form > , < fron > . Unlike the letter-sound knowledge task where children played 24 trials irrespective of the number of correct responses, the rime-unit and word-recognition tasks discontinued if more than 50% of responses were incorrect.

Oral- and paper-based measures

The oral measures included four tasks: initial phoneme identification, last phoneme identification, phoneme blending, and phoneme segmentation. The paper-based measures included eight tasks: vocabulary, letter names, letter sounds, Dynamic Indicators of Basic Early Literacy Skills (DIBELS) word reading, GL-word reading, similar-to-GL word reading, non-word reading, and oral reading fluency. Their descriptions, scoring, and reliability estimates of the tasks are given in Table 2. For each of the tasks, instructions were given in Hindi and English to ensure students’ understanding of the task.

Table 2 Oral and Paper-based Measures
Full size table

PPVT5 (Peabody Picture Vocabulary Test 5th Edition) was used to assess oral vocabulary at pre-test. PPVT-5 is a norm-referenced measure for English language that measures receptive vocabulary knowledge (Dunn, 2019). Of the 240 PPVT items, the 76 items in Form A were shown to the students. The letter name and letter sound measures were adapted from DIBELS 8th Edition letter naming fluency task for Grade 1 (University of Oregon, 2021). Words used in initial phoneme and last phoneme identification tasks were also taken from DIBELS battery for Grade 1 as it aligned with the English vocabulary contained in the Grade 1–2 level textbooks used in the intervention school. In phoneme blending and phoneme segmentation tasks, grade-appropriate CVC words were utilised.

Three word-reading and one non-word reading tasks consisting of 20 test items each were used. The DIBELS-word reading task was adapted from DIBELS battery for Grade K since DIBELS Grade 1 battery was too advanced for children in this study’s context. GL-word reading task constituted of in-game words. Similar-to-GL-word reading task words were phonologically similar to the in-game words to test for learning transfer from trained to untrained items. Words in the non-word reading task were taken from DIBELS Grade 1 battery for nonsense word fluency.

The oral reading fluency task (a 51-word long story) was selected from ASER which includes a four-part test to assess children’s basic reading skills in India. This was a timed task wherein students read aloud the text in 1 min. The task is criterion referenced and its inter-rater reliability is reported to be 0.64 (Vagh, 2012).

Data analysis

The data was analysed using SPSS version 28.0. Prior to examining the treatment effects, baseline equivalence on group characteristics, extent of exposure to the treatment (GL or Math Game), and pre-test scores were checked using independent samples t-test (see Table 1). In the preliminary analysis, distributions of the in-game and oral- and-paper-based measures were also examined. The in-game letter-sound knowledge, rime-unit recognition and word recognition resembled a normal distribution at pre- and post-test whereas only few oral- and paper-based measures had non-normal distribution of scores at pre- and post-test. Extreme outliers were found in the letter-name knowledge (6 cases), letter-sound knowledge (1 case), and phoneme segmentation (9 cases), and, in addition the non-word reading scores did not approximate normal distribution at pre-test. At post-test extreme outliers were found in letter names (6 cases) and oral reading fluency (1 case). After winzorizing the outliers, letter-name knowledge pre-test and oral reading fluency post-test approximated normal distribution.

Due to the slight non-normality of some of the distributions and small sample size both parametric (independent sample t-test) and non-parametric (Mann–Whitney U test) tests were conducted for the pre- and post-test group comparisons. The results showed that only for the letter-sound knowledge pre-test measure the results for parametric and non-parametric test differed and therefore the Mann–Whitney U test results were reported. Otherwise, the parametric t-test results were chosen. Effect sizes are reported for the parametric tests using Cohen’s d criteria: d ≤ 0.2 = small effect; d ≥ 0.5 = medium effect; and d ≥ 0.8 = large effect (Cohen’s, 1988).

Next, repeated measures ANOVA was used to compare the effects of time (development between pre-test and post-test level), group (effects of being in the GL or control group), and group*time interaction (group differences in change over time) on the two groups’ development between the pre-test and post-test. To examine how students’ attendance to classroom phonics instruction predicted gains in phonological awareness and reading skills, hierarchical regression analysis was run.

Results

The first research question examined whether students receiving both GL and classroom phonics instruction (GL group) performed better than students receiving only GraphoLearn-aligned phonics instruction (control group). The group comparisons for the in-game measures (Table 3) indicated no significant group differences at pre-test (d = 0.17 − 0.31). At post-test, significant group differences were found in favour of the GL-group in the in-game measures of letter-sound knowledge, rime-unit recognition and word recognition with large effect sizes (d = 0.87 − 1.01).

Table 3 Descriptive Sstatistics and Group Comparisons on the In-Game Tasks
Full size table

In the oral- and paper-based measures (Table 4) there were no significant group differences at pre-test (small effect sizes d = 0.04 − 0.29). There were also no significant group differences at post-test and effect sizes were again small (d = 0.02–0.36). Surprisingly, the GL group did not perform significantly better even in the task involving trained items such as the GL-word reading task.

Table 4 Descriptive Statistics and Group Comparisons on Oral- and Paper-based Tasks
Full size table

Next, to examine development, effects of time (change from pre- to post-test), group (GL versus control), and time*group interaction on the scores (group differences in change over time) were compared using repeated measures ANOVA. In the in-game measures, significant effects of time, group, and group*time interaction were found (see Table 3). The interaction indicated differential improvement in the two groups (See Fig. 1). The GL group had faster development in the in-game tasks than the control group. In the oral-and-paper based measures, however, only the main effect of time was significant (see Table 4) which indicated that improvement between pre- and post-test was similar in both the groups.

Fig. 1
figure 1

Group Comparisons of Development From Pre-Test to Post-Test on the In-Game Tasks

Full size image

Association between attendance to the classroom phonics instruction and students’ performance

Finally, we examined whether variation in attendance to classroom phonics instruction was associated with students’ learning outcomes. Students’ attendance ranged from 10–25 lessons out of the maximum 26 lessons, with an average of 19.91 lessons over the period of the intervention. To estimate the potential role of attendance to phonics instruction in predicting learning gains, Pearson’s correlations between attendance and the post-test scores were calculated. These correlations were positive. All correlations were non-significant except for those between attendance and oral- and paper-based measures of letter-name knowledge (r = 0.33, p < 0.01), letter-sound knowledge (r = 0.41, p < . 001), and phoneme blending (r = 0.35, p < . 01). The correlated variables were further examined using hierarchical regression analysis involving prediction of the three post-test measure scores with group and vocabulary (step one), auto-regressor (the pre-test score of the predicted measure) (step two), and the attendance measure (step three).

Attendance to classroom phonics instruction did not predict gains in letter-name knowledge (see Table 5). The step one explained 12% of the variance in the letter-name knowledge scores (F(2, 51) = 3.39, p < 0.05) and vocabulary was a significant predictor, while group membership was not. In the second step, the pre-test score of letter-name knowledge explained an additional 78% of the variance (F(3, 50) = 139.85, p < 0.001). In the third step, the attendance to phonics instruction added 1% of explanation which was insignificant (β = 0.08, t = 1.69, p = 0.10). The final model explained 91% of the variance (F(4, 49) = 109.47, p < 0.001). Figure 2 shows that since students were at ceiling at pre-test, there was limited room for further development in letter-name knowledge. However, the few lowest performing students seemed to develop during the intervention as variability was reduced in the post-test.

Table 5 Regression Analysis Summary on Attendance to the Phonics Instruction Predicting Letter Names Knowledge
Full size table
Fig. 2
figure 2

Development From Pre-Test to Post-Test in Oral- and Paper-Based Task for Letter-Name Knowledge for the Whole Group Note: LN1 – Letter-Name Knowledge pre-test; LN2 – Letter-Name Knowledge post-test

Full size image

In the letter-sound knowledge task, attendance to phonics instruction uniquely and significantly predicted gains at post-test (see Table 6). Step one with group and vocabulary explained 15% of variance (F(2, 51) = 4.32, p < 0.05). While the effect of the group was insignificant, students’ vocabulary predicted the letter-sound knowledge at post-test. The pre-test score (auto-regressor) was added in step two also had a significant effect and explained the variance by an additional 7% (F(3, 50) = 4.52, p < 0.01). In step three, attendance to the phonics instruction explained 11% of variance (β = 0.37, t = 2.87, p < 0.01). The final model explained 33% of the variance (F(4, 49) = 5.94, p < 0.001). Figure 3 describes variation in students’ letter-sound knowledge at the pre- and post-test. Despite gains in the means, the variation in letter-sound knowledge increased, suggesting that while some students developed, not all students benefitted similarly from the instruction.

Table 6 Regression Analysis Summary on Attendance to the Phonics Instruction Predicting Letter Sounds Knowledge
Full size table
Fig. 3
figure 3

Development From Pre-Test to Post-Test in Oral- and Paper-Based Task for Letter-Sound Knowledge for the Whole Group Note: LS1 – Letter Sound pre-test; LS2 – Letter Sound post-test

Full size image

In the phoneme blending measure, attendance to phonics instruction uniquely and significantly predicted development in students’ phoneme blending skill (see Table 7). In step one group and vocabulary explained 18% of the variance in phoneme blending at post-test; however, only vocabulary had significant effects (F(2, 51) = 5.58, p < 0.01). The phoneme blending pre-test score (auto-regressor) added in step two also had a significant effect and explained the variance by an additional 30% (F(3, 50) = 15.05, p < 0.001). In step three, attendance to phonics instruction added in step three explained 5% of variance (β = 0.23, t = 2.22, p < 0.05). The final model explained 53% of the variance (F(4, 49) = 13.41, p < 0.001). Figure 4 shows highly left-skewed distribution at pre-test. However, at post-test, the average and distribution of scores improved, indicating that students developed phoneme blending skills during the intervention. The spread of distribution though indicates that all students did not benefit similarly from the instruction.

Table 7 Regression Analysis Summary on Attendance to the Phonics Instruction Predicting Phoneme Blending Skills
Full size table
Fig. 4
figure 4

Development From Pre-Test to Post-Test in Oral- and Paper-Based Task for Phoneme Blending for the Whole Group Note: PhBlen1 – Phoneme Blending pre-test; PhBlen2 – Phoneme Blending post-test

Full size image

Discussion

In this study, we provided two groups of students with classroom phonics instruction with either GraphoGame (GL) supporting grapheme-phoneme correspondence and word-recognition skills or a control math game to examine the effects on children’s English literacy acquisition in India. Before the intervention, while all students had received English-medium instruction for over 1 year in Grade 1 and partly in Grade 2, most of the students were found to be at floor at the first assessment in all the measures except for letter naming. After the 7-week intervention, both groups’ learning gains at least doubled in all the measures, showing improvement in their literacy skills. Both groups improved in the skills which were explicitly taught such as letter names, letter sounds and phoneme blending during classroom instruction and via GL. Groups also improved in the skills which were not explicitly taught such as phoneme segmentation, initial and last phoneme identification, and word reading skills. In the oral- and paper-based measures, both groups showed significant development in phonological awareness and word reading skills in English over time whereas for the in-game measures the GL group's gains were significantly better than those of the control group exposed to Math game. The rapid development in the English literacy skills of non-native English speakers in this intervention indicates the inherent potential of the combined teacher-led phonics instruction and computer-based instruction for supporting learners, thereby addressing the literacy learning crisis in contexts like that of the present study.

Improvement in the in-game measures—letter-sound knowledge, rime-unit recognition, and word recognition—aligns with previous GL-based studies in India (Patel et al., 2018, 2022) which documented that the skills developed from playing GL benefited the GL group in attaining in-game literacy skills more than the control group. This finding also aligns with a meta-analysis indicating effectiveness of computer-assisted reading interventions (CARI) for even children at-risk of reading difficulties (Verhoeven et al., 2022); the study found highest effect size (d = 1.66 − 2.55) of CARIs on letter knowledge, phonological decoding, and spelling accuracy. However, the gains in our study were not significantly different between the GL and math control groups in the oral- and paper-based tasks, suggesting absence of GL’s added effect on skills assessed outside the game. Absence of GL’s added value aligns with prior studies showing lack of transfer in both native and non-native speakers learning English (McTigue et al., 2019; Worth et al., 2018). Given that we used GL-aligned lesson plans in classroom instruction and the exposure to phonics was increased via GL in the GL group, we expected the group would show significantly better gains in oral- and paper-based tasks than the control group (McTigue et al., 2019; Patel et al., 2018, 2022). However, group differences were absent even in the trained items in letter-sound and GL-word reading measures, indicating absence of near transfer outcomes. There is a need for future studies that compare gains against a business-as-usual group to further examine transfer effects.

One potential reason for absence of near transfer outcomes could be limited consolidation of the GL group’s skills during their GL playtime (mean = 11.17 h) and limited progress to more demanding levels in the game. Unless students’ progress in the game and access more demanding levels, more time spent on the game may not yield better learning gains (McTigue et al., 2019). It is possible that further progress in the game could achieve skills transfer, but we cannot know this, unfortunately, based on this study. At the end of our intervention, only 57% of the children in the GL group crossed stream 8 from where the instruction and practice in rime units and word formation become explicit. A reason for the limited progress in this sample could be students’ less-than-optimum pre-literacy skills as it has been shown to support students to move forward in the game (Patel et al., 2022; Richter et al., 2022; Worth et al., 2018). During the game, a positive cycle may emerge where with more in-game progress, students develop more reading subskills (Ahmed et al., 2020), and then the developed skills may also further support in-game progress. To examine the role of game progress in this sample, we calculated correlations between in-game progress and reading subskills in the in-game and oral- and paper-based measures for the GL group. The correlation coefficients were medium to large (r = 0.59 − 0.80; p < 0.001). These associations could be examined in future studies to better understand how progress and exposure time to CARIs supports learning gains in oral- and paper-based environments. It is possible that students’ in-game progress and consolidation of skills could improve with a longer intervention; but more exposure time to GL may not necessarily support consolidation.

In-game progress, consolidation of skills in GL, and skills transfer across learning environments may be supported by aligning language and content of GL and classroom instruction. While we aligned the in-game and classroom curriculum for phonics instruction, the language of instruction was different in both the mediums. Teachers utilised students’ first language, Hindi, during classroom phonics instruction, but the in-game instruction was only in English. As using children’s first language (L1) can advance effectiveness of the instruction and learning gains, the instruction in GL could further strengthen students’ in-game skills by utilising Hindi as its language of instruction. Integrating L1 in the in-game instruction could further support English literacy gains as students’ L1 has been shown to predict L2 acquisition (Patel et al., 2022).

The effect of attendance to phonics instruction on learning

The amount of attendance to classroom phonics instruction was identified as a significant predictor of the development in letter-sound knowledge and phoneme blending measures. Even though students’ oral vocabulary and pre-test skill level, initial skills level, and group membership (GL vs control) were controlled, students’ attendance to classroom phonics instruction had a significant and unique effect. This indicates that students’ English literacy skills improved faster with greater attendance to phonics instruction lessons, thus suggesting that with more classroom phonics instruction time, students may show better letter-sound knowledge and phoneme blending skills. This also aligns with a meta-analysis of 35 studies showing highest effect size (g = 0.69) when phonics instruction was greater than 2001 min (Odo, 2021). Shenoy et al. (2022) provided 30 min of phonics instruction 5 days a week for one and 2 years to two groups and found a difference (d = 0.36) on the English literacy skills of students who received 1 year and 2 years of phonics instruction. In our study, the whole group attended ~ 735 min of phonics instruction lessons over ~ 20 days and had variation in attendance. Since classroom phonics instruction in our study focused mostly on letter-sound knowledge, students’ gain score in the measure underlines the effectiveness of teacher instruction despite minimal training. Also, explicit instruction on letter sounds and letter names likely helped students develop an understanding of the connections between print and speech. Teacher provided explicit phonics instruction including an explanation of the concept in students’ L1; modelling of skills, especially letter names, letter sounds and rime unit recognition, phoneme blending; and feedback (Piasta & Hudson, 2022). In addition, teachers were observed to leverage students’ existing literacy skills in Hindi by drawing upon students’ vocabulary and letter sound knowledge in Hindi. Making letter-sound associations is a skill necessary to understand the alphabetic code for decoding and spelling familiar and unfamiliar words (Ehri, 2020)—a skill that the alphabet-spelling method has not been found to develop (Shenoy et al., 2022).

Needless to say, explicit phonics instruction on letter sounds and blending along with segmentation and phoneme identification could thus further improve students’ literacy skills (Odo, 2021; Shenoy et al., 2022). Knowledge of grapheme-phoneme correspondences and phonological skills supports the ability to decode, spell, and read words, thus aiding reading fluency and literacy skills (Castles et al., 2018; Ehri, 2020). The improvement observed in this study underlines the potential of phonics even in a multilingual context such as India where students are learning English as a second or third language in schools. Future studies should also consider including a business-as-usual group to further explore the relationship between classroom phonics instruction and literacy skills development.

Our findings also suggest that attending classroom phonics instruction can be beneficial for struggling readers. Students had low scores at pre-test, and they benefited from the instruction despite the short intervention time. Exposure to adequate phonics instruction can benefit students needing intensive support with English literacy development—a finding that Shenoy et al. (2022) showed in their study where the percentage of students (N = 627) needing intensive support dropped from 64.24% to 13.60% after 2 years of phonics instruction. Providing classroom phonics instruction to struggling readers can thus prevent gaps in reading fluency that may otherwise cumulate through the elementary grades. Early literacy intervention utilising a phonics-based approach can thus curb staggeringly low reading levels. For instance, at Grade 5 and 8, only 24.5% and 46.7% of students could read simple English sentences—I like to read.; She has many books. (ASER Centre, 2022). Future studies can further explore relationship between effectiveness of phonics instruction and individual differences in learning levels.

In summary, our results suggest that both groups’ phonological and reading skills developed rapidly over a short period of time. While GL supported learners’ literacy skills in the game, the added value of these skills was absent in the oral- and paper-based medium. Classroom phonics instruction has shown potential for phonological awareness and reading skills development in learners with low-reading levels in India. Future studies could include GL-only and business-as-usual groups to better examine the effectiveness and transfer of GL and classroom instruction. Future studies should also determine to whom and in which conditions phonics instruction and GL are most useful. These results can be drawn further to indicate that the efficacy of EdTech must be examined not only from its potential to provide personalised instruction, but also based on its ability to support learners in utilising their in-game skills in the real world as well. Finally, this study contributes to research on GL efficacy by indicating the potential of aligning classroom instruction with GL for literacy development.

Limitations

Limitations of this study must be considered while evaluating its efficacy. The results and their interpretation are limited by small sample size which lowered the statistical power of the findings. Further, skills transfer from in-game to the real world and efficacy of GL remained undetermined without GL-only and business-as-usual groups; thus, future studies can also include GL-only and business-as-usual groups. Also, since the in-game instructional content was not in students’ first language, it may have been a barrier for them in using GL meaningfully as it could have affected their in-game progress and engagement (Sampson et al., 2019). Furthermore, large class size (80 students) could have limited the effectiveness of phonics instruction; the instruction was given to the whole class as the teacher-student ratio needed for small-group instruction was not feasible. Next, while the researcher attending all classroom phonics instruction sessions ensured fidelity, it may not be possible when implemented at scale. To make the intervention more ecologically valid, the future studies can consider providing teachers lesson plans without continuous mentoring and classroom observation. Finally, to make our findings more generalisable, similar interventions must be done in different demographics.

Practical implications

In the light of our findings, we want to highlight the importance of classroom phonics instruction since it emerged as a significant predictor of students’ phonological awareness. Classroom phonics instruction benefited students’ phonological awareness and reading skills. Ofcourse, the relevance of the CARI and classroom instruction for struggling and at grade-level readers warrants further examination. Based on our experience in this study, potentially useful integrative approaches could support teachers’ understanding of grapheme-phoneme correspondences with GL and support classroom phonics instruction with lesson plans. We also want to note the minimal training and mentoring given in this study to support teachers’ literacy instruction skills which is an encouraging finding. Well-trained and supported teachers can provide better literacy instruction and strengthen decoding and word reading outcomes (see Hudson et al., 2021 for review).

We encourage uptake of a research-based integrated approach, especially for CARI to allow teachers and students opportunities to make connections between in-game and classroom content, especially to verbalise in-game learning and support skills transfer. One way to integrate CARIs is through evidence-backed lesson plans with curriculum alignment to optimise benefits of a CARI and teacher-led classroom instruction. GL and similar CARIs can provide adaptive learning and additional practice to students; teachers could scaffold learning better by utilising students’ context and existing literacy skills. CARI integration could be further strengthened by utilizing students’ L1 as a medium of instruction. Future versions of in-game instruction in GL could be provided in students’ L1 to support in-game learning.

Besides classroom phonics instruction, determining the conditions under which CARIs are most supportive is important especially with the global push for foundational literacy development and over 10,000 EdTech products in the market (Goyal et al., 2022). The absence of GL’s added value in the GL-group’s oral- and paper-based scores highlights an important concern in both the contexts for India—one, where EdTech is viewed as a learning tool in classrooms with limited or no teacher capacity and students’ low attendance; and two, where EdTech is used to improve the effectiveness of classroom instruction. In either case, benefiting from a CARI likely requires students to meet certain conditions—adequate pre-literacy skills, access to demanding levels in a CARI, and consolidation of skills. Otherwise, improvement in the in-game skills may not necessarily advance literacy skills in the real world. Further, the added value of GL and similar CARIs might differ for students given the variation in their scores.

Future considerations

The findings of our study have implications for further research in utilising classroom phonics instruction together with a CARI such as GL. Firstly, as this study was a pilot, the findings could be further examined by utilising our study design with a larger sample to increase the statistical power. Secondly, as all students in this study did not participate in all phonics instruction lessons and play sessions, it is essential to motivate students to attend school more regularly. Further, future studies must ensure an equivalent amount of playtime and classroom phonics instruction between both the groups. More accurate tracking of playtime between both the groups is also needed. In the current study, while the differences in the exposure to the treatment were not statistically significant, there was a trend towards more playtime and classroom phonics instruction in the GL group.

Third, the next phase of the research can include GL-only and business-as-usual groups to further look into the value of combining GL and phonics instruction in classroom, especially with respect to transfer of skills from an in-game to oral- and paper-based environment—an important factor for determining whether a CARI like GL can support development of literacy skills in the real world. Fourth, existing literacy skills, especially in students’ L1, could be worth examining to understand their importance in literacy development in L2, English; this would be relevant while utilising L1 in a CARI and classroom phonics instruction. Fifth, variation observed in the learning gains in letter sounds and phoneme blending indicates the need to further examine which students are benefiting most from the intervention. Thus, future studies could examine effectiveness of exposure to only GL, only phonics classroom instruction, or both to indicate value of the interventions for struggling, grade-level, and high-performing readers as adoption of CARIs at scale, while cost-effective, may not support all learners alike.

Given the growing number of EdTech tools for foundational literacy skills, examining the effectiveness of GL vis-à-vis other CARIs could help determine GL’s merit as a tool for developing phonological skills compared to similar technology-based literacy development tools. This comparative approach may be useful in determining essential features of an effective CARI, and thus for selecting a CARI more beneficial than the other. Such a study could be a next step after determining the potential benefits of a CARI compared to receiving combined instruction from classroom phonics instruction and CARIs. Finally, we would like to conclude that as adopting EdTech for supporting literacy development seems to become more and more common, we must examine and determine effective ways of including CARIs benefiting struggling readers. One potential approach for improving English literacy skills is combining a CARI and classroom phonics instruction.