Educational Technology Research and Development

, Volume 62, Issue 4, pp 417–436

The effect of using a mobile literacy game to improve literacy levels of grade one students in Zambian schools

  • Jacqueline Jere-Folotiya
  • Tamara Chansa-Kabali
  • Jonathan C. Munachaka
  • Francis Sampa
  • Christopher Yalukanda
  • Jari Westerholm
  • Ulla Richardson
  • Robert Serpell
  • Heikki Lyytinen
Open Access
Research Article

Abstract

This intervention study was conducted to document conditions under which a computer based literacy game (GraphoGame™) could enhance literacy skills of first grade students in an African city. The participants were first grade students from Government schools (N = 573). These students were randomly sampled into control (N = 314) and various intervention groups (N = 259). GraphoGame™ was administered on cellphones to students at their schools under supervision. Each student in the study was assessed using a battery of locally developed cognitive tests that measured emergent literacy skills (Orthography test), decoding competence (Spelling test), vocabulary (Picture Vocabulary Test—PVT) and arithmetic (Zambia Achievement Test—ZAT). There was a positive effect of the game for the Spelling test—which closely targeted the skill GraphoGame™ is designed to promote. The most effective intervention combined exposure of both the teachers and the students to the game. Initial letter knowledge was a good predictor of final letter knowledge on GraphoGame™.

Keywords

Students Graphogame Intervention Literacy skills 

Introduction

Literacy is a major cultural resource for effective communication and an essential skill for individuals to prosper in a modern society. Accordingly, Zambia’s national government has invested heavily in providing universal access to basic education. Yet the proportion of school students achieving even the minimal expected standard of literacy by Grades 5 and 6 is exceptionally low, with only 32.3 % of students attaining minimal acceptable mastery of skills and knowledge (Ministry of Education 2010).

In an attempt to improve reading levels, the Government of Zambia introduced a new educational policy (Ministry of Education 1996). This led to the introduction of the Primary Reading Programme (PRP). The policy emphasized that students should be taught to read and write in a familiar language. This meant that the seven Zambian official languages were to be used for teaching initial literacy, depending on the geographical location.

The PRP is a reading programme that consists of three components, the New Breakthrough to Literacy (NBTL), which is implemented in grade 1; Step into English (SITE), which is introduced in grade 2; and the Read on Course (ROC), which is implemented from grades 3 to grade 7. The PRP works on the premise that the orthography (reading/spelling system) of the Bantu languages of Zambia is transparent or consistent in nature. Transparent alphabetic codes are much easier to teach and learn (McGuinness 2005). Furthermore, the local languages are the principal language of play for most students. When students begin school in grade 1 they are already familiar with the spoken form of the language in which they have to learn to read and write. In some instances though, the regional official language may not coincide with the dialect used by the family at home (Sampa 2005).

Reading improvements have been documented since the introduction of the PRP. For example, an increase from 23 % to over 60 % has been reported in achieving minimal standards in the local languages in Grade 5. This improvement has been regarded as one of the major signs of success of the PRP in grade one (Kanyika 2002). Sampa et al. (2003) further reported that students in PRP pilot schools performed better in both English and Zambian languages than students from ordinary schools. However, subsequent evaluations of PRP schools report that these gains in performance were not sustained (Ministry of Education 2012, 2010; Musonda and Kaba 2011).

Students in Zambia’s public schools are taught initial literacy in any one of seven languages. These Zambian languages are part of the Bantu languages. They have spelling systems that are transparent in nature (Banda et al. 2008). This means that they have consistent phoneme-grapheme correspondences. This makes it easier for students to learn to read and write (Kashoki 1990; Aro 2006). Seymour et al. (2003) refer to such languages as having “shallow orthographies” (p. 144). They conclude that shallow orthographies make it easier to support the process of learning to read and write because of the single alphabetic nature of these languages. Empirical results of the research conducted in the European languages shows the result of transparency. Finnish students whose orthography is fully consistent at the phoneme-grapheme level, learn to read faster than most students from other European countries whose orthographies are deeper. They excel in literacy acquisition rates especially in comparison to English speaking students, whose writing system is the most nontransparent (Aro 2006; Aro and Wimmer 2003). English students lag behind their peers who are learning to read in more consistent spelling systems by almost 2 years as revealed by Seymour et al. (2003).

In the English language practically none of the letters are sounded the same way across all the contexts in which the letter occurs. This makes it difficult for students to master reading and writing skills because the number of connections between spoken and written is much larger (less than 30 in transparent spelling systems compared to more than one thousand in English). The challenge associated with segmenting the speech sound to units, with a relatively inconsistent connection to writing, is demanding. This is in comparison to transparent spelling systems, where letter/phoneme units can be easily segmented. The low learning burden needed for acquiring initial literacy in the Zambian languages which, like the Finnish language have a transparent spelling system, should have a positive impact on the reading acquisition levels of Zambian students.

In this paper we report on the findings of an applied research project entitled Reading Support for Zambian Children (RESUZ) conducted in 2011 with a representative sample of 573 grade 1 students at public schools in the multilingual capital city of Lusaka. The project is a collaboration between the University of Zambia and University of Jyväskylä (Finland). Its principal objective was to assess the efficiency and effectiveness of a supplementary, computer-mediated learning resource in the form of a phonics game (GraphoGame™) played on a hand-held cell-phone and factors associated with its efficiency.

The game was designed in Finland and has since been used to help students learn to read in Finnish and also other languages (see www.grapholearn.info). The effectiveness of GraphoGame™ in helping students to read has been documented (e.g. Lyytinenet al. 2009; Saine et al. 2010; Kyle et al. 2013). Research on the effectiveness of the game in other languages is being conducted in various parts of the world. Computer-assisted instruction (CAI) is a relatively new practice in most African countries, where the majority of the population only access modern Information and Communication Technology (ICT) through their use of cell-phones for personal communication. It is therefore important that ecologically valid intervention using ICT in the classroom be considered, such as teacher variables that relate to how well the intervention can be effectively conducted. If the potential cost-saving benefits of ICT are to be realized in the region, the efficacy of CAI must be empirically verified in African sociocultural and economic contexts.

We examined whether playing the game makes a significant impact on initial literacy acquisition of students enrolled in Lusaka public schools, within the wider contextual constraints of their classroom environments, curriculum, teaching practices and family backgrounds. Earlier observations reveal that an important factor affecting reading acquisition is the widespread social prestige attached in Zambian society to literacy in the English language. As it is the language through which knowledge guiding reading instruction is most widely available, there seems to be an interference of English letter name knowledge, despite the independent and explicit goal of teaching reading in the local languages. For example,/ei/,/i:/,/ai/, (in English) instead of/a :/,/e:/,/i:/etc.) (in the local language) to the extent that students have problems replacing English letter names with letter sounds in the local language (Ojanen et al. 2013). In this regard, the letter-sound knowledge of Zambian teachers is the subject of an on-going, related study.

The main focus of this paper is to document the effectiveness of GraphoGame™ in the Zambian context. In the subsequent paragraphs we present the objectives, design and methods of the RESUZ project. We describe GraphoGame™, the several ways in which it was administered to selected students, and how its impact on individual literacy acquisition was determined. The results are analyzed in terms of the degree to which they support or disconfirm several hypotheses. Subsequently a broader discussion regarding the significance of the project’s findings and their implications for the way forward in operational planning to improve the effectiveness of Zambia’s public education for imparting basic literacy skills is presented.

The more specific goals of this study were as follows:
  1. 1.

    To examine the effectiveness of the desktop computer or cell phone based phonics game (GraphoGame™) in improving initial literacy skills of first grade students in the context of prevailing conditions, as a supplementary resource for literacy instruction in Zambian public schools.

     
  2. 2.

    To establish which of several methods of providing intervention with (GraphoGame™) (students only, teachers only or a combination of both) is most effective.

     
  3. 3.

    To investigate the influence of initial letter knowledge and amount of time playing GraphoGame™ (play time) on final letter knowledge.

     

Methodology

Participants

Participants consisted of 573 students of whom 52.4 % were females with age range 5–9 years old, and their teachers, N = 68. Due to attrition the total number of participants available for the post test was 312 of whom 52.5 % were females. Table 1 shows the cross tabulations of the age and gender of the students. Information on age was missing for 4 female participants.
Table 1

Age and gender distribution for the intervention groups and the control group

 

Gender

      
 

C

LG

LG + TG

LG + TIG

TIG

TG

Total

Female

163

33

24

31

41

8

300

Male

151

30

23

38

26

5

273

Total

314

63

47

69

67

13

573

Age

       

 5

0

0

0

1

0

0

1

 6

25

1

4

8

3

3

44

 7

146

28

31

33

30

8

276

 8

122

30

11

26

31

2

222

 9

18

4

0

1

3

0

26

Sampling procedure

The participants were randomly selected from 42 Government schools in Lusaka District. The schools were selected randomly from the total number of 102 Government schools in the District, after excluding the few schools serving outlying, quasi-rural neighborhoods. Within each school two grade one classes were randomly selected and randomly assigned to either a control or intervention class. Six students, 3 males and 3 females were then randomly sampled from each classroom using class registers. The cognitive and achievement tests were administered to all the students at baseline.

Tests

Each student was assessed using a battery of tests of emergent literacy skills, competence in spelling, vocabulary and arithmetic. All tests were administered in ciNyanja, which is the lingua franca of Lusaka and the official medium of initial literacy instruction in all Government schools. During baseline testing, each student was asked a set of 7 questions relating to his/her home language background. Responses to each of these items was scored +1 if the language cited was one of the varieties of ciNyanja and 0 if another language was cited. The total of these scores, ranging from 0 to 7, was treated as an index of familiarity with ciNyanja.

Orthography test

This test assesses orthographic awareness using the ideas developed, implemented and illustrated by Ojanen et al. (2013). It comprises 108 items arranged in order of increasing difficulty. It has three sections; the first one presents visual symbols half of which are letters and the other half non-letters; the second section comprises comparably chosen larger units: legitimate and illegitimate ciNyanja syllables or their combinations. The third section tests discrimination between orthographically impermissible or permissible sequences of 3–4 letters, simple words versus non-words, and more complex six-letter words versus non-words. Students were required to underline the items that corresponded with the conventional alphabet letters, syllables and words. The test had several practice items for each section. After the practice items, students were given 3 min to complete the actual test. One mark was given for every correct response and subtracted for every incorrect response. The test consisted of 18 rows. Each row contained 6 items, 3 correct and 3 incorrect. Therefore the range of possible scores on this test was between −108 and 108. This test was piloted before the actual study commenced. The test had a test–retest reliability of r = .50 (N = 44).

Spelling test

This test was developed by Ojanen et al. (2013). It is a measure of early grade students’ competence in spelling ciNyanja. It consists of 20 dictated items. Phonemes, syllables and words are presented in this order and are arranged in order of increasing difficulty. The student was required to choose from four possible answers and underline the correct response. A score of 1 was given for every correct response and 0 for every incorrect response. The range of possible scores on this test was 0–20. This test had a test–retest reliability of r = .82 (N = 43).

In order to provide a contextual profile of the skills of Zambian first graders, two additional tests were introduced to assess developmental characteristics that are relevant to student’s educational progress, but were not expected to be directly affected by the intervention. Math assessment was included for evaluating the possible Hawthorne effect resulting from getting more attention when participating in the interventions.

Vocabulary

The Picture Vocabulary test (PVT) was originally developed in the USA by Dunn and Dunn (1997) as a measure of receptive language competence or verbal intelligence. It has since been adapted and used widely around the world to assess children ranging in age from 5 to 15. In this test, the child is presented with a series of spoken words in order of increasing difficulty with a visual display of four pictures on each trial. The task for the child is to choose the picture which corresponds with the spoken word. Each correct response receives a score of 1. The range of possible scores for this test is 0–30. In Zambia, adaptation and translation of the instrument into the four most widely spoken indigenous languages (ciNyanja, iciBemba, siLozi and ciTonga) was conducted by the Zambian Early Childhood Development Project, which was a collaboration between the University of Zambia and Harvard University (Fink et al. 2012).

After pre-testing, adaptation and translation, the project administered the Zambian versions to a diverse sample of 1,686 rural and urban 5-year-olds not yet enrolled in school and found a wide range of raw scores, with a value of α = .83 on the Cronbach index of internal consistency.

Mathematics

The Zambia Achievement Test—Mathematics (ZAT-M)—was developed by researchers from the University of Zambia and Yale University (Stemler et al. 2009). It is used to measure competence in mathematics in academic domains across a wide range of primary school grades, ages and number of years spent at school. This test has been standardized on a large population of primary school students in Zambia. It reported a satisfactory internal consistency among items of α = .77. Test–retest reliability using Spearman rank coefficient revealed a significant correlation of ρ = .56 (Stemler et al. 2009). Items are arranged in order of increasing difficulty. The test administrator asked the participant to choose the correct response from four possible responses. For each correct response, a score of 1 was given. Therefore the range of possible scores on this test was 0–30. The test–retest reliability for a small subsample in this study was found to be r = .80 (N = 12).

Graphogame intervention

GraphoGame™

This is a computer-mediated online environment for learning letter-sound correspondences developed in Finland. It was designed with efficient feedback properties and can be installed on PC, cell-phone or tablet platforms. It was initially developed for students with specific learning difficulties learning to read (individuals with dyslexia or at risk of dyslexia). The game provides an index of initial letter knowledge based on assessment of the child’s letter knowledge before they play the game. It also provides an index of final letter knowledge after the child has played the game. The focus of the training was initially letter-sound knowledge and the steps needed thereafter for learning to read, if the training continues over a prolonged period of time. It is worth mentioning in this context that the intervention time provided in this research was too short (<100 min) to go beyond letter sounds.

In this study GraphoGame™ was administered to students and some of their teachers. A version of GraphoGame™ in the medium of first-grade instruction, ciNyanja, was mounted on cell phone handsets with headphones. It was presented to Grade 1 students at their schools in small supervised groups. The groups played the game concomitantly.

GraphoGame™ is based on repeated trials in which the player must choose from 2 to 8 alternatives, the letter that corresponds to the phoneme s/he hears from headphones. It is highly adapted to the progress of the students, keeping the success rate high (^80 %). It starts from easy to differentiate phonemes and proceeds to more difficult ones, before moving to larger units such as letter combinations of two, three or more. The computer programme of GraphoGame™ records the duration of each play session and the player’s choices on each trial. Each players’ selections are analyzed as a time series, showing the development of the player’s performance as a function of time.

The daisy graphs of which an example is shown below in Figs. 1 and 2 provide a detailed individual performance profile generated by GraphoGame™. The letter in the center of the circle represents the target letter that corresponds to the stimulus sound presented. The player had to learn to pick the target letter instead of the alternative letters shown at the same time. The letters inside the leaves in the illustrations below are letters that the child sometimes selected incorrectly when presented with the target letter. The closer to the center the widest proportion of the leaves is, the greater the indication that this letter was frequently chosen by the player, instead of the target letter.
Fig. 1

Daisy graph (a) showing the initial pattern of responses by a player to the target letter D

Fig. 2

Daisy graph (b) showing the same player’s responses to the same target letter after several sessions of playing GraphoGame™

Fig. 3

The order of estimated marginal mean scores on the spelling test for the control and the various combined intervention groups that played GraphoGame™. The Spelling pretest scores were used as a covariate

The daisy graph (a) shows the players’ initial pattern of responses when the sound for the target letter was presented in the game.

This graph illustrates that this particular player had difficulties differentiating K, H, C, G and A from the correct letter D when the /d/ sound was presented. These letters may have been frequently chosen because the English letter name of D is /dii/ containing a strong common feature with those of C /sii/ and G /djii/. This association with the target sound illustrates how the problem of inappropriate introduction of the English letter names in teaching reading of local languages affects the students. Teachers and parents tend to contribute this irrelevant learning experience to students because the Zambian environment is heavily affected by the official language English and yet the local language used for literacy instruction has a transparent writing system with clearly different letter sounds from English.

The data in graph (b) shows the same players’ response to the same letter after several sessions of playing GraphoGame™.

The graph indicates that the child had progressed substantially in mastering the different sounds of letters that were initially difficult to differentiate. The numbers on the outer ring indicate the number of trials on which each letter had occurred during the playing sessions. All other incorrectly selected letters were still chosen only 1–2 times over many trials but the letters A, B, E and I were still chosen repeatedly instead of the correct letter D. It is worth noting that some of these incorrectly chosen letters are still confounds possibly affected by the English letter names containing the same/i/or/ii/sound.

Teacher training

A total of 68 teachers from the classes that were randomly sampled (two teachers from each school) were included in the study. They were invited to a 2 day workshop at the University of Zambia. The workshop had four objectives
  1. 1.

    To orient all the teachers on the importance of letter-sound knowledge in teaching of initial literacy in transparent writing system like ciNyanja.

     
  2. 2.

    To introduce some of the first grade teachers to the desktop and phone installed GraphoGame™. This was done only for the teachers who were in the TIG group (N = 19).

     
  3. 3.

    To share with the teachers how they could incorporate the GraphoGame™ in the existing school curriculum.

     
  4. 4.

    To test the letter sound knowledge of some of the teachers in ciNyanja, the language of initial literacy instruction in Lusaka. This was done only for the teachers in the TIG (N = 19).

     

On the first day, participants in all five conditions were introduced to the overall objectives of the RESUZ project. They were also given an overview on the administration of the four assessments to be carried out in the schools. On the second day, a sub group of 19 first grade teachers was introduced more intensively to GraphoGameTM, for example, how it can be used to help students with difficulties in acquiring basic reading skills in languages with transparent orthographies. This activity was conducted after the phonological test had been administered to them. They were then introduced to the Nokia phones, which had the GraphoGame™ installed. After a demonstration by one of the Project Leaders each of these teachers was given a phone and headphones, with instructions to play the game at least twice per day for at least one hour. The teachers understood the instructions quite well and responded to the game positively. This was observed in the way they enthusiastically played the game.

These 19 teachers were then introduced to the desktop installed GraphoGame™ connected to the internet. Each participant was given a username and password in order to access the game. A demonstration of how to play the game on the desktops was conducted. Each participant was then given an opportunity to play for about 30 min.

Training of research assistants

Twelve University of Zambia undergraduate students from the Psychology Department and School of Education were selected to become research assistants (RAs). They received training from the project leaders on the use of the four tests and GraphoGame™. The project leaders in this study were five PhD students who were responsible for supervising the RAs and managing the data collection process. Important aspects of research were first presented to the RAs. These aspects included ethics in research, establishing rapport with the students and the importance of uniformity of the testing procedures. The various instruments were then explained to them in detail; their use, administration and scoring. The RAs then conducted role plays with each other. They were given an opportunity to ask any questions regarding the use of the tests.

The RAs also received one day training in the use of GraphoGame™. Presentations were given on the origins of the game, how and why the game has been used and its effectiveness in helping students learn to read. The administration of the game was explained in detail and RAs were given an opportunity to ask questions. They were given time to play the game individually. Role plays on the administration of the game were conducted thereafter. After the training, each RA was given a phone and headphones. They were then encouraged to play several sessions per day. This was done to ensure that the RAs became well acquainted with the game before administering it to the students. In addition, the RAs went on field visits in order to orient them to logistical challenges, for example, preparation of space where the administration of the tests and GraphoGame™ intervention would take place.

Procedure

Through schools, consent was obtained from the parents for their students to participate in the research. At each phase of testing, students were made to feel at ease. They were invited to give their assent to participate. Both teachers and students were informed that they were welcome to ask questions about the research. During the process of conducting the research some students dropped out for various reasons which included transfers and absenteeism. The data from five schools had to be excluded due to procedural errors during baseline testing. All other students willingly completed all the tests and the play sessions. Data collection was conducted in three phases. Baseline data was collected in the first phase, which was followed by an immediate retest within two weeks. The GraphoGame™ intervention was conducted in the second phase. This was immediately followed by a posttest in the third phase. Table 4 in Appendix provides descriptive statistics at each of these phases for the pencil and paper tests.

Design

The study is an experimental design, at the core of which is an intervention with a representative sample of grade one students and their teachers. Participants–students and teachers-received exposure to the game in two phases, one in term 2 and the other in term 3. Each of these phases comprised multiple short sessions spread over several consecutive days amounting to a total planned exposure time of about 4–5 h. On each day of exposure, a student was given six short play sessions with the game, lasting, between 7 and 9 min and separated by rest intervals of 1–10 min.

In phase one each student received 18 play sessions spread over 3 days. As absenteeism reduced the playing time of many students, the lowest playing times were below 50 min. In the second phase, the play sessions increased to 30 over a period of 5 days. The total exposure time for these phases was initially planned to be at least 3 h. This was reached by only few students. The longest exposure time was about 190 min Due to practical difficulties, the mean time was only 94 min. This was also the approximate median time. Data on actual exposure time were retrieved from the GraphoGame™ records. For each of the phases, post tests were conducted for all the four pencil and paper tests at the end of the term 2 and term 3 respectively.

Intervention groups

The intervention groups were formed to investigate the potential effects of GraphoGame™, on individual groups and various combinations of the groups.

C—control group comprised of students and teachers that did not receive any intervention. Arrangements were made to ensure that these students did not receive exposure to the game. The teachers were informed about the study.

TG—teacher exposed to Graphogame—teachers were given GraphoGame™ to play. They received minimal instructions about the principles of the game. Students did not play GraphoGame™.

TIG—consisted of the 19 teachers who received intensive instruction and exposure to Graphogame. They received instructions about the principles and the benefits of knowing letter sounds. They played the game but their students did not.

LG—students played GraphoGame™. Their teachers were merely informed about the study as was the case for group C.

LG + TG—students and teachers played GraphoGame™. Teachers received minimal instructions about the game.

LG + TIG—students and teachers played GraphoGame™. Teachers received intensive instructions about the principles and the benefits of knowing letter sounds.

Table 1 shows the cross tabulations for age and sex distribution for the intervention groups, N = 259 of which 52.90 % are females and the control group, N = 314, of which 51.26 % are females.

Data analysis

Basic descriptive information about the intervention group is presented in Table 2. Data analyses were conducted using SPSS 19. Reliability analyses were conducted using test–(immediate) retest correlations with raw scores for the four tests. One-way ANCOVA with pretest scores as a covariate was used to find out the effect of GraphoGame™ training on the performance of the students on the Spelling and Orthography tests. These analyses were conducted for several united intervention groups (LG, LG + TG &LG + TIG, TIG &TG vs C). The main motivation for combining the groups for purposes of analyses was to overcome the limited statistical power associated with small Ns of the individual groups and to determine which combination would be most effective to use with the GraphoGame™. Our hypotheses were as follows: we expected that students exposed (directly or indirectly through their teachers) to GraphoGame™ would show greater improvement in their mastery of spelling than control group students. No impact of the intervention was expected on the tests of vocabulary or arithmetic. With respect to orthographic awareness we expected an influence of GraphoGame™, less prominent than on the decoding test.
Table 2

Posttest scores for the spelling and orthography test

 

N

M

SD

Spelling

   

 No GraphoGameTM intervention (C)

147

8.69

4.20

 Students only played the game (LG)

38

10.13

4.33

 Students and teachers played the game (LG + TG & LG + TIG)

63

10.35

5.23

 Teachers only played the game (TG + TIG)

42

10.79

5.41

Orthography

   

 No GraphoGameTM intervention (C)

160

22.18

7.04

 Students only played the game (LG)

41

23.66

7.75

 Students and teachers played the game (LG + TG & LG + TIG)

68

21.49

8.20

 Teachers only played the game (TG + TIG)

43

23.23

9.85

Control versus various intervention conditions

Further analyses were conducted to find out if the post-test group means were different after adjusted pre test scores in the means of the various grouping conditions for the Spelling, Maths and Orthography tests. This was conducted firstly for the control versus the various intervention combinations (LG—students only played the game, LG + TG and LG + TIG—both the students and the teachers played the game and TIG + TG—teachers only played the game). Secondly, it was conducted for a comparison between the control group and the individual intervention groups (C vs LG, LG&TG, LG&TIG, TIG and TG respectively). Table 3 below presents results for the ANCOVA analyses, while Fig. 3 represents the order of estimated marginal means for the Spelling test, which were also tested against the linear trend and found to be statistically significant (difference = 1.285, SE = .431, p < .01)

Note A detailed decription of the intervention combinations is given in the sub-section titled "Intervention groups"

For the different intervention groups, we expected that the greatest effect would be observed according to the extent of GraphoGame™ related training firstly in the LG + TIG group, secondly by LG + TG, followed by the LG group, then the TIG group, then TG and lastly C. The united intervention groups would be ordered as follows: LG + TIG & LG + TG (teachers and students both played the game) followed by LG (students only played the game) then TIG&TG (teachers only played the game) and lastly C (control group). This expected ordering of the groups is based on the intensity of students’ exposure to the GraphoGame™.

Results

Table 4 in Appendix presents descriptive results for the pre-intervention assessments for the control and the individual intervention groups in the form of sample sizes, mean and standard deviation scores for all the four tests at pre and posttest. A reduction in the sample size at posttest, compared to the initial sample size at pre-test was due to the attrition of pupils experienced between the two testing periods. The attrition rates for all the paper and pencil tests as observed at posttest was 46 %. A detailed explanation on the missing data is provided in the discussion section. Results also show that on each test except for the PVT, there was an increase at post-test when compared with pre-test scores approximately 7 months after collecting the baseline data (see Table 4 Appendix).

To assess the reliability of the tests, test–retest correlations were conducted using raw test scores at baseline and retest scores at various intervals. The first retest was conducted with a small sample within two weeks of collecting the baseline data. The correlation of initial retest scores with baseline scores for the Orthography test was r = .50, N = 44, p < .01 and for the spelling test r = .82, N = 43, p < .001. These results show good reliability for the spelling test but only moderate reliability for the Orthography test.

The PVT and the Math test were not administered during the immediate retest due to limited time. The second retest was conducted for all the four measures. Since the interventions were expected to influence the students’ scores on at least some of these tests, this reliability analysis was conducted for the control group only. Correlation coefficients for the measures were as follows; Orthography test r = .61, p < .001, N = 160; Spelling test r = .65, p < .001, N = 147; PVT r = .23, p < .01, N = 159 and Arithmetic r = .42, p < .001 N = 160. These results further confirm the reliability of the spelling test and suggest fair reliability for the orthography test and rather weak reliability for the PVT and the Math. Baseline scores of the measures were positively and significantly correlated with the students’ self-reported familiarity with ciNyanja as follows: Orthography test r = .12, p < .01, N = 572; Spelling test r = .12, p < .01, N = 544; PVT r = .16, p < .001, N = 573 and Maths r = .14, p < .01, N = 570.

These small, positive and significant correlations confirm the expectation that students with less exposure at home to the language used as medium of instruction will perform less well on various cognitive measures administered in that language than students with greater exposure. However, the magnitude of the correlations is quite low, showing that even students with lower home exposure to ciNyanja were able to demonstrate some of their competence in that language.

In order to determine the impact of GraphoGame™, one way ANCOVA analyses (presented in Table 3) were conducted of posttest scores on the decoding and orthography tests using pretest scores as a covariate. Table 2 presents the means of the Spelling and Orthography test post scores derived from these analyses.
Table 3

ANCOVA results for spelling, orthography and maths tests for various intervention combinations and individual intervention groups

DV

IV

df (df1,df2)

F

Sig

Partial eta square

Mean difference

SE

Spelling (post)

C, LG, LG + TIG&LG + TIG, TG&TIG

3,285

3.88

<.05

.04

  
 

Covariance: Spelling (pre)

1,285

138.09

<.001

.33

  
 

Contrasts: (C vs. (LG + TG) & (LG + TIG))

  

*<.01

 

1.86

.58

 

Contrasts: Linear trend

  

<.01

 

1.29

.43

 

C, LG, LG + TG, LG + TIG, TG, TIG

5, 283

2.34

<.05

.04

  
 

Covariance Spelling (pre)

1,283

136.36

<.001

.33

  
 

Contrasts: (C vs. (LG + TIG))

  

*<.05

 

2.00

.67

 

*p value was adjusted with the Bonferroni correction to avoid type I error

      

Orthography (post)

C, LG, LG + TIG&LG + TIG, TG&TIG

3,307

.29

>.05

.003

  
 

Covariance: Orthography (pre)

1,307

152.93

<.001

.33

  
 

C, LG, LG + TG, LG + TIG, TG, TIG

5,305

1.04

>.05

.02

  
 

Covariance: Orthography (pre)

1,305

155.87

<.001

.34

  

Maths (post)

C, LG, LG + TIG&LG + TIG, TG&TIG

3,307

.59

>.05

.006

  
 

Covariance: Math (pre)

1,307

44.08

<.001

.13

  
 

C, LG, LG + TG, LG + TIG, TG, TIG

5,305

1.26

>.05

.02

  
 

Covariance: Math (pre)

1,305

46.15

<.001

.13

  

Data generated from GraphoGame™ logs of the students who played the game were subjected to regression analysis to find out if initial letter knowledge or GraphoGame™ exposure time would predict final letter knowledge. Exposure time represents the total time the student was exposed to the learning content in the game. Pause times are not included in the play time. The sample for the analyses consisted only of the students who played the game (N = 96). This sample size is less than the original sample that played the game (N = 148) due to technical errors with game log data transmission. The tested regression models fit significantly well to the data. Model1, F (1,94) = 24.22, p < .001 shows that initial letter knowledge, as the only predictor of final letter knowledge. It explains (adjR2) 20 % of the final letter knowledge β1 = .91, p < .001. Model 2, F(2,93) = 16.96, p < .001, in which the amount of exposure time was added as a new predictor to model 1, only explained 6 % more of the variance of the final letter knowledge scores. The new predictor was significant β2 = .05, p < .01 and β1 = .88, p < .001.

An analysis of GraphoGame™ logs and pencil and paper tests showed that letter-sound knowledge was significantly correlated with decoding (spelling) test scores at baseline r = .50, p < .01, N = 94 and at posttest r = .42, p < .01, N = 72.

Discussion

The study reported here was conducted to establish the effectiveness of a computer based literacy game (GraphoGame™) in improving literacy skills of grade one students in Zambian public schools. The results support our research hypotheses. GraphoGame™ produced significant improvements in the performance of the students who were exposed to it directly (when the individual student was exposed) or indirectly (when the teacher alone played the game) compared to students in the control group as evidenced by the larger increase of mean scores from pretest to posttest as a function of the exposure to GraphoGame™.

Analyses further show that this improvement was most prominent when both the students and teachers played the game and the teachers were introduced to the phonics approach so that they would not provide incompatible instruction. Results from the teachers and the students’ individual play logs from separate phones were combined and analyzed. A significant effect of playing the game was observed on participants’ decoding skills but not on orthographic awareness, on which limited exposure to the game (<100 min) cannot be expected to yield any substantial improvements. Intervention conditions in which both the students and the teachers were exposed to the game produced the greatest effect on the decoding test. Our analysis also found that initial letter knowledge was a good predictor of performance in predicting final letter knowledge in GraphoGame™. Furthermore, letter-sound knowledge documented by the GraphoGame™ programme, is a significant predictor of performance on a conventional paper and pencil test of decoding skill.

The above findings provide evidence that GraphoGame™ is a tool that can be used to help Zambian students learn to read by enhancing their letter-sound knowledge. The effect of playing the game is greater, when teachers are introduced to letter-sound knowledge in ciNyanja in order to avoid interference from English letter names. The PRP used in Zambian schools emphasizes letter-sound correspondence as basic introduction to literacy especially at grade one and grade two levels (NBTL & SITE). This means that these two approaches (PRP course and GraphoGame™) can complement each other in the teaching of basic literacy skills of students especially in the early grades. However, it is clear that there are other factors affecting literacy learning in first grade Zambian schools. These include individual differences due to biological and/or social factors beyond the classroom.

Our finding that initial letter knowledge accounts for 20 % of the variance of final letter knowledge is consistent with findings among Finnish students varying in terms of familial risk for dyslexia by Lyytinen et al. (2009). The findings that duration of exposure time to playing GraphoGame™ accounted for a further 6 % of the variance suggests that over and above individual differences, the opportunity to play GraphoGame™ increased students’ decoding skills.

The sample in this study was randomly sampled from the population of public primary school entrants in Lusaka, the capital city of Zambia. The research team has followed this cohort of students since they were in second term of grade one in 2011. This year they entered the third grade. Plans are underway to conduct further testing of this cohort during the course of their third year of schooling. A major characteristic of this sample is that it is composed of participants mostly from underprivileged backgrounds. Parents from the middle and upper income groups in Zambia tend to enroll their students in private schools. Students from low social economic backgrounds thus make up the great majority of enrolled students in Government schools. Thus our study shows that GraphoGame™ is an effective intervention for a segment of the national population in need of additional educational support in the area of literacy.

Given that students in this sample were randomly sampled and assigned to different intervention groups, these findings show a reliable impact of the game on students’ mastery of spelling even though the exposure time was very short. Also noteworthy is the fact that GraphoGame™ was administered under less than optimal but ecologically realistic conditions. The research team experienced frequent interruptions and distractions due to various adverse circumstances that affect most public primary schools in Zambia and indeed many other African countries. These include overcrowding, weak infrastructure, poor working conditions of teachers, low levels of family literacy, scarcity of teaching materials, and absenteeism of students.

The instruments used in this study were selected with great care. The PVT and the arithmetic tests have undergone local adaptation with a sample similar sample of students. Moreover, our findings showed modest reliability for the tests especially the Spelling test, which produced highest correlations for the immediate retest and the final posttest. Students in the control group who were not exposed to GraphoGame™ intervention showed a significant improvement in decoding, orthographic awareness and arithmetic skills (see attached Table 4 in Appendix), from the middle of term 2 to the end of term 3, as would be expected from the general experience of grade one enrolment. These results provide an indication that the Spelling test can be used in future research to measure decoding skills although its sensitivity might be enhanced by increasing the number of items that assess letter-sound knowledge as well as knowledge of syllables.
Table 4

Pretest and posttest descriptive results for the control and individual intervention groups on all four tests

  

Pretest

 

Postest

  

C

LG

LG + TG

LG + TIG

TIG

TG

 

C

LG

LG + TG

LG + TIG

TIG

TG

 

Orthography

             
 

 N

314

62

47

69

67

13

N

160

42

22

46

38

5

 

 Mean

15.44

16.68

15.26

14.87

16.69

13.38

Mean

22.18

23.79

22.68

20.91

22.82

26.40

 

 SD

7.73

7.96

6.45

7.41

7.91

9.26

SD

7.03

7.70

8.76

7.94

10.4

1.51

 

Spelling

             
 

 N

299

57

46

65

64

13

N

160

42

22

46

38

5

 

 Mean

7.58

7.42

7.43

7.63

8.39

7.23

Mean

8.59

9.88

9.55

10.43

10.76

10.20

 

 SD

3.28

2.87

2.98

3.30

3.68

3.85

SD

4.10

4.49

5.52

5.10

5.21

7.19

 

PVT

             
 

 N

314

63

47

69

67

13

N

159

42

22

46

38

5

 

 Mean

23.25

24.02

22.81

22.45

22.84

22.31

Mean

23.19

23.74

20.64

24.61

24.47

24.80

 

 SD

3.19

2.93

3.02

3.27

3.65

3.61

SD

5.49

5.13

7.41

2.45

4.29

2.28

 

Maths

             
 

 N

312

63

47

69

66

13

N

160

42

22

46

38

5

 

 Mean

15.68

16.86

15.23

15.52

16.62

12.15

Mean

17.19

18.45

17.55

16.46

17.34

20.60

 

 SD

4.95

4.61

4.77

4.46

4.98

5.68

SD

5.00

5.13

6.47

5.16

4.85

4.56

Three aspects of this research make it unique over and above the use of digital technology as a resource for early literacy instruction. Firstly, it is the only literacy study in Zambia that has worked with such a large sample of grade one students. Secondly, it will generate knowledge that was never available about the progression of literacy acquisition of first grade students. Thirdly, one of the strengths of the study is its ecological validity. However, including the adverse ecological conditions gave rise to multiple sources of “error variance” in the data. It is thus not surprising that the proportion of variance in decoding skill improvement attributable to GraphoGame™ exposure was only modest (partial. η2 = .04) since many other variables were not indexed in this study that would be expected to have an influence. Several linked sub-studies offer a way out of this constraint by generating systematic ways of partitioning that error variance.

Learning to read is based on adequate instruction. It is also affected by individual differences, some of which are biologically determined, while others arise from variations in home language and home literacy environment. These are the subjects of ongoing sub-studies. Based on the findings of these sub-studies, the research team hopes in future to include quantitative indicators of students’ home environment (Chansa-Kabali) and teacher characteristics (Jere-Folotiya), linguistic competencies and special needs as additional independent variables predicting the rate of students’ initial literacy acquisition in a multivariate research design.

Lack of sound proof rooms for the administration of GraphoGame™ and inadequate space to conduct the testing was a major constraint. Absenteeism experienced during the various phases of GraphoGame™ intervention was another factor, especially during the second phase of GraphoGame™ intervention which took place from late November to early December. This is the time national examinations for grade seven, nine and twelve classes take place. Students in lower grades are advised to stay away from school in order to use their classrooms for examinations. It is also the rainy season. Absenteeism during this season increases due to difficulty of navigating footpaths and increased levels of disease. Absenteeism of students who were randomly sampled for GraphoGame™ intervention meant that they had reduced exposure time or did not participate in the posttest assessments. This largely explains the reduction in the mean total exposure time from the intended 3.5–4.5 h to the observed actual mean of 93.82 min. The total attrition rate for the sample was 46 %. Other than absenteeism, the high attrition rate could also be attributed to transfer of students to other schools and administrative challenges experienced in some schools.

The findings reported in this article have implications for the way in which literacy instruction is taught in the local languages, not just in Zambia but in other African countries. In Zambia, the use of local languages, which have a transparent alphabetic writing system aids the mastery of connections between letters in printed and spoken form. In order for this to take place effectively students need to be taught the letter-sound correspondence in a structured way. This structure is offered by the PRP but needs substantially more emphasis given to the basic letter-sound knowledge. GraphoGame™ helps to further cement that structure.

This leads to our recommendation that GraphoGame™ can be used as a tool to supplement the teaching of literacy in Zambian schools. It further suggests that intervention with the game should focus on both students and teachers, as the greatest effect was observed when the students and teachers were both exposed to the game. The GraphoGame™ could also prove useful to other African countries that use local languages with transparent orthographies to teach early literacy but still encounter low literacy acquisition levels. Currently, the game is being piloted in Kenya and Tanzania, which are countries that have similar ecological challenges as Zambia.

Limitations

The RESUZ study has been the only large scale study conducted in Zambia using GraphoGame™. But the sample was not representative of the entire Zambian population of first grade students. The study was confined to Lusaka District where ciNyanja is used as the language of instruction. The Zambian curriculum uses seven local languages to teach initial literacy in the different regions of Zambia. The fact that GraphoGame™ proved effective in ciNyanja, cannot be generalized to the other six languages, although results of the National Assessment conducted at grade 5 level show that literacy learning in ciNyanja was significantly weaker than in the other three main Zambian languages (MOE 2010).

Due to various constraints, a high level of attrition was experienced in this study. Further research should seek to minimize this as it can be a source of uncontrolled bias. The amount of playtime was also a limiting factor. The students were not exposed to the amount of actual play time that was found to be optimal for students at this stage of literacy acquisition in Finland. Increased playtime beyond the duration achieved in this study might have yielded a greater effect of the game.

Differences of culture and technology are also worth mentioning. GraphoGame™ was designed in Finland where students are exposed to technology at an early age. This is unlike the sample that was used in this study, whose exposure to technology is very minimal. Based on this difference, it is possible that the Zambian students in this sample may have needed more time to learn the rules and adapt to the game. With the above background in mind, it is possible that this cross-cultural phenomenon may have been a source of noise in the study. One possible solution to this cultural difference is to gradually expose Zambian students to the game. This means more frequent exposure to the game over a prolonged period of time. This might give rise to a stronger impact of GraphoGame™ on the literacy learning of Zambian students.

It is therefore recommended that, before the game is introduced on a large scale in Zambian schools, further research be undertaken that will take the above mentioned limitations into consideration. Researchers should be able to state what the optimal play time and essential teacher-guided introduction to the game should be for the Zambian population. This information will help provide guidelines on how long the game should be played before an effect can be observed.

Conclusion

The low literacy acquisition level of Zambian students is a problem to which a solution urgently needs to be found. The failure to attain desirable literacy acquisitions levels at the early primary stage continues to affect their performance not only in reading and writing in ciNyanja but in English as well. GraphoGame™ can be used as an independent supplementary learning activity to support the current curriculum used in Government schools. It will reinforce the letter-sound knowledge and phonemic awareness already being taught in the classrooms. With the fairly large number of students in a classroom, the game can be played by the students without the direct supervision of the teachers. A schedule can be prepared so that students can take turns playing the game for a specified period of time each day. All the students would need to be adequately trained in the use of the game. Regular monitoring of their GraphoGame™ logs would need to be conducted. This would help establish the progress of the students and identify problematic areas with the aim of providing additional intervention where needed. It can also be used as a training tool to enhance teachers’ metalinguistic awareness of the difference between the Zambian languages and English orthography. This will bring attention to the challenges this poses for initial literacy learning by Zambian grade one students.

Notes

Acknowledgments

This research report was made possible through funding received from the Academy of Finland for the Reading Support for Zambian Children (RESUZ) project. The authors would like to thank colleagues, that took part in the study from the Agora center at the University of Jyväskylä for all their efforts in programming the GraphoGameTM, University of Zambia for the technical support, the research assistants, learners and teachers that made this research possible.

References

  1. Aro, M. (2006). Learning to read: The effect of orthography. In P. Aaron & M. Joshi (Eds.), Handbook of orthography and literacy (pp. 531–550). Mahwah: LEA.Google Scholar
  2. Aro, M., & Wimmer, H. (2003). Learning to read: English in comparison to six more regular orthographies. Journal of Applied Psycholinguistics, 24, 621–635.Google Scholar
  3. Banda, F., Mtenje, Al., Miti, L., Chanda, V., Kamwendo, G., Ngunga, A., et al. (2008). A Unified Standard Orthography for South Central African Languages (Malawi, Mozambique and Zambia). (Second, Revised Edition). Cape Town, South Africa: Centre for Advanced Studies of African Society (Monograph Series No. 229).Google Scholar
  4. Dunn, L. M., & Dunn, L. M. (1997). Peabody picture vocabulary test (3rd ed.). Bloomington: Pearson Assessments.Google Scholar
  5. Fink, G., Matafwali, B., Moucheraud, C. & Zuilkowski, S.S. (2012). The Zambian Early Childhood Development Project—2010 Assessment Final Report. (Retrieved November 2012 from http://developingchild.harvard.edu/activities/global_initiative/zambian_project/).
  6. Kanyika, J. (2002). Reading comes first: Follow-up on baseline reading study. Lusaka: Ministry of Education.Google Scholar
  7. Kashoki, G. E. (1990). The factor of language in Zambia. Lusaka: Kenneth Kaunda Foundation.Google Scholar
  8. Kyle, F., Kujala, J. V., Richardson, U., Lyytinen, H., & Goswami, U. (2013). Assessing the effectiveness of two theoretically motivated computer-assisted reading interventions in the United Kingdom: GG Rime and GG Phoneme. Reading Research Quarterly, 48(1), 61–76.Google Scholar
  9. Lyytinen, H., Aro, M., Holopainen, L., Leiwo, M., Lyytinen, P., & Tolvanen, A. (2006). Children’s language development and reading acquisition in a highly transparent orthography. In R. M. Joshi & P. G. Aaron (Eds.), Handbook of orthography and literacy (pp. 47–62). Mahwah: Erlbaum.Google Scholar
  10. Lyytinen, H., Erskine, J., Kujala, J., Ojanen, E., & Richardson, U. (2009). In search of a science base application: A learning tool for acquisition. Scandinavian Journal of Psychology, 50(6), 668–675.CrossRefGoogle Scholar
  11. McGuinness, D. (2005). Language development and learning to read: The scientific study of how language development affects reading skill. Cambridge: MIT Press.Google Scholar
  12. Ministry of Education. (1996). Educating our future. National Policy Education Zambia. Lusaka: Educational Publishing House.Google Scholar
  13. Ministry of Education. (2010). Learning achievements at the middle basic level: Zambia National Assessment Survey Report—2008. Lusaka: Zambia Examination Council.Google Scholar
  14. Ministry of Education. (2012). 2012 Examinations Performance Review. Examinations Council of Zambia (ECZ): Lusaka.Google Scholar
  15. Musonda, B., & Kaba, A. (2011). The SACMEQ III Project in Zambia: A study of the conditions of schooling and quality of education. In SACMECQ III, National Reports, Zambia.Google Scholar
  16. Ojanen, E., Kujala, J., Richardson, U., & Lyytinen, H. (2013). Technology-enhanced literacy learning in Zambia: Observations from a multilingual literacy environment. Insights on Learning Disabilities, 10(2), 103–127.Google Scholar
  17. Saine, N. L., Lerkkanen, M.-K., Ahonen, T., Tolvanen, A., & Lyytinen, H. (2010). A computer-assisted remedial reading intervention for school beginners at-risk for reading disability. Child Development, 82, 1013–1028.CrossRefGoogle Scholar
  18. Sampa, F. (2005). Zambia’s primary reading programme: Improving access and quality education in basic schools: ADEA report.Google Scholar
  19. Sampa, F., Linehan, S., Chibowa, G., & Edwards, F. (2003). Output to Purpose Review, Primary Reading Programme. http://www.adeanet.org/adeaPortal/adea/downloadcenter/CD/04_zambia_en.pdf.
  20. Seymour, P. H. K., Aro, M., & Erskine, J. M. (2003). Foundation literacy acquisition in European orthographies. British Journal of Psychology, 94, 143–174.CrossRefGoogle Scholar
  21. Stemler, S. E., Chamvu, F., Chart, H., Jarvin, L., Jere, J., Hart, L., et al. (2009). Assessing competencies in reading and mathematics in Zambian children. In E. Grigorenko (Ed.), Multicultural psychoeducational assessment (pp. 157–186). New York: Springer.Google Scholar

Copyright information

© The Author(s) 2014

Open AccessThis article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.

Authors and Affiliations

  • Jacqueline Jere-Folotiya
    • 1
  • Tamara Chansa-Kabali
    • 1
  • Jonathan C. Munachaka
    • 2
  • Francis Sampa
    • 3
  • Christopher Yalukanda
    • 4
  • Jari Westerholm
    • 5
  • Ulla Richardson
    • 6
  • Robert Serpell
    • 1
  • Heikki Lyytinen
    • 6
  1. 1.Department of PsychologyUniversity of ZambiaLusakaZambia
  2. 2.Department of Educational Psychology, Sociology and Special EducationUniversity of ZambiaLusakaZambia
  3. 3.USAID Read to Succeed Education ProjectLusakaZambia
  4. 4.c/o Zambia National Union of Teachers (ZNUT)LusakaZambia
  5. 5.Niilo Mäki InstituteUniversity of JyväskyläJyväskyläFinland
  6. 6.AGORA Human Technology Center, University of JyväskyläJyväskyläFinland

Personalised recommendations