Introduction

Reading is one of the main conditions for individuals’ integrating into society, but a number of international and national surveys indicate that a considerable proportion of today’s young generation leaves school with poor reading skills. Importantly, many problems of poor reading are rooted in early development; early diagnostic assessment of precursor skills is therefore essential for successful decoding and fluent reading. Furthermore, it is vital to monitor the progress children make with the skills and cognitive factors at the foundation of reading comprehension (Hsuan et al., 2018). The early stage of skill development and thus future reading success are determined by several components. A number of studies have demonstrated that phonological awareness plays a crucial role among the linguistic skills in developing early reading skills (Lonigan et al., 2013); it is confirmed as an early predictor of reading, while the predictive power of phonological awareness measured during the kindergarten years is significant (Afsah, 2021; Derby et al., 2020). There are numerous variations in defining phonological awareness in the literature, and both psychological, linguistic, and pedagogical approaches are equally relevant. Phonological awareness is an umbrella term encompassing conscious access to the internal structure of words and the ability to break them down into different sized units (such as syllables and phonemes) (Csépe, 2006), accessing linguistic units and performing operations with them (Blomert & Csépe, 2012; Gillon, 2004), as well as skills involving the identification and manipulation of phonological units within words (Hayes & Flanigan, 2014).

The growing number of studies showing the significance of phonological awareness and the number of instruments examining that area are also growing. Given the potential for digital technology, methods of assessment are becoming more child-friendly and objective. A number of fundamental studies and books have been published which provide details on this increasingly varied research methodology for this cognitive factor of reading (Landerl et al., 2018; Muñoz et al., 2017; Palmer et al., 2018).

Previous methods for measuring phonological awareness have been based on face-to-face, live-voice delivery, but, as technology has advanced, the potential for ICT (Information and Communications Technology) has come into prominence. Computer-enhanced instruments and computer/software-based developmental games have emerged, whose objectivity and cost and time savings are significant. The child population in the twenty-first century uses ICT tools even before learning to read and write (Furman et al., 2018; McKenney & Voogt, 2012; Romero-Tena et al., 2020), and well-designed apps and online games can also boost early literacy skills as well (Sari et al., 2019). Also, phonological game-based modules could play a significant role in developing phonological awareness skills (Kanapathy, 2019). After reviewing survey methods for phonological awareness inside and outside Hungary, we have taken advantage of the opportunities provided by ICT. Having also considered teachers’ and pupils’ positive approach to this area and children’s knowledge of information technology, we have undertaken the task of designing an online phonological test for use in the kindergarten and primary school environments.

Development of Phonological Awareness and its Role in Acquiring Reading Skills

The psychological, linguistic, pedagogic and logopedic approaches to phonological awareness are evenly relevant. According to Bruinsma (2003), Ellis (2009) and Graves et al. (1998), phonological awareness is a significant metalinguistic skill in which language appears as the object of thinking and is dominated by operations among language elements. This skill is a transition between spontaneous speech acquisition and learning and teaching how to read. Phonological awareness encompasses: the conscious skills of accessing the internal structure of words and of segmenting words into units of different lengths (into syllables and sounds) (Csépe, 2006); access to different linguistic elements and the ability to do operations with them (Blomert & Csépe, 2012); and the skills of identifying and manipulating phonological units within words (Zugarramudi et al., 2022).

Research suggests that the ability to identify and differentiate phonemes represents an outstanding predictor of later reading performance (Gharaibeh et al., 2019; Ziegler & Goswami, 2005). The most important factor in early literacy is phonological awareness (Carson et al., 2013; Deacon, 2012; Duff et al., 2014; Wackerle-Hollman et al., 2013). Schuele et al. (2008) highlight that early phonological awareness training for low-achieving children has a positive impact on children at risk for reading difficulties (e.g. on spelling). According to Barbour et al. (2003) and Goswami (2002), there are two different levels that can be identified: the phonological and phonemic levels. The phonological (or non-phonemic) level consists of making up rhymes, segmenting syllables and separating syllables and rhymes. Phonemic awareness includes manipulating skills as well, deconstructing (segmenting) words into sounds, blending words from sounds (synthesis), identifying the position of individual phonemes in words (isolation) and substituting or omitting sounds within a word (deletion). Based on the approach presented by Farrall (2012), phoneme-level awareness develops by the final phase of kindergarten. Several theories agree that phonological awareness is one of the most determining factors in the early period of reading development (Carson et al., 2013; Wackerle-Hollman et al., 2013); moreover, within phonological awareness, phonemic awareness measured at preschool age becomes prominent (Duff et al., 2014).

According to the available literature on phonological awareness, syllable-level phonological awareness emerges during preschool age in an individual's language development (Ziegler & Goswami, 2005). Certain "simpler" subskills of phonemic awareness, such as identifying or differentiating speech sounds, also develop at this stage. It is important to note that in Anglophone cultures, schooling begins at the age of five, and children in the last year of preschool, in the so-called preschool phase, start to learn letters within the educational framework. In Hungary, targeted education for children of similar ages is not common, but this does not mean that there are no children in Hungarian kindergartens who acquire the ability spontaneously and/or in other non-institutional settings.

The development of phonological awareness in English and Hungarian-speaking children, as well as the similarities and differences in their development, are highlighted by Jordanidisz (2011) in a study examining phonological awareness in bilingual children. Jordanidisz (2011) explains the developmental differences in terms of variations in phonological awareness development, differences in phonological structure, and the different timing of introducing reading instruction in the two cultures. The comparison presented by Jordanidisz (2011) shows that, except for the mentioned two linguistic units, the same components appear in both languages, but the timing and pace of their development differ. For example, in English, rhyming appears first, followed by syllable segmentation and synthesis, while in Hungarian, syllable segmentation and synthesis emerge first, followed by rhyming. Both cases are followed by manipulation of phonemes. Differences in phonological structure are influenced by the isolating nature of English and the agglutinative nature of Hungarian and related aspects of complex words. Another important factor in teaching reading is that, unlike in the Anglophone world, where institutionalized reading instruction begins at the age of five, Hungarian children start to learn the basics of reading and writing 1–2 years later, at 6–7 years old (Jordanidisz, 2011). Furthermore Hungarian orthography is renowned for its high level of transparency. In a transparent writing system, there is a direct one-to-one correspondence between letters and phonemes, simplifying the task of decoding and enabling learners to read words with accuracy (Csépe, 2005). Consequently, the pronunciation of words can be reliably predicted based on their spelling. The Hungarian language boasts a finely detailed writing system, where individual sounds are typically represented by distinct graphemes, such as letters or letter combinations (Bóna & Váradi, 2022). Additionally, the spelling of words closely mirrors their pronunciation. In languages with high granularity, readers can depend on consistent sound-symbol relationships, facilitating the process of decoding and reading. Hungarian maintains a robust grapheme-phoneme correspondence, ensuring that each letter consistently represents a specific sound or phoneme (Ihász et al., 2023). This regularity streamlines the learning of reading and allows for the dependable application of phonics principles.

Investigating phonological awareness has become increasingly important since the 1970s. Investigation and survey trends on the national and international levels show great diversity; nevertheless, the innovative survey options of the twenty-first century have not been exploited entirely, if at all. To become a proficient reader, there are four main areas that require development: (1) phonemic awareness, which involves recognizing and manipulating individual sounds within words, (2) phonics, which focuses on understanding the connections between letters and sounds, (3) fluency, which entails reading with speed and accuracy, and (4) comprehension, which involves understanding the meaning of the text. Additionally, experts have identified five key steps in the process of learning to read: (1) phonological awareness, which involves becoming aware of the sounds in words, (2) letter-sound relationships, which entails learning the letters and the sounds they represent, (3) vocabulary expansion, (4) fluency in reading smoothly and accurately, and (5) using comprehension strategies to understand what is being read. Phonological awareness, including its subskill phoneme awareness, plays a crucial role in these processes. Despite operating at different linguistic levels (syllable and phoneme levels), phonological awareness involves common mental operations, such as segmentation, synthesis, isolation, and manipulation of language units (Elmesalamy & El-Ater, 2022). Rathvon (2004) examined phonological awareness based on (1) a lack of standardization among tasks, (2) the relative predictive utility of various tasks during different periods of reading development, and (3) examiner and scorer variance. It was found that the order of the recordings of the task types during the tests was not determined in every case, with the context of the tasks, their level of difficulty, their complexity, the person who supervised the survey and the method of providing answers showing differences. The predictive value of tasks is not quite clear; there are divergences by language and instrument as well. In Hungary, recordings based on face-to-face, live-voice delivery are common; however, graphics, tasks with previously recorded audio materials (sample materials) and manual aids to help complete the tasks have emerged in line with international trends. Among the national tests in Hungary, there are proprietary tests (e.g. Gósy, 1995/2006) and adapted instruments (Jordanidisz, 2011). The most widely used instrument is DIFER, a school readiness test battery (Nagy et al., 2004); some of its subtests are already available in an online version as well (Csapó et al., 2014).

As the majority of the studies reported in the literature were related to English (see Blomert & Csépe, 2012), an investigation carried out in Hungarian may enrich the results of the research on learning reading in shallow orthography languages generally (see Landerl et al., 2018; Pfost et al., 2019). In languages with a shallow orthographic system, that is, where elements in word recognition consist of relatively few and/or small units, components of the writing system and spoken language elements show many similarities and thus the level of phoneme awareness develops sooner. In contrast, in deep orthography languages, where the correlation between writing system components and spoken language units is less clear, the first phase of reading development is delayed.

The development of phonological awareness can also be tracked by changing task complexity. The difficulty level of phonological awareness tasks can be approached not only by the size of each language unit and the complexity of the operations to be performed (identification, differentiation and manipulation/operation level), but also by changing word frequency (high-frequency and low-value words) as well as breadth and depth of vocabulary knowledge (Hadley & Dickinson, 2020) and semantic interpretation (real words or non-words) (Racsmány et al., 2005), as well as the position of the voice within a word (at the beginning, middle or end), quality (voicing, duration, and manner and place of articulation) and environment (before or after a vowel or consonant or between two vowels or consonants).

In different orthographies, word frequency comes to the forefront for word recognition, decoding and spelling and other areas of literacy (e.g. letter identification and writing) (Yoon, 2015). In a cross-lingual comparative study, Csépe (2013) found that decoding is a key element in the development of phonological awareness in all orthographies. She included five languages in her study, the highly transparent Finnish and Hungarian, the moderately transparent Dutch and the least transparent Portuguese. Juhász and Kálló (2017) found that the length and number of syllables in the word could also be significant for serial perception, and word frequency may be an influencing factor with the same word length. They found that children achieve better with familiar words or with words to which they have attached a positive emotional element, perhaps because those words may feel shorter.

Familiarity with particular words can be eliminated with non-words. The Hungarian non-word test (Racsmány et al., 2005) provides an adequate set of non-words for phonological awareness assessments and other tests. The non-words correspond to the phonology of the Hungarian language in their phonotactic structure, but they have no semantic meaning. We have seen an increasing number of attempts to find empirical evidence for the construct validity of phonological awareness tests. Research in the domain has produced some theoretical models. Adams (1990) claims that phonological awareness comprises five levels, Goswami and Bryant (1990) argue that three levels may be distinguished, and Barbour et al. (2003) propose a two-dimensional model. According to Adams (1990), the theoretical levels refer to the cognitive skills involved in the manipulation of the linguistic units. However, in the latter two models, the size of the linguistic units is relevant.

Assessment of Phonological Awareness

In a number of countries (e.g. in Britain and the USA), children can already read and write in the year before primary school, so the development of their phonological awareness accelerates, because in these areas preschool program is available before school entry (Van Tilborg et al., 2018) found that children recognise that sounds can be manipulated or that different sounds are represented by different letters as early as in kindergarten. Pullen and Justice (2003) and Storch and Whitehurst (2002) demonstrated that letter-speech correspondence and word decoding capability are formed before children begin learning to read or write. In an international overview, the subtests and tasks of phonological awareness are highly diverse. The order of including tasks and task types is not always predetermined, and the context, difficulty level, complexity, the person administering the test, and the mode of response can vary. The predictive power of tasks is not well established and may differ from language to language and even from one test to another. These assessment tools examine at least one aspect of phonological awareness and include tasks designed for preschool-aged children and first-grade elementary school students. The assessment tools can be categorized into two main components: The first group contains tools that examine the components of early reading skills, along with the investigation of specific subdomains of phonological awareness, such as letter-sound correspondence, pseudoword reading, oral reading, listening comprehension, spelling, word recognition, orthographic, semantic, and syntactic processing, as well as reading fluency (e.g. DIBLES, Good & Kaminski, 2002; Phonological Awareness Screening Test, Adams, et al., 1998). The second group of assessment tools is based on reading components, including tasks related to rapid automatic naming, decoding, handwriting, and spelling. Some of the reviewed assessment tools offer a wide range of tasks and complexities, making them suitable for use from preschool to adulthood (e.g. Phonological Awareness Test–2nd edition, Robertson & Salter, 2007).

In Hungary, phonological awareness tests administered in the kindergarten–school transition contain phoneme identification, differentiation and manipulation, while tasks for the same age group internationally include rapid automatized naming, phoneme-grapheme correspondence and print awareness tasks. This pattern is followed by the work of Csépe (2013) and Tóth (2012); their 3DM-H test, which contains a rapid automatized naming test, is already available for Hungarian children. In Hungary, phonological awareness is mainly examined using face-to-face, pre-recorded test tasks, but similar to international trends, tasks with pre-recorded audio samples and manual aids for task-solving and illustration have emerged. Some of the local tests include tools developed and adapted for the Hungarian language. One of the most frequently used test batteries is the GMP test (Speech Perception and Comprehension Performance Examination; Gósy, 1995/2006), which monitors language skills through 20 subtests in speech perception, speech comprehension, writing skills, and reading comprehension. The DIFER (DIagnostic Developmental Examination System) (Nagy et al., 2004) assesses the basic skills that determine successful school entry in children aged 4–8 years through seven skills measured in face-to-face assessments, with some subtests using task sheets or manual aids. The Phonological Awareness Test (Lőrik & Májercsik, 2015) consists of 40 items, with four items per subdomain (from a total of 10 subdomains), mainly focusing on the phonemic level. The Hungarian adaptation of the American NILD Phonological Awareness Skills Survey (Barbour et al., 2003) was conducted by Jordanidisz (2011) with 22 children. Additionally, the 3DM-H (Tóth, 2012) is the Hungarian adaptation of the Dutch 3DM test, used as an official tool for dyslexia diagnosis, focusing on phonological awareness in monosyllabic word phoneme deletion tasks. The test is standardized for grades 1–4 and also for adults. It can only be administered by trained psychologists, speech therapists, or educational specialists, and the results are interpreted through a cognitive reading and spelling index. The test requires training or purchase to use.

Several studies have explored the construct of phonological awareness and the underlying measurement model. Wagner and Torgesen (1987) and Schatschneider et al. (1999) found that phonological awareness was a one-factorial entity, but Yopp’s results (1988) show a two-factor model. Hoien et al. (1995) found separate factors for phoneme, syllable and rhyme sensitivity, Meira et al. (2019) found distinct factors for syllabic, intrasyllabic and phonemic awareness, Stanovich et al. (1984) data’s show a ten-dimensional model, and Stahl and Murray’s (1994) findings demonstrate a 14-dimensional model.

In sum, different national and international measurements are broad in their assessment of the phonological awareness spectrum, but tests and test methods do not show a unified picture. The issues of standardization of tests, evaluation differences in the criteria and the difference in recording the test, the variability of the testing environment and the qualifications and subjectivity of the administrator raise a number of questions.

Challenges and Promises of Technology-Based Assessment of Phonological Awareness

Technology-based assessment has paved the way for the creation of novel test items (Csapó et al., 2014), while switching from face-to-face to online testing has inspired the creators of the phonological awareness tests to rethink the tasks as well. Between the two measurement methods, differences/changes mainly in the types of tasks and the way of responding can be grouped around changes in their purposes and test conditions.

Face-to-face testing is based on the personal presence and active involvement of the administrator. At the request of the administrator, the child mostly provides a verbal response. One of the most sensitive points in live voice delivery testing plays a part in the phonological awareness test-taking procedure: hearing and differentiating sounds/phonemes. Phonological awareness tasks, like other face-to-face tasks, require verbal answers or clapping, knocking or other playful elements. The quantity and shape of manual aids depend on the age group under examination. During online measurements, face-to-face, sometimes open-ended tasks have been turned into closed items, where children can answer questions by clicking or dragging and dropping.

Molnár (2016) classifies technology-based measurement according to nine criteria of effectiveness. The author emphasizes (1) the economics of testing; (2) the test/testing design; (3) the possibility of immediate, objective, standardized feedback; (4) changes in children’s motivation; (5) innovative task-editing opportunities; (6) the potential for adaptive testing; (7) an expanded test sample; (8) contextual testing and efficient data capture and analysis; and (9) the potential to improve test reliability in innovative testing. These criteria are all met in testing phonological awareness online in kindergarten.

During the face-to-face phonological awareness test, the administrator read aloud the tasks from a pre-purchased, printed or copied sheet of paper and recorded the children’s answers on a pre-purchased, customized evaluation sheet. During online testing, instructions and item prompts are presented by pre-recorded voices and immediate feedback is available. Manual and sometimes subjective evaluation is therefore unnecessary. The feedback provides more information for administrators. They log wrong answers, time spent on tasks and on the entire test, and modified responses.

A number of previous studies explored the effect of assessment mode. Typically, paper-and-pencil and computerized tests were compared, and the results usually indicated positive impacts of digitization on the reliability and validity of the instruments, thus further improving motivation among test-takers (Chua & Don, 2013). There were no significant mode effects found even when the cognitive load of the two testing modes was compared (Prisacari & Danielson, 2017). Previous studies have not indicated a media effect when young children were tested with tablets either (Csapó et al., 2014).

Aims of the Present Study and Research Questions

This study aims to examine some subskills of phonological awareness through online tests, take advantage of the opportunities and meet the challenges of online examinations, and focus on a description of the online testing and on the steps of the test development with kindergarten-aged children.

Thus, three research questions are intended to be answered:

First, RQ1 examined the psychometric features of the test:

  • RQ1 Are the tests suitable to measure the phonological awareness subskills in the particular age groups? Are the psychometric features of the tests appropriate?

Second, RQ2 observed the construct of the test:

  • RQ2 Is the construct validity of the tests acceptable?

Third, the significance of the relationship between tablet using skills and performance on the test was observed (RQ3):

  • RQ3 Is the ability to use the transmitting medium influencing the results reached in the phonological test in the kindergarten-age groups?

Methods

Participants

The sample for the study was drawn from local kindergartens through individual organisational arrangements. The primary aspect of sampling was to assess children turning six by 1 September of the test year, as they were expected to enter school that year. On the one hand, due to the heterogeneous group structure of the kindergarten, the test was also completed by children who did not meet this criterion, on the other hand as online measurement methods are not widely used in Hungarian kindergartens, it was perceived as a novelty by the children, who viewed the testing as a game. In Hungary, there are several cases of heterogeneous groups where children of compulsory school age coexist with younger peers. If parents consented to their children being measured, these younger children could also take the online test. However, their results were not included in our research. Similarly, children with special educational needs status were not included in the analyzed sample, as their measurements are conducted by educational support services. Thus, the data to be analyzed only includes the information of children who (1) are of compulsory school age (turn 6 years old by August 31 of the given year) and (2) do not have a diagnosis from educational support services.

Therefore, in sum, 472 children in 18 kindergartens completed the test, but only 317 of them were in their final year of kindergarten. Of those, 311 children completed the tests. The average age of the participants was 6.61 years (SD = 0.54 years). The children featured in the sample did not participate in direct reading instruction or learning process, they did not receive writing education, and they do not have special educational needs.

Instruments

The phonological awareness test contains 44 closed items. The tasks cover two linguistic levels (phoneme and syllable) and four operation levels (identification, synthesis, segmentation and deletion) (see Table 1). In the case of the syllable-level exercises, we did not consider the identification tasks important because kindergarten-aged children are able to manipulate on the phoneme level. For each task, one point is awarded for a correct solution.

Table 1 The structure of the phonological awareness test
Full size table

During the measurement of phoneme awareness, four types of tasks are presented. In phoneme identification tasks, the children must determine whether they heard the target sound in the word or not. If their response is affirmative, they click on the green tick, and if negative, they click on the red cross (Fig. 1a). The complexity of the phoneme identification task is increased by also asking the child to identify the position of the sound. In this task, they decide by clicking on the front (locomotive), middle, or end (first or second carriage) of the little train to indicate whether they heard the sound at the beginning, in the middle, or at the end of the words during the tasks. In the example task, the child is asked to identify the sound "v" at the beginning of a word “vonat”, and the correct answer is given by clicking on the locomotive (Fig. 1b). During the identification of initial sounds, the child is provided with the names of pictures and then decides, by clicking, which two pictures share the same initial sound. In the sample task, it can be observed that the initial sounds are in the same phonological position, as they are followed by a vowel in all three cases. The task is solved by clicking on the pictures of "banán" (banana) and "barack" (peach) (Fig. 1c).

Fig. 1
figure 1

Example tasks from the phoneme identification subtest. Instructions: (a) Do you hear the sound within the word? If so, click on the pipe, if not, click on the x! (b) Where do you hear the sound within the word? At the beginning, in the middle or in the end? Click on first, second or third part of the train! (c) It's a banana, it's a peach, and it's a horseshoe. Which two start the same way? Click on them! (Hungarian examples: banán, barack, patkó)

Full size image

The operational-level components of phonological awareness (synthesis, segmentation, deletion) were examined at both the syllable and phoneme levels. During the measurement of syllable awareness, three types of tasks are presented. In syllable synthesis tasks, the child hears the prompt divided into syllables, and then they click on the picture representing the heard stimulus. Thus, the correct answer is given by clicking on the picture of "citrom" (lemon) after hearing its syllables. In syllable segmentation tasks, the child breaks down the heard word into syllables and then drags their doll to the field with as many dots as there are syllables in the stimulus word. In syllable deletion tasks, the child hears a word and, by omitting the first or last syllable, receives the name of an object shown on the screen. The task is solved by clicking on the picture. In the example task, by omitting the first syllable ("rádió"), the word "dió" (walnut) is formed, so the correct answer is given by clicking on the picture of "dió" (walnut).

In the phoneme-level synthesis, segmentation, and deletion operations, their appearance and response method show similarity to the syllable-level task types. In the phoneme-level synthesis task, the child hears the sound-separated name of one of the pictures on the screen, then decides by clicking which one they heard (/l-ó/) (Fig. 2a). In the phoneme-level segmentation task, the child click on as many petals as they hear sounds in the word (e.g. bab) (the child can click on any petals, they do not have to be consecutive) (Fig. 2b). The phoneme-level deletion subtest asks the student to omit an initial or final sound in each word (e.g. hód (beaver)-hó (snow). The student needs to click on the picture symbolizing the resulting word on the screen. The sample task can be answered by omitting the final "d" sound and clicking on the middle picture (Fig. 2c).

Fig. 2
figure 2

Example tasks from the operation level’s tasks. Instructions: (a) I will say a word to you, but listen carefully, because I will say it slowly, broken down into sounds. Which image name might it be? Click it!. (b) You will hear words, click on as many petals as many sounds you hear in the word! (c) You will hear a word. Say the word to yourself without the last sound. Which picture represents the word you are saying? Click it!

Full size image

The tasks contain high-, medium- and low-frequency words from the Hungarian corpus Szószablya and non-words from the Hungarian non-word test (Racsmány et al., 2005). In designing the tasks, we preferred simple, concise, but child-friendly instructions and figures (e.g. a robot and a worm), and we were careful to avoid colourful images that would distract children’s attention from the task at hand (see Figs. 1 and 2). Every exercise contained only audio instructions, which children listened to with a two-second delay. The aim of the two-second delay was to provide some time for the children to process the images on the screen and thus not strain their visual and audio perception at the same time.

Design and Data Analysis

At the beginning of the measurement process, before the phonological awareness test, children completed tasks in which we focused on ICT usage. They tapped on items and practised dragging and dropping to ensure that the operations on the tablets were familiar. The ICT infrastructure for preschool testing is still in its infancy. Before measuring, we performed a small number of kindergarten pilot studies, which were not only to explore and ascertain the technical conditions for preschool measurements, but initial tests were also highlighted. In the autumn of 2014, improvements to the eDia system made it possible to convert tests which had been available on a desktop computer for touchscreen devices. In line with the design of the tools used by children of preschool age, we thought it was advisable to develop a touchscreen test format using a large-screen device; therefore, 10.1'' tablets were used to test the kindergarteners. Headphones ensured the test was taken at an individual pace. Technical conditions only allowed five children to be tested at a time.

Although the eDia platform was accessible outside kindergarten, data collection took place in a quiet room of the participating kindergarten. The measurements were conducted in the morning, outside the period of supervised group activities during free playtime, thus not disrupting the children's daily routine. We measured a maximum of five children at once, with the attention of two assessors. The testing took up to 30 min per child, system logged, recorded and automatically scored the answers.

University students supervised the testing sessions. The students had previously attended a university course, where they were familiarized with the theoretical background and practical application of the test. The implementation of preschool measurements required the presence of suitable raters. The administration of online tests was carried out by students majoring in kindergarten education, primary school education, and social pedagogy. Prior to the measurements, the students received guidance on the practical aspects of the work. Attention was drawn to specific situations that might occur in kindergartens (e.g., health breaks), and solutions were sought through situational exercises. The students completed the practice in pairs, and since several students had been recurrent course attendees by the time of data collection in 2016, the pairs of raters consisted of one student who had already gained experience and one student who had recently taken up the position. Preschool measurements require a dual role for the raters, as they demand increased attention throughout the process, from taking the children out of the group room, calming them down, getting them ready for play, to their final storytelling experiences, more so than in the case of students participating in general school testing. Therefore, during the allocation of rater tasks, we suggested that one rater should focus on the successful administration of the test, recording the anonymous measurement identifiers, monitoring and assisting with the dynamic progress in the test, while the other rater should focus on the children's potential needs, requirements, and maintaining their motivation.

The examination was carried out with the approval of the Ethical Committee of the Doctoral School of Education [masked for review], and the parents of all the children in the study signed a letter of informed consent.

Rasch analysis was applied to uncover the abilities of children and the difficulty levels of items in the test. Rasch analysis, based on Item Response Theory (IRT), is a valuable statistical method in educational research for evaluating the psychometric properties of measurement scales and assessing test item performance (Bond et al., 2021). It provides insights into item difficulty and individual ability on a common logit scale, aiding in the identification of misfitting items and enhancing the precision and validity of educational assessments (Boone et al., 2014).

To examine the construct of the test, we established an internal correlation matrix, which served as the basis for the CFA analysis and the formulation of the related model. In the realm of educational research, Confirmatory Factor Analysis (CFA) and Structural Equation Modeling (SEM) are highly valued statistical methods that researchers widely employ to explore complex relationships between observed and underlying variables (Kline, 2016).

Linear regression is a widely used statistical method in scientific research for investigating the linear relationship between a dependent variable and one or more independent variables (Draper & Smith, 1998). It allows researchers to model and predict outcomes, providing valuable insights into the associations between variables in diverse fields of study (Kutner et al., 2005).

Results

The entire test proved reliable (Cronbach’s α = 0.84). The internal consistency of the tasks within the two dimensions is still high as well (Cronbach’s αsyl = 0.66, Cronbach’s αphon = 0.83), though somewhat higher for phoneme awareness than syllable awareness.

A Rasch analysis was performed to detect the fit of the difficulty of the tasks and the ability of the children (EAP/PV 0.84). The person-item map is shown in Fig. 3. The numbers on the right side of the chart show the positions of the items on the difficulty scale, while the left side indicates the children’s level of ability. The Xs on the left side of the figure represent two students each.

Fig. 3
figure 3

The person-item map of kindergarteners for the phonological awareness test (each X represents 2 pupils)

Full size image

The items cover a wide range of difficulty scales with a logit value of between −3.324 and 2.310. Those representing the two extreme values stand out. Those calling for segmentation have a logit value of between 0.926 and 2.310, identification tasks are between logit 1.186 and 0.191, and items requiring synthesis and deletion are located below this value. An item calling for a single identification is separated from the item type requiring the same operation by a −1.010 logit value. The level of difficulty of the tasks is therefore independent of the size of the language unit according to the operations. The analysis drew attention to the fact that the tasks were separated on the level of operations, where segmentation tasks were the most difficult parts of the test.

The correlation matrix (see Table 2) of the subtests shows the internal structure of the test. The correlation values between the subtests are mostly weak. There is a negligible correlation between the segmentation subtests (syllable and phoneme segmentation). The highest correlation value was found for the relationship between the phoneme deletion and phoneme synthesis subtests. The phoneme identification subtest showed a significant correlation with all the syllable subtests.

Table 2 Correlations of the subtests
Full size table

Confirmatory factor analysis was used to further explore the validity of the main measurement model underlying the phonological awareness test. Five models were tested (see Table 3): the 1-dimensional model combines all the possible dimensions under one general factor; the 2-dimensional model reflects the size of the linguistic units (syllable and phonemic level) being manipulated; the 4-dimensional model corresponds to the nature of the cognitive aspects/operations applied (analysis, synthesis, segmentation and deletion); the seven-dimensional model is in line with the theoretical test structure separating each subtest into different groups (phoneme identification, deletion, segmentation, synthesis and syllable segmentation, deletion and synthesis); and the 9-dimensional model puts all task types tied to phoneme identification into different factors. The 9-dimensional model proved to be suitable (χ2 = 1088.60; p < 0.001; CFI = 0.905; TLI = 0.901; RMSEA = 0.027).

Table 3 Goodness of fit indices for testing the dimensionality of phonological awareness
Full size table

Three nine-dimensional theoretical models were tested to identify the internal construction of the test. Model 1 shows that all the measured subtests connect directly to phonological awareness. The main organisational component in model 2 is the size of the linguistic units, and in model 3 it is the cognitive operations (see Table 4). The path diagram for model 3 is depicted in Fig. 4. The model builds from items to the latent variable of phonological awareness. Observing Fig. 4, we can see that the model is built from test items (44 items), which are categorized into subtests based on language levels (syllable and phoneme) and specific operations. Out of the nine subtests, two are related to synthesis, two to segmentation, two to deletion, and three contribute to the identification latent variable. These four latent variables are directly connected to the latent construct of phonological awareness. Although the value of the factor weights associated with the latent construct of phonological awareness in the model is acceptable, the latent component of segmentation is distinguished from the value of the other three components by a negative factor weight.

Table 4 Goodness of fit indices for testing the internal construct of phonological awareness
Full size table
Fig. 4
figure 4

Model 3, The dimensionality of phonological awareness/PA phonological awareness, S_syn syllable synthesis, S_seg syllable segmentation, S_del syllable deletion, Ph_IPh identification of initial phonemes, Ph_Seg phoneme segmentation, Ph_del phoneme deletion, Ph_Synphoneme synthesis, Ph_Id phoneme idetification in different sound environments, Ph_pos Identification of the position of phoneme, Syn synthesis, Seg segmentation, Del deletion, Id identification

Full size image

In order to examine the media effect, we used regression analysis to show the explained variance, which determines the relationship between the use of computers and phonological awareness tests. In kindergarten, the transmitting medium was a touchscreen tablet computer; therefore, the mouse was not used as an output device attached to the computer. The explained variance between the total test and the tablet-based test was 5.3% (F (1.315) = 17.63, p < 0.01). In the case of drag-and-drop tasks, the correlation was not significant (F (1.310) = 0.056, p > 0.05) and the explained variance was negligible. In the case of tapping tasks, the variance was 6.25% (F (1.312) = 20.23, p < 0.01) (see Table 5).

Table 5 The connection between the tablet-based test and the phonological awareness test
Full size table

Discussion

A great deal of research at both the national and international levels with a large number and variety of measurement tools, both online and face-to-face, has studied children’s performance in phonological awareness and identified the relationship between the measured subtests. However, these instruments show a number of differences within the types of tasks and within their assessment and test recording mode. There have only been a few studies which measure phonological awareness separately, not as one part of a test battery (e.g. Dynamic Indicators of Basic Early Literacy Skills (DIBELS) (2018) which can detect risk and monitor early literacy and reading skills in kindergarten through eighth grade). Other problems are that there are few measurement tools available for free, with most of them not supported by empirical studies and/or significant data. Therefore, the aim of our study was to develop an online measurement tool covering several sub-areas of phonological awareness, which could become an objective and reliable instrument for kindergartener, and to explore the applicability of the measurement instrument which has been developed in educational practice.

Regarding to RQ1, according to IRT analysis, the distribution of the difficulty level of the tasks was separated on the level of operations, not on the size of the language unit. As the data show, segmentation tasks proved to be the most difficult part of the test. Following Wagner and Torgesen (1987), phoneme awareness tasks can be grouped into three categories: (1) sound comparison tasks, (2) phoneme segmentation tasks, and (3) phoneme blending tasks. The sound comparison tasks are the easiest phonemic awareness tasks, while the other two groups already assume a higher level of manipulation skills, so these can be considered as more difficult, complex tasks in the sensitive phase of the kindergarten-school transition. These tasks are not used in the development of Hungarian kindergartens, their practice can be placed in the first year of primary school, although the spontaneous development of certain partial abilities can be measured already in kindergarten and their predictive power is decisive (Rathvon, 2004). The online measurement tool's subtests which require manipulation, are more difficult for children. Of these, one segmentation task proved to be the most difficult. Segmentation tasks can be considered as closed tasks in the online system we use, the child not only had to decide how many to applaud, how many to divide the word, but also had to count and click on the corresponding number of petals. Thus, the child's mathematical and counting skills probably played a significant role in the task. It is worthwhile to correlate this task with the children’s mathematical ability later on and observe whether there is indeed a correlation between the segmentation task and the child’s mathematical ability. The correlation values are weak; however, concerning the task types and the entire test, phoneme identification with determining the initial sound of the word (r = 0.77, p < 0.05), as well as identification of the initial sound (r = 0.65, p < 0.05), and phoneme synthesis (r = 0.66, p < 0.05), show the strongest associations with the overall test. In terms of the relationships between the operations and the entire test, except for segmentation, strong correlations are found in all cases (identification, synthesis, and deletion) (r = 0.71–0.88, p < 0.01).

The results of the confirmatory factor analysis (RQ2), where several models were tested, based on relevant empirical studies (Høien et al, 1995; Stahl & Murray, 1994; Stanovich et al., 1984; Yopp, 1988) underlined a new aspect, where it is not the linguistic unit that is the main determining factor in the structure of our phonological awareness assessment tool. At this point, the research is essential. In Farral’s theoretical summary (2012), we find that an older child can manipulate even smaller linguistic units, reaching a beginning sound level at the end of kindergarten. The results of these studies demonstrate that by the end of kindergarten, when the child is presumably already able to identify the first sound in a word, the operation level is the main directing principle within the construct of the phonological awareness test.

RQ3 focuses on the relationship between tablet using skills and the performance on the phonological awareness test. In the case of kindergarten children, the media effect is particularly important because the mediating medium is different: face-to-face testing has been replaced by online assessment. The low score of explained variance between phonological awareness and tablet-based tests is negligible. In the case of the kindergarten test, the mediator medium was the touch screen tablets. The mouse was therefore not a transmitter. Because children manipulated their fingers, we did not find a significant correlation between drag-and-drop tasks.

All analyses highlighted that the segmentation tasks posed difficulties; however, due to the small amount of explained variance, it cannot be assumed that this is related to the children's tablet usage skills. The closed nature of the segmentation tasks requires calculation skills from the students; however, the present test does not include tasks suitable for measuring mathematical abilities. Rausch and Pásztor (2017) found that the level of explained variance between tests can be influenced by additional factors. In their research, they examined the preconditions for learning mathematics in an academic readiness test, which was preceded by a computer skills test and a face-to-face version of the DIFER test for elementary numerical skills. Although their main goal was to validate the online mathematics test, they also found a correlation between the face-to-face test and the computer skills test. Therefore, they suggested that other factors may play a role in exploring the level of explained variance during the analysis of test interrelations. Another option is to explore the development methodology for the population under examination following Carlisle (1991) finding that child performance on onset-rime-type tasks can be better defined by type of task than actual language knowledge. This idea should definitely be integrated into practice and examined to ascertain what kinds of exercise tasks occurred during the sessions before the testing period.

In sum, the online phonological awareness test is a new form of approach compared to the measurement method used in Hungary in recent decades. Compared to the previous measurement methodology, the online test not only shortened and makes the measurement practice more objective and facilitates the work of the educators, but also expands with new types of subtests and test items, new components of phonological awareness. The current methodological repertoire expands with the examination of the operational level of phonological awareness, the examination and future development based on the examination can better establish those skills, which are the basic pillar of reading and writing.

The Hungarian measurement and development system develops along with the size of the language units, followed first by the practice of syllable-level and then phoneme-level operations. And when performing tasks with sounds, at the preschool level, it is limited to identifying and differentiating them, beyond which the present assessment tool exceeds.

The construction of the present online test highlights the role of operations independently of the size of the linguistic element. These results—both in terms of the measurement and development process—can turn kindergarten teacher’s attention to the importance of the operation of linguistic units independently of their size. This finding may point to a new developmental, methodological line in kindergarteners’ language improvement toolkits and can draw kindergarten teachers’ attention to task type and type of manipulation instead of size of linguistic units. Online assessment could also be a challenge to convince kindergarten teachers and this innovative testing mode and method can reduce the time in which children are assessed and provide them immediate, objective feedback. According to Flewitt et al. (2015), iPad-based literacy exercises can improve children’s motivation and concentration. This new way of testing offers an opportunity to create a new, technology-based curriculum.

The purpose of the online test is on the one hand, to draw attention to the raison d'être of online testing and the fact that the measurement and development of phonological awareness cannot be separated according to the size of the language unit, thus encouraging teachers to rethink their development methods. On the other hand, it draws attention to the fact that a new kind of methodological tool is available to educators that they can use in daily practice. The tool is tailored to the needs of children but also can be a challenge for educators with lower ICT literacy. Although digital competence is an expectation among kindergarten teachers, the new type of measurement may affect them as strangers, therefore they may need methodological support. With this form of measurement, we want to offer a new opportunity for Hungarian kindergartens.

Limitations of the Study

The test version measures only one aspect of phonological awareness; adaptation of rhythm-related tasks, for example, appears to be a missing area. Moyle et al. (2013), who measured rhyming knowledge via rhyme and nursery rhyme awareness tasks, found that children who had practice in this area had stronger vocabulary skills.

In order to explore the additional factors underlying children’s abilities, it is essential to use a parents’ background questionnaire, as socio-economic status and family background can significantly determine children’s performance and the success of later achievements (Agirregoikoa, et al., 2021; Borre et al., 2019; Burris et al., 2019). A close family member with reading difficulties can also influence a child’s development (Leavett et al., 2014). Further development of the test may also include an adaptation of a hearing test in an online system (Nassrallah et al., 2018).

Individual task types appear in a small number of items, which can be used to describe the performance of individual subtests but are not suitable for exploring deeper relationships. In this very sensitive phase of the kindergarten–school transition, it would also be important to ascertain the importance and relationship of aspects of the phoneme, the phoneme’s position and the quality of the sound. However, expanding the tasks from this aspect and thus increasing the number of items would place a heavy burden on the children being examined. Adaptive testing can provide a solution, which can be based on the results of the person-item map obtained in previous studies.