A research of analysing the effectiveness of speaking-pen on English learning in consideration of individual differences using a linear mixed-effect model

The Ministry of Education, Culture, Sports, Science and Technology in Japan (2011) started the ‘Foreign Language Activities’ in fifth grade based on the new Courses of Study in April of 2011. The ministry has considered beginning this course in third grade and the ‘English Course’ in 5th grade in order to improve students’ reading, writing, listening, and speaking skills. The ministry also developed supplemental teaching instruments, such as the CALL system. The purpose of this study is to model and analyse the effectiveness of a speaking-pen on English learning among elementary school children in consideration of individual differences using a Liner Mixed-Effect Model. The authors constructed models representing students’ overall abilities in four English skills, and analysed the effectiveness of the tools such as a speaking-pen and an audio CD on English learning in consideration of students’ backgrounds including their English learning experiences and individual differences.


Background of this study
In an era of globalization, the Ministry of Education, Culture, Sports, Science and Technology in Japan (MEXT 2011) began implementing 'Foreign Language Activities' as a compulsory class beginning in fifth grade, which is based on the new courses of study that were introduced in April 2011. The 'Foreign Language Activities' aim to familiarize children with English by focusing on intonation and pronunciation, listening and speaking. The MEXT (2014) has considered beginning 'Foreign Language Activities' in third grade which aims to improve 'listening' and 'speaking' skills, and implementing an 'English Course' in fifth grade, which aims to not only improve 'listening' and 'speaking' skills, but also 'reading' and 'writing' skills.
They have also proposed adopting effective ICT materials in order to help children recognize alphabetical letters and notice differences in intonation, characteristics, and structure between Japanese and English as a guide for the teaching support material. Many researchers study and survey teaching support materials used for teaching elementary school children.
Although many studies on tools used for early English education have been conducted, there are few studies on English education that have analysed longitudinal learning data of children using tools and modelled the effectiveness of the tools in consideration of individual differences.

Review of previous studies related with the use of technology in English language learning and modelling the educational effectiveness in consideration of individual differences using the Linear Mixed-Effect Model
According to Pourhosein Gilakjani (2017), technology assists learners in adjusting their own learning process and they can have access to a lot of information that their teachers are not able to provide. Parvin and Salam (2015) carried out a study and declared that by using technology, learners get the chance to increase their exposure to language in a meaningful context and make their own knowledge. Pourhosein Gilakjani (2014) maintained that using technology can create a learning atmosphere centered around the learner rather than the teacher that in turn creates positive changes.
However, their papers did not analyse longitudinal learning data of children using tools and model the effectiveness of the tools in consideration of individual differences.
On the other hand, there are many educational studies that have used the Linear Mixed-Effect Model (LME) in order to take individual differences into account in the modelling of the educational effectiveness. In Japan, Kawaguchi (2009) propose to use a LME to analyse school effects. His models include school level variables and children level variables as fixed effect and random effect variables accompanying fixed effects. In other countries, Xu, Yuan, Xu, and Xu (2014) analysed Chinese high school students' time management with regard to their math homework using the LME. Their models depict class level and children level variables such as 'Motivation', 'Arranging environment', 'Family homework help', and 'Gender', and they analysed these factors in detail. Hsu and Kuan (2013) explore the factors that influence the elementary or junior high school teacher ICT integration by analysing a detailed model according to the level of schools and teachers in Taiwan. Roman and Murillo (2012) used the model to analyse achievements in math and language of the elementary school in Latin America according to factors such as country level, school level and family socio-economic level. Kwok, Lai, Tong, Lara-Alecio, Irby, Yoon and Yeh (2018) analysed complex longitudinal data of project of English Language and Literacy Acquisition (ELLA) in educational research.
Although many educational studies using the LME have been conducted, there are few studies that have analysed longitudinal data according to detailed modelling of individual differences of children, for students' detailed English educational experience.

The purpose of the study
Therefore, the purpose of this study is to analyse longitudinal data in order to propose modelling the effectiveness of a Speaking-pen in support of four English skills (reading, writing, listening, and speaking) in consideration of individual differences depending on the detailed experiences of English learning. The learning that was conducted during the investigation utilized two tools; namely, a speaking-pen and an audio CD, which were used to enhance English learning. Using a speaking-pen, children can learn English in a similar way as one uses a pencil, without prior knowledge of, and preparation for, PC. This method differs from learning using CALL materials, which are generally used for English education. The speaking-pen adopted in this study has an extraordinary function. It can record and play back users' voices in addition to its conventional function, in which English pronunciations are already recorded. Users can compare between their own voices and English pronunciations already recorded (the speaking-pen was made by Gridmark Inc.).
In section 2, the investigation method of this study is explained. In subsections 2.1, 2.2, and 2.3, we describe the experimental design, the construction of the test, and the construction of textbook, respectively. In section 3, we propose modelling the effectiveness of a speaking-pen in support of four English skills in consideration of individual differences depending on the detailed experiences of English learning. In section 3.1, the model of the total score is shown, and in section 3.2, the model of each of four skills is shown. In section 4, the results of the effectiveness of speaking-pen based on each model of total score and four skills are shown and discussions are presented.
2 Investigation method and constructions of implemented test and textbook

Investigation method
The research for this study was conducted from October 2013 to March 2014. Ninety second-grade private school students at Shukutoku elementary school participated in this study with the consent of their guardians. In this school, children learn English twice a week beginning in first grade. They learn it by focusing on conversation skills with a native English teacher. Therefore, they have high proficiency in spoken English, which increases their motivation to continue learning. This differs from children in general public elementary schools. In this study, a two-period (2 × 2) cross-over design was adopted as the experimental design. The children were divided into two groups in which both groups were able to use both a speaking-pen and an audio CD in different periods. A pre-questionnaire was implemented before the research experiment began. The pre-questionnaire included some items for investigating the children's English learning background, which can be found in Chapter 3. Based on the pre-questionnaire, the children's responses were categorized into eight categories in accordance with the results of three categories of responses, 'Experience of English Learning', 'Practice of Home Learning' and 'Experience of Using a Speaking-Pen'. The children in each group were allocated to two groups using a Bernoulli trial, in which the probability of success was set at 0.5 as the Bernoulli probability parameter, so that there would be no differences among the children's background between the two groups. As a result, 45 children were assigned to Group 1 and the other 45 children were assigned to Group 2 (Fig. 1).
The children in Group 1 learned in their home using a speaking-pen during the first six weeks of the investigation (the first period), and using an audio CD during the last six weeks (the second period). Both six-week sessions were separated by a four-week long inactive term. The children in Group 2 learned in their home using an audio CD during the first period and using a speaking-pen during the second period. In terms of home learning during the investigation, the children were not forced to use either a speaking-pen or an audio CD. Their learning conditions, which included frequency, time, and the means of use depended on their own independence of will and volition to learn. There are four achievement tests that measured the initial skills or improvements in their learning. The first test was implemented before the first period, the second test was implemented after the first period, the third test was implemented before the second period, and the fourth test was implemented after the second period. A postquestionnaire was administered to the children after the research experiment was complete. The post-questionnaire included some items for investigating timing, frequency and the hours of use. This can be seen in Chapter 3.

Construction of achievement test
The construction of four tests looked similar to each other. This paper cites the second test in the explanation. All of the tests are composed of six sections, and their total scores add up to 100. In terms of the first test, we referred to the previous study conducted by Tsubaki, Gonda, Kato and Maeda (2015). In Part 1 of the test, after the children read the spelling of a word and see an accompanying picture, they connect the word and the pictures with a line. This is considered to be an appropriate test for measuring a child's reading ability. This section has fifteen questions and one point is given per one accurate combination so that fifteen possible points can be earned in Fig. 1 Allocation method total. In Part 2 of the test, after the children see a picture, they fill in the blank with one letter for each question. This is considered an appropriate test for measuring a child's writing ability. This section has ten questions, and two points are given for each correct answer. In Part 3 of the test, after the children read a question and see a picture, they choose the correct answer. This is considered to be an appropriate test for measuring a child's reading ability. This section has five questions, and two points are given to each correct answer. In Part 4 of the test, after the children hear a question, they choose an appropriate answer sentence. This is considered an appropriate measurement of a child's listening ability. This section has five questions, and four points are given to each correct answer. In Part 5 of the test, after the children listen to a sentence that contains one blank in the place of a missing word, they fill in the blank with a letter. This is considered an appropriate measurement of a child's listening and writing abilities. This section has five questions, and four points are given to each correct answer. In Part 6 of the test, a native English teacher asks each child three questions in English, and each child answers the question in English. This is considered an appropriate measurement of a child's speaking ability. Some examples of questions are, 'Is this a dog?' (accompanied by a picture of a dog); 'What colour is this?' (accompanied by a picture of a yellow cat); 'What's this?' (accompanied by a picture of an umbrella). (Fig. 3).

Construction of textbook
The textbook is composed of four units, and each unit is composed of seven sections. In the first period, the children study from the first and second units of the textbook, and in the second period, the children study from the third and fourth units. In this section, we refer to the second unit in order to describe the components of the textbook. One may refer to the study by Tsubaki et al. (2015) for a further understanding of the first unit. The components of the four units in the textbook are similar.
In Section 1, the children learn basic conversational phrases that align with the theme of the unit. The children read and listen to conversational questions and their corresponding answers, such as 'What colour is this?'; followed by the response: 'It's blue.' They can compare their pronunciation with the native English speaker's when they use the speaking-pen to record their pronunciation.
In Section 2, the children learn a set of words that corresponds with the theme of this unit. The theme of the second unit is colour. The children learn pronunciations of colour words, such as 'red' and 'yellow'. The themes of the other units are animals, Fig. 2 Experimental design of investigation food, and a birthday party. In this section, children can practice their pronunciation as they did in Section 1.
In Section 3, the children can listen to question sentences that correspond with the theme of the unit and choose correct answers after seeing a set of pictures. In this unit, the children can learn the words of colours by listening to their names. In the other units, the children can learn their numbers, as well as how to answer 'yes' or 'no'.
In Section 4, after the children listen to a word, they use the speaking-pen to spell the word. This section is only available to the children who use the speaking-pen. For example, the children in Group 1 can practice spelling words during the first period, and the children in Group 2 can practice spelling words during the second period.
In Section 5, after the children see pictures of objects and listen to their corresponding names, they can practice writing the correct spelling of the words.
In Section 6, after the children read question sentences and listen to questions using a speaking-pen, they can practice choosing correct answers. This section contains two questions and is only available to the children who use the speaking-pen.
In Section 7, after the children listen to a group of words that align with the theme of this section, they can practice spelling the words. In terms of using the speaking-pen, they can learn the correct pronunciations of words by comparing their own pronunciations with those of a native English speaker.
The Common European Framework of Reference for Languages (CEFR) is a set of guidelines used to describe the achievements of students of foreign languages throughout Europe. In Japan, CEFR-J is based on CEFR, has been proposed by Touno et al. (2010Touno et al. ( , 2012a, and was adjusted for Japanese English learners. CEFR-J descriptions corresponding to each unit of the test and text in this study are shown in Table 1. In the first column, reading, writing, and listening are denoted as R, W, and L, respectively.

Fig. 3 Construction of achievement test
Speaking is divided into two ability categories. S1 refers to 'Spoken relationship', and S2 refers to 'Spoken production'. In the second column, proficiency levels are arranged in numerical order. For example, if we consider listening, PreA1 corresponds to 'perception of pronunciation with which Japanese are familiar as a katakana word (loan word)'; A1.1 corresponds to 'greeting, name, date, day of the week, numbers, words, and expressions which are used in daily life'; A1.2 corresponds to 'words, short sentences, question, familiar and personal requests and preferences (like or dislike, route guiding, etc.)'; and A1.3 corresponds to 'informal speech in daily conversation (personal questions, daily instructions, requests, etc.)'. In the third and fourth column, the numbers correspond to the section number in the test or textbook respectively.
To illustrate this table, the sections of the test and text principally focus on the A1 level of 'a beginner who just began learning English'.

Modelling the effectiveness of speaking-pen in consideration of individual differences using a linear mixed-effect model
In this section, we construct and propose models that can analyse the effectiveness of learning based on variables given in the pre-and post-questionnaire data, variables of time and variables of tools, such as the speaking-pen and audio CD.
In this study, we are interested in the effect of child i, the effect of time j, the effect of tool k, and the interaction effect between time j and tool k for the test score. Then, we model the test score of the child i at time j with tool k (y ij(k) ) by the parameter δ i of each child i, the effect β j of time j, the effect γ k of the tool k, the interaction βγ jk between time j and tool k., and the error ε ij(k) at the first part of Table 2.
Further, we are interested in the effect of gender and the fixed effect depending on the experiences of English learning for the parameter δ i of each child i, then we model the parameter δ i of each child i by the parameter μ of 'Mean over individual,' the fixed effect α 1 of 'Gender,' the fixed effect depending on the experiences of English learning (like the fixed effect α 2 of 'Private English School'((1) in Table 3), the fixed effect α 3 of 'Tutor' ((2) in Table 3), the fixed effect α 4 of 'kindergarten with English Lesson' ((4) in Table 3), the fixed effect α 5 of 'Parents Speaking English Very Well' ((5) in Table 3), the fixed effect α 6 of 'Speaking-pen Experiences' ((6) in Table 3), the fixed effect α 7 of 'Home Learning' ((7) in Table 3), the fixed effect α 8 of 'Homework from Private English School' ((8) in Table 3), fixed effect α 9 of 'Favour' ((9) in Table 3),) and the parameter ω i of 'Individual Differences' at the second part of Table 2. A prequestionnaire was implemented before the research experiment began. The prequestionnaire included items for investigating the children's English learning background, which can be found in Table 3. We show above the correspondence between the fixed effect α m and pre-questionnaire item number in Table 3. We are interested in the effects of their children's English learning backgrounds of the test scores.
And also, we model the effect β j of time j by the parameter π j of 'Mean of Time j,' and the interactions 'Gender effect at Time j' α 1j , the interaction effect between time j and variables depending on the experiences of English learning (α 2j -α 8j ) at the third part of Table 2. Furthermore, the effect γ k of tool k is modelled by the parameter ο k of 'Mean of Tool k,' 'Gender effect using Tool k' α 1k , the interaction effect between tool k and variables depending on the experiences of English learning (α 2k -α 8k ) at the forth part of Table 2.
Finally, we model the interaction effect βγ jk by the 'Mean of Time j × Tool k' parameter ξ jk , the fixed effect of 'Gender effect in Time j × Tool k' α 1jk , the interaction effect between time j, tool k and variables depending on the experiences of English learning (α 2jk , − α 10jk ) at the last part of Table 2. A post-questionnaire was administered to the children after the research experiment was complete. The post-questionnaire included some items for investigating timing, frequency and the hours of use. This can be seen in Table 4. The fixed effect α 10jk shows the interaction among Frequency×Time j × Tool k in Table 2. We are interested in these interactions.
And then, we analysed the variables using a one-way analysis of variance (one-way ANOVA) in order to choose effective variables among all 69 variables. In the one-way ANOVA, we set the variables of the total test scores and the four English skills' scores on the first test as response variables. We adopted the variables in which clear trends were observed significantly, and used them to construct Linear Mixed-Effect Models. Furthermore, in the first period, the improvements of each child's score were calculated by subtracting the scores of the first test from the scores of the second test. In the second period, the improvements of each child's score were calculated by subtracting their scores from the third test from the scores of the fourth test. The variables chosen using a one-way ANOVA were included in models as the effect of time, tools, and interactions between time and tools in the sections 3.1-3.2. Table 2 represents the variables and parameters of five models (Total score model and four English skills models) proposed in this section.

Total score modeling
In this section, we propose a Model (T) of the total score.
The total score is modelled as follows: 2 I can convey simple information (e.g. times, dates, places), using basic phrases and formulaic expressions. Interaction between Gender × Time j j(4 levels) × Gender (2 levels) = 8 levels Interaction among Frequency × Time j × Tool k 0: seldom (He/She seldom used pen or CD.) 1: once / two weeks (He/She used pen or CD once every two weeks.) 2: once / week (He/She used pen or CD once a week.) 3: twicefour times / week (He/She used pen or CD two to four times a week.) 4: fiveseven times / week (He/She used pen or CD five to seven times a week.) y ij(k) is the total test score of the child i at time j with tool k. δ i is defined as the parameter of each child i, β j as the effect of time, γ k as the effect of the tool, and βγ jk as the interaction between time j and tool k. ε ij(k) is the error. Furthermore, the parameter δ i of each child i in Eq. (1) is modelled by the variables whose clear trends were observed significantly using a one-way ANOVA of the total score of the first test. (2) Do you learn from a tutor or from your parents? Have you ever learned from a tutor or from your parents?
(3) Have you ever lived abroad?
(4) Have you ever learned from an English teacher in kindergarten?
(5) Do you often hear fluent English spoken by your parents?
(6) Do you usually use a speaking-pen (which makes sounds when you push the pen point)?
Have you ever used a speaking-pen?  (2) How often did you use Speaking-pen or an Audio CD? a. once a week b. two to four times a week c. five to seven times a week d. once in two weeks e. I seldom used it.
(3) How long did you use a Speaking-pen or an Audio CD every time you used it? a. about 10 min b. about half an hour c. about an hour d. over an hour e. I seldom used it.
The parameter μ is defined as 'Mean over individual', α 1 as fixed effect of 'Gender', α 2 as fixed effect of 'Private English School', α 3 as fixed effect of 'Tutor', α 6 as fixed effect of 'Speaking-pen Experiences', α 7 as fixed effect of 'Home Learning', α 8 as fixed effect of 'Homework from Private English School' and α 9 as fixed effect of 'Favour'. The parameter ω i expresses 'Individual Differences' and is assumed to be ω i ∼N 0; σ 2 δ À Á . The effect β j of time j in Eq. (1) is modelled by the variables whose clear trends were observed significantly using a one-way ANOVA of the total improvements in test scores.
The parameter π j is defined as 'Mean of Time j' and α 1j as 'Gender × Time j', which means the fixed effect of 'Gender effect at Time j'.
The effect γ k of tool k in Eq.
(1) is modelled by the variables whose clear trend was observed significantly using a one-way ANOVA of the total test score improvement.
The parameter ο k is defined as 'Mean of Tool k' and α 1k as 'Gender × Tool k', which means the fixed effect of 'Gender effect using Tool k'. The effect βγ jk in Eq. (1) is modelled by the variables whose clear trends were observed significantly using a one-way ANOVA of the total improvements in test scores.
The result of the one-way ANOVA showed that 'Frequency' was significant regarding the improvement of first period students, and 'Gender' was significant regarding the improvement of second period students. We considered this to be the result of interactions between these variables and the effect of time. Accordingly, these interactions are included in the model.
The parameterξ jk is defined as 'Mean of Time j × Tool k', α 1jk as 'Gender × Time j × Tool k', which means the fixed effect of 'Gender effect in Time j × Tool k', and α 10jk as 'Frequency × Time j × Tool k', which means the fixed effect of 'Frequency effect in Time j × Tool k'.

Four English skills Modelling
In this section, we propose the models of four English skills. The four models are constructed in the same way as Model (T).
(1) The Model of reading score, Model (R) Reading score is modelled using parameters in Table 2 as follows: (2) The Model of writing score, Model (W) Writing score is modelled using parameters in Table 2 as follows: (3) The Model of listening score, Model (L) Listening score is modelled using parameters in Table 2 as follows: (4) The Model of speaking score, Model (S) Speaking score is modelled using parameters in Table 2 as follows: Model (R), Model (W), Model (L), and Model (S) are adopted among all the possible models regarding all combinations of variables chosen using a one-way ANOVA. Comparing all the possible models' value of AIC, the models that had the minimum value of AIC were adopted.

Discussions for the models
The variables and parameters composing all models indicated in Section 3.1 and 3.2 are shown in Table 5. The variables and parameters included in the models are represented as '1' and those not included in the models are left blank.
In all the five models, the effects of 'Private English School' α 2 , 'Tutor' α 3 and 'Favour' α 9 were modelled in the parameter δ i as fixed effects. The distinct feature of Model (T) modelled the interactions between the effect of 'Gender' and 'Time' α 1j , 'Tool' α 1k , 'Time × Tool' α 1jk respectively. In addition, the effect of 'Frequency' α 10jk affects Model (T). The distinct feature of Model (R) modelled the interactions between the effects of the children's backgrounds, such as 'Parents' (α 5j , α 5k , α 5jk ), 'Home Learning' (α 7j , α 7k , α 7jk ), and 'Homework of Private English School' (α 8j , α 8k , α 8jk ), and 'Time', 'Tool', 'Time × Tool' respectively. The effects of Model (W) were fewest compared to all the other models. As can be seen in Table 1, it seems that Model (W) became the simplest model of all the models since the level of writing in the textbook was only PreA1. The distinct features of Model (L) and Model (S) modelled the interactions between the effects of 'Private English School' and 'Time' α 2j , 'Tool' α 2k , 'Time × Tool' α 2jk respectively. It is noted that the interactions (α 1j , α 1k , α 1jk ) between 'Gender' were modelled in Model (L) and Model(W), while these interactions were not modelled in Model (S).

Results and discussions for all the models
An analysis of this study was carried out in the restricted maximum likelihood estimation using the MIXED PROCEDURE of SAS. One of the levels of each variable was assumed to be 0 in order to estimate values of the other levels. For example, for estimating the variable 'gender', the estimated value of 'girl' is estimated on condition that the estimated value of 'boy' (level 0) is set at 0. Table 6 represents the estimated values of each model that were significant. In Table 6 Table 7 shows the mark of the p values in Table 6.

Results of model (T)
The interaction (8.781) of 'Time (j = 2) × Tool (k = 2: speaking-pen)' was significantly positive. The results show that the children who learned using a speaking-pen during the first period were, on average, able to improve their own overall ability more than the children who used an audio CD. It seems that the speaking-pen is more effective than an audio CD in the early stage of English learning.
In terms of variances, the estimated value of the variance of 'Individual Differences' (283.92) was larger than 'Error '(86.42). It seems that the individual difference was large as a factor of a variance in Model (T).

Results of model (R)
The effect (3.719) of 'Frequency (five-seven times / week)' on the interaction between 'Time (j = 1) × Tool (k = 2: speaking-pen)' was significantly positive. The result shows that the children who used a speaking-pen five to seven times a week tended to get a high score on the first test more than those who seldom used a speaking-pen.
In terms of variances, the estimated value of the variance of 'Individual Differences'(24.43) was larger than 'Error '(12.39). It seems that the individual difference was a significant factor of a variance in Model (R).

Results of model (W)
A few significant effects were observed in Model (W) compared to the other models. It seems that the structure of writing is simpler than the other models. Any significant effect of speaking-pen was not observed more than an audio CD.
In terms of variances, the estimated value of the variance of 'Individual Differences'(42.14) was larger than 'Error '(19.07). It seems that the individual difference was large as a factor of a variance in Model (W).

Results of model (L)
The interaction (3.598) between 'Time (j = 2)' and 'Tool (k = 2: speaking-pen)' was significantly positive. The listening abilities of the children who used a speaking-pen during the first period tended to increase more than the abilities of the children who used an audio CD. The speaking-pen provides the children with an opportunity to listen to the same words repeatedly, while using an audio CD requires one to listen to the entire track. Thus, children can listen to pronunciations slowly and clearly using speaking-pen, which may account for these results.
The effect (−6.667) of 'Gender (girl)' in the interaction between 'Time (j = 2)' and 'Tool (k = 2: speaking-pen)' was significantly negative. Therefore, the boys were able to improve their own listening ability more than the girls in the first period.
In terms of variances, both of the estimated values, 'Individual Differences' (14.13) and 'Error'(13.40), were at the same level. It seems that individual difference (14.13) was relatively small in listening since the tools focused on improving listening ability.

Results of model (S)
The interactions between 'Tool (k = 2k = 2: speaking-pen)', 'Private English School (past)' and each of the 'Time (j = 2)'(9.441), 'Time (j = 3)'(6.601), and 'Time (j = 4)'(5.455) factors were significantly positive. It seems that the children could remember knowledge or experiences when they learned at a private English school.   In terms of variances, the estimated value of the variance of 'Individual Differences' (4.38) was smaller than 'Error' (7.67). It seems that the children could consistently acquire speaking skills using tools such as a speaking-pen and an audio CD.

Summary of the study
In this study, with the era of the effects of globalization, we modelled and analysed the effects of learning tools such as a speaking-pen and an audio CD, in consideration of the children's backgrounds and individual differences, using the Linear Mixed-Effect Model in order to investigate how the children's experiences of English learning affect their potential English skills and improve their learning.
In section 1, we revealed that it was important for Japanese to adopt ICT tools in order to learn four skills (reading, writing, listening, speaking) related to English learning, because it is an era of globalization. The purpose of the study was explained after reviews of previous studies related to the use of technology in English education.
In section 2, the investigation method was illustrated and detailed explanations of the test and the textbook were provided based on the four English skills to be assessed.
In section 3, we modelled the effects of the children's backgrounds based on the prequestionnaire and 'Time' and 'Tool' by using a Linear Mixed-Effect Model after the effective variables, whose clear trends were observed using a one-way analysis of variance, were selected.
In section 4, the estimated values of each model were represented and the discussion of the significant interaction effects between the use of speaking-pen and the children's background, and 'Time' were provided.
We found these four significant results and they are an indication that students studied more with the pen.
The interaction (8.781) of 'Time (j = 2) × Tool (k = 2: speaking-pen)' in Model (T) and the interaction (3.598) of 'Time (j = 2) × Tool (k = 2: speakingpen)' in Model (L) was significantly positive. The results show that a speakingpen was effective for improving overall skills and particularly listening ability in the first period.
In addition, it should be noted that the differences of effects depending on children's individual backgrounds were observed in each skill. In terms of reading, the effect (3.719) of 'Frequency (five-seven times / week)' on the interaction between 'Time (j = 1) × Tool (k = 2: speaking-pen)' was significantly positive. The result shows that the children who used a speaking-pen five to seven times a week tended to get a high score on the first test more than those who seldom used a speaking-pen. In terms of listening, the speaking-pen was effective for boys who had learned in the first period, because the effect (−6.667) of 'Gender (girl)' in the interaction between 'Time (j = 2)' and 'Tool (k = 2: speaking-pen)' was significantly negative.
In terms of speaking, the speaking-pen was effective for the children who had previously learned at a private English school, because the interactions between 'Tool (k= 2: speaking-pen)', 'Private English School (past)' and each of the 'Time (j = 2)'(9.441), 'Time (j = 3)'(6.601), and 'Time (j = 4)'(5.455) factors were significantly positive.
In terms of variances, the individual differences of the total model (283.92), reading model (24.43), and writing model (42.14) were larger than the error variances, whereas the individual differences of the listening model (14.13) and speaking model (4.38) were smaller than the error variances. It seems that the tools, such as the speaking-pen and the audio CD, provide stable effects on listening and speaking.
Compliance with ethical standards To the best of our knowledge, the named authors have no conflict of interest, financial or otherwise.
This research involves human participants. Informed consent has been obtained from all parents of pupils included in this study.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.