Keywords

The main purpose of this research was to develop a psychometric framework to assess the relationship between parental involvement and reading literacy. The framework incorporates country specific differences, both at the item level and the scale level, to gain insight into cultural differences in the parental involvement component and its relation to student achievement in reading literacy. We conducted secondary analyses on the PIRLS-2011 data of 41 countries. A review of the research literature distinguished four dimensions of parental involvement: (1) home-based involvement from a parent perspective; (2) school-based involvement and home-school communication from a parent perspective; (3) home-based involvement from a student perspective; and (4) school-based involvement and home-school communication from a school perspective. Based on items available in the PIRLS data, the first dimension was split in two components: early literacy activities and helping with homework. IRT analyses provided item-by-country interactions indicating CDIF. The five components were first modeled using the unidimensional GPCM. Using these analyses, potential items with CDIF were identified and subsequently modeled using country-specific parameters for the 10 and 20 % most extreme interactions. These methods for identifying and modeling CDIF were compared with two other models. The first was the GPCM with random item parameters, where the variance of the parameters across countries provided an indication of possible CDIF. The second was a bi-factor GPCM where a country-specific covariance matrix gave an indication of the extent to which the scale loaded on the intended latent variable and the extent of loading on a country-specific dimension. Finally, multilevel analyses were conducted to explore the association between parental involvement and student achievement for all countries that participated in PIRLS-2011. A three-level (student, school and country) random intercept model was explored, as well as a random three-level model.

This study addressed three central research questions.

  1. (1)

    Which dimensions of parental involvement can be discerned and to what extent is there empirical evidence that these dimensions are related to student attainment?

From the literature review, we constructed a framework of four dimensions that combined the different perspectives from which parental involvement can be perceived (i.e., parent, student and school perspectives) with frequently-mentioned dimensions of the construct, such as home-based involvement, school-based involvement, and home-school communication. The literature review and analysis of the meta-studies in particular, further indicated an overall positive effect of parental involvement on student achievement, but considering the individual dimensions separately leads to more variable interpretations. For example, the relation between helping with homework and student achievement is positive in some studies, but non-existent or negative in others (Hoover-Dempsey et al. 2001; Patall et al. 2008). These contrasting results were explained by the complexity of measurement and lack of agreement between scholars in measuring (the dimensions of) parental involvement and the use of single-source data. None of the meta-studies recognized other possible reasons for the variable results, such as cultural differences in how parents perceive parental involvement or cultural differences in their attitude towards their child’s education. In this study, we wanted to examine whether these cultural differences could be identified and, if so, whether controlling for these differences revealed new information regarding the association between components of parental involvement and student’s reading literacy.

  1. (2)

    To what extent are there any cultural differences (differences between countries) in the components that measure dimensions of parental involvement?

We developed tools for the identification and modeling of CDIF that were based on five models: the GPCM, the GPCM with 10 and 20 % country-specific parameters, GPCM with random item parameters and the bi-factor GPCM. Firstly, we found that all models clearly and consistently supported the identification of CDIF. However, we also found the results obtained by the models varied. There was reasonable agreement for components 2 (helping with homework), 4 (student’s perception of parental involvement) and 5 (school practices for parental involvement from a school perspective). The methods clearly disagreed for component 1 (early literacy activities) and for component 3 (school practices on parental involvement, parent perspective); the latter was likely because of the poor reliability of this component, probably due to the shortness of the instrument. Disagreement in the other four tests is because different aspects of model fit are assessed by the models. In fact, the method using residuals specifically targets uniform CDIF, while the bi-factor GPCM specifically targets non-uniform CDIF. In conclusion, practitioners should not rely on one model and one approach to investigate CDIF, but diversify in their methods.

Finally, and most importantly, analyzing the influence of CDIF on the estimates of country means and on the outcomes of latent regression analyses led to the conclusion that CDIF did not influence the results. Considering all the differing components of parental involvement, CDIF had no influence on these items within the PIRLS survey.

  1. (3)

    To what extent are the different dimensions of parental involvement related to student achievement in reading literacy, taking into account student background characteristics and differences between countries?

The results of the three-level models with a random intercept showed that, controlled for student’s gender and SES, and taking into account between-schools and between-countries variance, there is a rather weak but positive relationship between early literacy activities and student achievement in reading literacy at grade 4. This positive association supports the Dutch study of Kloosterman et al. (2011), who also found that early literacy activities were positively related to student reading achievement at primary school. We may here only confirm a positive association and cannot make any claims about causality, as PIRLS is cross-sectional. The results only indicate that other types of studies (experimental studies) measuring the real effects of early literacy activities on reading achievement are relevant, assuming that there is agreement among scholars in how these activities should be measured. Our analyses have shown that if written questionnaires are applied, the current PIRLS scale seems to work identically in a large number of countries and cultures. Further, the reliability of the first two components, early literacy activities and help with homework, meets the minimum standard for a survey of 0.70 within all countries. The scale for parental involvement from the school perspective often met this standard. The scale for parental involvement from the student perspective did not meet this standard, though it consisted of 15 items. Component 3 (school practices on parental involvement, parent perspective) turned out to be an unreliable scale, probably because it contained only three items. There is clearly margin for improvement in these last three scales.

The results of two meta-studies on homework involvement and its relation with student attainment were inconclusive (Hoover-Dempsey et al. 2001; Patall et al. 2008). The exploration of the PIRLS data revealed helping with homework had a small negative effect. This may be explained by the so-called reactive hypothesis, suggesting that parents tend to react with a higher level of involvement if their child is falling behind at school (McNeal Jr. 2012). McNeal Jr. (2012) and Cooper et al. (2000) suggested another explanation for this negative association; helping with homework might also interfere with learning if parents are not sufficiently equipped to help, if they are too eager (which affects the self-confidence of their child), or if their instruction is very different from the instruction of the teacher. Again, based on the PIRLS-2011 data, it is difficult to analyze how helping with homework affects student reading literacy, but further exploration of the indirect effect of helping with home via some measure of student self-confidence in reading would be useful. Another suggestion for future PIRLS studies would be to ask parents how confident they feel about helping their child with homework and whether they feel sufficiently informed about how reading is taught at school.

Both early literacy activities and helping with homework are home-based activities, confirming that what parents do at home with their child is important for student achievement. In this study we found school-based involvement from the perspective of the school (component 5) had negligible effect. As the constructed scale for school-based involvement from the perspective of the parent (component 3) turned out to be unreliable, we are unable to draw valid conclusions for this component regarding its relation with student achievement.

Overall, the impact of parental involvement on reading literacy is not large. When all five components were entered into the model, it explained approximately 15 % at the student, 46 % at the school level, and 69 % at the country level. However, the impact differences across countries proved to be quite large, especially for helping with homework, where regression coefficients, with a mean value of 11.8, range over countries from −134.0 to 159.5. Finally, the country-level intercept and slopes for helping with homework have a substantial positive correlation of 0.34. In low-achieving PIRLS countries, the effect of helping with homework is smaller than in high-performing countries. This means that, in exploring the achievement effect of helping with homework, the educational context should be taken into account. The sometimes contradictory results of earlier studies on this subject (Hoover-Dempsey et al. 2001) may also be explained by such differing effects between countries.

Another explanation for the positive correlation between intercepts and slopes on the country level would be that, in low-achieving countries, parents’ reading competency will also be low, so parents are themselves less able to read and hence provide effective support. However, it is beyond the possibilities of the present research to draw conclusions in this respect.

In PIRLS, the literacy test and background questionnaires are translated and adapted for each country. Considerable effort is devoted to guaranteeing the international validity of these instruments. For example, the translations and the layout of the instruments are thoroughly reviewed by independent verifiers, and all necessary adaptations are documented in detail. However, it is not unlikely that there are cultural differences in the way respondents interpreted some of the questionnaire items. The main purpose of this study was to establish whether there were any cultural differences in the measurement of parental involvement in PIRLS and, if so, whether correction for these differences led to different results with regard to its relation with reading literacy. Although some of the PIRLS scales for parental involvement may require improvements to increase their reliability, the overall conclusion is that these scales are internationally valid.