Introduction

The purpose of this paper is to provide sound empirical evidence on how a community-based grass-roots intervention can help caregivers engage in high-impact parenting practices with few resources in an underserved population. The current consensus on early child development (ECD) initiatives is that the early interventions are key because the brain is most plastic in the first three years of life (Center for the Developing Child, 2016; Dawson et al., 2000), and good interventions targeting the quality of caregiving are most crucial (Center for the Developing Child, 2016; Osher et al., 2020). Although much is known about the importance of caregiver reading and playing—with randomized control trials having validated their efficacy (Attanasio et al., 2020; Knauer et al., 2019; Vally et al., 2015)—there has been a broad call for more translational evidence on in-vivo community-based interventions (Luthar & Eisenberg, 2017; Osher et al., 2020). It is termed in-vivo because it provides tailored care that is context-specific and translational in nature (see Agrest et al., 2019). The current study provides a rigorous statistical approach to analyzing a program while accommodating challenging methodological factors such as individual variability, rolling-admissions, and pandemic disruptions. The literature review below brings intervention and child development data from various cultures and locations and these are used to support the assumptions and hypotheses embedded in this work. However, the current study is conducted in a Zulu South African township which is much less resourced compared to many of the studies cited herein.

Reading & Playing as High-impact Parenting Practices

Some of the most high-impact parenting practices for early child development are instilling the habit of reading and encouraging play. A recent analysis looked at datasets from seven different African countries and concluded that, of the five components of the Nurturing Care Framework (WHO, 2018), the most crucial predictor of healthy early child development was “opportunities for early learning”, which was operationalized by reading and playing behaviors (Pierce, 2021). More specifically, parents’ reading to children in the home was the single greatest predictor of healthy early child development (Pierce, 2021). A five-year longitudinal study revealed that early home exposure to books and parental involvement in reading was predictive of early literacy (Sénéchal & LeFevre, 2002). Children who have storybooks read to them frequently enter school with larger vocabularies and more advanced comprehension skills than their peers who grow up in poorer home literacy environments (Logan et al., 2019; Mol & Bus, 2011).

Reading-based parenting interventions in low-income communities have documented success and demonstrated that early reading improves early childhood outcomes. In northern Brazil, a cluster-randomized control trial aimed at enhancing parent-child reading interactions significantly improved the intervention group’s receptive vocabulary, working memory, and IQ among children ages 2–4 (Weisleder et al., 2018). An intervention among young children in Kenya developed a storybook sharing program and caregivers were trained on how to be more active and have higher quality interactions (i.e. dialogic reading). This intervention yielded higher vocabulary gains than control groups and was successful despite caregivers’ low literacy rates (Knauer et al., 2019). Similarly, a book sharing program also documented significant improvements in reading indicators among a South African community with high rates of illiteracy (Vally, et al., 2015). This book sharing intervention also improved the quality of the caregiver-infant relationships and higher prosocial behaviors and social understanding (Murray et al., 2016). Similarly, a dialogical storybook reading intervention with parents of 3–4 years olds in Turkey was linked to an increased receptive vocabulary and better attitudes towards reading (Kotaman, 2020). Even when engaging reading techniques are not taught to parents, simply distributing books can increase reading behavior (Thakur et al., 2016) and story comprehension (Knauer et al., 2019). The breadth of these cultural contexts and convergence of their findings suggest that interventions that can enhance reading behaviors will have positive effects on early child development, especially in low-income contexts.

In addition to reading, encouraging self-directed creative playing is one of the most important strategies for early brain development (Center for the Developing Child, 2016). Pretend play is positively correlated with assertiveness and cooperation (Li et al., 2014), social competency (Colwell & Lindsey, 2005), and emotion regulation skills (Cabrera et al., 2017). When parents engage their children in high quality play, children broaden their vocabularies (Cabrera et al., 2017) and are more likely to have higher kindergarten reading scores (Jung, 2016). In a weekly home visitation intervention in Columbia, researchers found that the high-quality play demonstrations were linked to significant improvement in cognitive scores and receptive language among a sample of 1,420 toddlers (Attanasio et al., 2014). A follow-up analysis concluded that the intervention’s impact was accounted for by increased parental involvement and the study called for more research on how to best target and treat the most disadvantaged children across the globe (Attanasio et al., 2020). Similar to that intervention, the current study also employs females that share the same cultural background as the target homes, and provide home visitation support in low-income communities.

Parental Beliefs of Child Development

Although it is uniformly demonstrated that active engagement with children in their first years of life have positive long-lasting effects, the literature has documented a lack of awareness among caregivers. In a descriptive study with Peruvian mothers of infants and toddlers, participants said they did not read to their children before the age of three because they believed their children to be too young (Erkel et al., 2019). In American samples, parents who believed there is not much they can do to prepare their children for school, spent less time reading with their children (Weigel et al., 2006) and mothers of preschoolers who more strongly endorsed reading beliefs have children with better academic outcomes (Cottone, 2012). This may be because parents who know about the positive role of play in child development are significantly more likely to engage with their children’s play, as a recent Qatari sample revealed (Ihmeideh, 2019).

Some researchers have documented cultural contexts that discourage parents from talking to their children due to a misunderstanding of its value to infants (Weber et al., 2017). In that study, when parents in rural Senegal attended classes on child development and engaged in a home visitation intervention, they had higher levels of speech during play sessions (Weber et al., 2017). This is an encouraging area of intervention science, as it suggests that parental education and daily high-impact behaviors in reading and playing can significantly improve early childhood development. Similar to the Senegal sample, the intervention of this current study has anecdotally identified lack of knowledge of the importance of play, reading, and speaking to infants as a barrier to healthy child development.

Social Support

Scholars agree that the most effective way of maximizing child resilience is to support the caregivers (Luthar & Eisenberg, 2017). Parental social support has been associated with more warmth and less hostile behavior towards their children (Lippold et al., 2018), more authoritative parenting behaviors (Byrnes & Miller, 2012; Green et al., 2017), and more secure attachment styles (Green et al., 2017). A study of over 1,700 families with young children in Korea revealed that, when mothers’ perceived support was high, they provided a more stimulating learning environment for their children, and their social support predicted better language development (Chang, 2017).

Parents’ mental health during the first few years of life significantly impacts children’s brain development and long-term behavioral outcomes (Dawson et al., 2000). Thus, lack of sufficient social support is a high-risk factor for early child development problems and caregivers of young children should be a priority group for social support interventions. Family driven interventions that provide social support buffer adversity and enable parents to continue responding appropriately to their children (Osher et al., 2020).

Relationships strengthen the human capacity to face adversity (Center for the Developing Child, 2016) and interventions that focus on strengthening the parent-child relationships are most promising in improving early developmental trajectories (Valentino, 2017). For mothers who suffer the outsized stressors of poverty, social support is particularly crucial. However, the negative effects of poverty are moderated by family and community resources and social networks (Osher et al., 2020). Interventions in low-income settings have successfully documented improvements in caregivers’ social support networks, which in turn was linked to stronger well-being, higher parental competency (Hung & Zhou, 2017). Although social support alone is not going to solve complex social problems, it should be an essential component of any multi-pronged effort that aspires to realistic long-term change in early child development.

ECD in a Pandemic

A full picture is yet to emerge on how the pandemic shutdowns influenced caregivers’ abilities to stimulate ECD. A recent paper with data from rural Bangladesh suggested that the pandemic shutdown may have decreased caregiver well-being which may have altered the ability to provide responsive care, subsequently affecting ECD (Pitchik et al., 2021). Research has reported high rates of caregiver anxiety, depression, and perceived child stress among parents of young children (Russell et al., 2020; Roos et al., 2021). A lack of access to financial and nutritional resources due to the pandemic resulted in worsened caregiver mental health (Pitchik et al., 2021). One study found that child outbursts were worst among children of distressed caregivers, indicating how adult mental health can impact relationships and conflict (Russell et al., 2020). Among 1,200 caregivers in Japan, 24% had moderate mental distress early in the pandemic, and 29% had severe mental distress (Horiuchi et al., 2020). Qualitative data from North America revealed high levels of parenting self-doubt and high psychological distress during the shutdown (Roos et al., 2021). A survey during the first pandemic shutdown revealed that parents’ distress influenced kids’ emotional regulation but it was mediated by parenting self-efficacy and parents’ emotional regulation (Morelli et al., 2020). This indicates that internal parental belief variables seemed to be a key factor in protecting families’ overall well-being. However, these studies have predominantly studied samples in more affluent contexts and less has been published in non-Western, low-income samples. There is also a dearth of studies that have multiple longitudinal data points after the initial shutdown. Little research has been published looking at ECD interventions and how they have managed to support their communities and what strengths were most relevant to their success. This study shows how ECD initiatives have persevered and promoted positive outcomes even in a pandemic and with little financial support or governmental infrastructure.

Context

This study analyzes the effects of a community intervention located in a peri urban community in the outskirts of Pietermaritzburg, South Africa. This is a racially segregated and under-developed township traditionally reserved for non-whites, a long-lasting effect of apartheid. Due to pervasive unemployment and lack of opportunities, many are forced to move to the city for jobs, leaving family or children. This community is considered the highest HIV infected community in the world (HSRC, 2018) and has left many children orphaned. Homes in this community rarely represent a western nuclear family model. A large percentage of men are absent from the community, or at least from the lives of their children. Sometimes this absence is explained by the need to migrate to larger cities for work or the HIV pandemic. Women are largely responsible for raising children, and most homes have multiple buildings where different extended family members live. It is very common for children to talk about their “mom” who is really their aunt, or a “sister” who is biologically their cousin. For this reason, this paper uses the term “caregiver” instead of “parent”. However, the manuscript does use the term “parenting practices” and applies it to all types of primary caregivers.

Approximately 80% of children in South Africa do not have access to ECD programs (Knowledge, Information and Data Solutions, 2019). While some ECD centers do exist in this community, the quality is typically far below standard because they have historically had to be privately funded, the practitioners are untrained, and there is a pervasive lack of understanding of the importance of ECD. Thus, many centers are seen as babysitting services, rather than places for holistic development of young children. Poverty in the area means that ECD centers are not affordable for many. It is not uncommon for young children to be cared for by their older siblings who are still attending school themselves.

Importance of the Current Study

Children in poverty face unique and outsized risks for their healthy development; therefore, we need translational research on how community-based interventions can promote positive caregiving behaviors and better support caregivers who are on the front lines of ECD. A survey of South African ECD experts revealed that the top priorities of research should be in understanding how interventions can be implemented and assessed in rural low-income settings, and how paraprofessionals could deliver interventions effectively (Tomlinson et al., 2019). Researchers have called for translational research in real-world community settings (Luthar & Eisenberg, 2017; Osher et al., 2020).

The purpose of this study is to provide a rigorous quantitative assessment of a community based ECD intervention to understand the main drivers of caregivers’ habitual reading and playing practices. It is shown these are high-impact practices, and there is evidence that they need to be supported by positive beliefs about ECD and strong social support systems. Additionally, this paper conducts exploratory analyses to understand the impact of the pandemic shutdown on the measures of social support, program engagement, caregiver beliefs of ECD, and positive parenting behaviors. ECD interventions are especially impactful in developing countries and communities affected by poverty (Blair & Raver, 2016; Rao et al., 2017). However, impoverished communities with little infrastructure, low literacy levels, and high crime rates present unique challenges to research. Community-based interventions often have rolling enrollment, no control group, and heavily rely on paraprofessionals. These programs have many individual differences in child ages and caregiver characteristics and varying attrition levels due to little control in recruitment and little budget for incentives. However, there are now many advanced statistical tools that expand opportunities of analysis to account for individual longitudinal differences, differing start dates, and variation in quantity of assessment points. It is more possible now than ever to conduct robust analyses on quasi-experimental community-based interventions. The research questions driving this paper are:

I. What individually-varying and time-varying predictors explained positive parenting practices (reading and playing)? Predictors included are parental beliefs on ECD, number of home visits, caregiver social support system, additional program opportunities attended, and caregiver involvement in the home visits.

II. How did the pandemic shutdown impact the trajectories of the variables of interest?

Methods

Program Description

Each home in the program is visited biweekly for approximately an hour by their assigned ECD mentor. These visits are designed to demonstrate quality ECD in a one-on-one environment and the focus is on two key behaviors: reading and playing. During this time, caregivers learn about ECD, interact with their children, develop at-home resources, and borrow educational toys and books. There is no library in the community and usually homes have no books or toys at the beginning of the program. The program provides resource bags which are exchanged out every month. Bags include a gross motor toy, a fantasy toy (doll or soft toy), toy car, construction toys, two books, and occasionally a puzzle. Baby toy bags include a floor blanket, a doll or soft toy, soft ball, rattle, and another baby toy. In addition, an isiZulu children’s book is exchanged at every visit. Many caregivers, especially grandmothers, are illiterate. In that case, the mentors train them to tell stories using the pictures.

To develop the program, the National Early Learning Development Standards (NELDS) were combined with the organization’s practical ideas and input from an occupational therapist to acknowledge the context of the families. The program consists of 20 lessons with each visit following a similar routine while being recorded via a smartphone utilizing the KoBo Toolbox open-source software designed for humanitarian field research. During each visit, mentors greet and chat as they prepare a place for the lesson. The topics range from the importance of movement, play, reading, to using household and recycled objects, to dancing and emotional development. Typically, the child plays with provided toys during discussion. Approximately 25 min are spent on activities, including gross and fine motor skills, literacy, numeracy and conceptual development, as well as activities linked to the topic discussion. This curriculum is different for the 0–18 months; 18–36 months and 3–6 year age groups.

The mentor observes how the child interacts with the toys and finishes the hour-long visit by reading a new book to the child or sometimes the caregiver reads one. The focus of the visit is on non-judgmental relationship building and working together, not explaining or lecturing. Caregivers are expected to participate the whole time, such as sitting on the floor with the mentor and the child, telling stories, and chatting. The mentors rate the caregiver engagement after each visit to track this across time. The mentor completes the KoBo toolbox survey and logs out at the end of each visit. The program has rolling enrollment and homes typically stay in the program for two years, or graduate when the child goes to grade R (i.e. kindergarten). If the caregiver has a younger child and asks to stay in the program longer than two years, they are permitted to do so.

The program also provides monthly Fundanathis (translation “let’s read together”), outdoor playgroups within walking distance of participating homes. These events focus on a special book and provide games, music and crafts based on the story. During this time, books are provided and caregivers spend time reading to children and are encouraged to share success stories. These opportunities vary in engagement due to weather and the ages of children. These events are meant to strengthen relationships between caregivers and improve their support system in the community. Many of these are held throughout the community so each household has one within walking distance. The availability and engagement of these groups are also tracked by the mentor each visit.

During the initial pandemic shutdown (March–August 2020), there were neither Home visitations nor Fundanathis. However, mentors kept in touch with the homes via Whatsapp messaging system and delivered home programmes (KhulaEkhaya - translation “grow at home”) that provided encouragement and ideas on how to engage children, sharing new ways to create toys out of recycled materials or how to play and learn through daily activities. An isiZulu podcast series with home stimulation ideas was also created and links posted on Whatsapp groups. Home mentors assisted families in establishing home food gardens which provided opportunities to keep up the relationships. Mentors estimated at least 50% of the homes had engagement on Whatsapp, but that data was not collected. The program played an active role in disseminating science-based information about the Covid-19 pandemic. They created one of the first videos in isiZulu that was widely disseminated even by the governmental organizations to spread accurate information about masking, distancing, and hand-washing practices. Beginning in September of 2020, mentors resumed visiting with masks and prioritizing outdoor activities. Even after the Fundanathis resumed in October, attendance was lower due to concerns of contagion.

The current costs of the program include two full-time mentors, both Black Zulu women who share the same cultural background as the community. The program leader has oversight responsibilities in the non-profit that include resource support for local ECD centers and other organizational administrative roles. The non-profit organization is South African, has been working in the community for 25 years, and the ECD program is primarily supported by South African humanitarian grants and individual donors.

Participants

All participants were caregivers of children ages 0 to 4 who live in the community, a township outside of Pietermaritzburg in the province of KwaZulu Natal. The province of Kwa-Zulu Natal has the second largest population in South Africa, with 21.1% of the population being under the age of 15 (South African Government, 2019). Black South Africans represent the largest ethnic group and isiZulu is the most commonly spoken language (South African Government, 2019). The province of KwaZulu Natal has a per capita income lower than the national average and Black South Africans outside the city are the most affected by poverty and unemployment (Provide Project, 2005).

All participants were Black, Zulu women, including mothers, aunts, grandmothers, with the exception of one father, two Malawian women, and one Nigerian woman. All homes spoke isiZulu as their primary home language. In most cases, the caregivers were unemployed. When mothers were employed and living with their children, the grandmothers often were the primary caregivers and the ones enrolled in the program and completed all visits and assessments. The focus of the program was to meet the needs of those who do not attend an ECD center. However, as caregivers progress through the program, many realize the value of ECD and enroll them in a center.

The questionnaires did not gather data on participants’ education and employment status due to ethical and logistical considerations. However, the census data are very specific to the community and these are believed to be an accurate representation of the current study’s sample. Previous data from this ECD program (n = 68; not directly overlapping with this sample) revealed that the majority of caregivers completed grade 12 in school (59%), 26% completed grade 11, and 12% completed grade 10 or less (Good, 2020). Exact demographic percentages of participants in the current study are difficult to ascertain with precision because of the confidential process of the data collection. Asking questions that would seem irrelevant to the intervention, such as education or employment history would risk skepticism and weakened trust between the caregiver and mentor. However, previous data from the ECD (kept separate from current data files; Good, 2020) are consistent with the most recent census data on unemployment and education status (Wazimap, 2011), strengthening the ecological validity of the data from the study participants. According to the most recent census, 67% have completed ninth grade or higher, and 37.7% completed grade 12, and less than 4% completed more than grade 12 (Wazimap, 2011). Only 31% of adults between ages 18–65 are employed and approximately 30% of households have internet access which is typically from a cell phone. In this community 36% of the population is under the age of 18, 5.5% of children have no living biological parents, and 34% of households are headed by children under the age of 18 (Wazimap, 2011). No participants were removed from the dataset. All households that participated in the intervention during the time-period, are included in the analysis, strengthening the ecological validity of these translational data.

Procedures

Participants were recruited mostly through word of mouth, as is a common practice in non-profit programs. Word-of-mouth recruitment is the practice that generates the most bottom-up community trust, a prerequisite for longitudinal change. When the program first began in 2016, ECD mentors would walk through the community looking around for young children or homes with baby clothes hanging from the clothesline and introduce themselves to the homes. As the program became more visible, community members would stop mentors and ask if they could join the program and participants often refer friends or neighbors. While this snowballing recruitment practice increases the risk of ascertainment bias, it strengthens the ecological validity of the findings.

This paper utilizes data from March 2019 until May of 2021. Prior to March of 2019, there was turn-over with the mentors and training was still ongoing to establish inter-rater consistency. There were 157 homes that participated during this time frame and all were included in this study. No participants were excluded from the dataset for any reason. The piloting of the questionnaires used in this paper began in 2016, but establishing reliable metrics, training mentors, and having a long-lasting season without mentor turn-over took an initial acclimation period. The program coordinator had multiple sessions with the mentors, where they engaged in role-playing scenarios and discussed real-life examples in their mentor meetings. In these evaluation trainings, the team would self-reflect on possible biases, and share how they interpreted certain behaviors and when to ask follow-up questions. For example, if the caregiver was reporting regularly playing with the child, but the child did not act as if they were familiar with the toy, the mentors would ask follow-up questions and provide non-judgmental prompts about the challenges of taking time to engage the children. The relational nature of the program strengthened the validity of the data by strengthening the trust between caregivers and mentors. The mentors also took turns accompanying each other to multiple homes during this validation process, rating the same interaction, and discussing any discrepancies. The program coordinator visited multiple homes with each mentor, providing consistent feedback and aligning evaluation scores. This practice solidified the scale and credibility of the mentor ratings. This acclimation led to much more confidence in the validity and reliability of the data, which is verified through the statistical assumptions testing described in the sections below.

ECD mentors were trained to guide caregivers to understand Likert scales without influencing their responses. Before collecting surveys, they would practice items with sentences such as “I don’t like ice cream” and other examples to reiterate how there are no right or wrong answers. This training process was repeated every time the caregiver surveys were completed (~every six months). Beliefs and Support questionnaires were completed approximately every March and October, or whenever they enrolled in the program but were only completed once after the pandemic’s onset (November, 2020).Participants verbally consented and all information was de-identified to the researchers and the usage of the data was approved by the Institutional Review Board of the first author.

Measures

Caregiver beliefs

Beliefs about child development were assessed by a twenty-item beliefs questionnaire. This questionnaire was created based on the Parent Play Beliefs Scale (Fogle & Mendez, 2006), the Parents’ Opinion Survey (Hogan & Tudge, 1994) and additional written items based on the program’s priorities. The additional items were all in the area of “reading” because the aforementioned scales did not have a separate scale for reading beliefs. These were written to be similar to the “playing” and talking” items, but focused on reading. All items were translated from English to the community’s native language of isiZulu then back-translated by a separate native speaker to ensure the accuracy of the questions. The questionnaire had seven items on Play (2 reverse-coded), three items on Hope (1 reverse-coded), seven items on reading (2 reverse-coded), and 3 items on Talking (1 reverse-coded). However, when conducting preliminary factor analyses for this study, all reverse-coded items loaded on their own separate factor, thus resulting in a five-factor solution; this persisted whether reverse-scored items were in their raw or reversed form. For this reason, along with the cultural understanding that double-negative phrases are especially problematic in isiZulu, we eliminated the reverse-coded items from the analysis. This resulted in five items on Play (e.g. “Playing at home will help my child prepare for grade R”; α = 0.73), two items on Hope (e.g. “I am convinced my child faces a very bright future”; α = 0.56), five items on reading (e.g. “I believe my child will benefit if I read to him/her at a young age”; α = 0.74), and two items on talking (e.g. “I believe that it is important to spend a lot of time talking to my children”; α = 0.63). Reading and Talking items were collapsed into a single factor identified as Communication (α = 0.82) due to improved reliability and consistent loading of both scales’ items onto a single factor. Each question was given a rating on a 6-point Likert scale, with one being “strongly disagree” and six being “strongly agree”. The sum score of each item was created in order to develop a composite score.

Caregiver Social Support

This was assessed with a slightly modified version of the Multidimensional Scale of Perceived Social Support (Zimet et al., 1988). The original scale was modified to specify a parenting scenario. For example, “There is a special person with whom I can share my joys and sorrows” was changed to “-the joys and sorrows of parenting” and “I can talk about my problems with my friends” was modified to “I can talk about my problems regarding my children with my friends”. This measure under-went the same back-translation process described above. The original scale was divided into Family, Friends, and Significant other, with four items for each scale; no items were reverse-scored. For the purposes of this intervention, the scales of Friends (e.g. “I can count on my friends to help me with my child(ren) when things go wrong”; α = 0.90) and Special person were utilized (e.g. “There is a special person in my life who cares about me and my child(rens) needs”; α = 0.92). The Family subscale had been piloted years before, but the scores were very high with negligible variability due to the high loyalty to family. Further, Friends and Special Person constructs had extraordinarily high correlations in preliminary analyses so these scales were collapsed into a composite Support scale (α = 0.90). Each question was given a rating on a 6-point Likert scale, with one being “strongly disagree” and six being “strongly agree”. The sum score across items was created in order to develop a composite score.

Mentor Forms

Mentors completed questionnaires after every visit utilizing KoBo Toolbox on smartphones. On a five point scale, mentors selected how involved the caregiver was at the visit (1 = does not engage at all; 2 = participates in less than a quarter of the activities; 3 = participates in less than half of activities; 4 = actively participating; 5 = smiling, engaging, actively participating), the number of times a week the caregiver read to the children (<1 day; 1–2 days; 3–4 days; 5–6 days; 7 days a week), played with the children (less than 30 min a day 5 days per week; 30 min per day 5 days a week; 1 h each day 5 days a week; 1 h everyday 7 days per week; more than 1 h everyday 7 days per week). Further, in seeing the high correlation between the reading and playing behaviors in preliminary analyses (r = 0.73, p < 0.001) and the nearly identical effects of all other constructs on these two variables, it was determined that these two variables should be considered a composite Positive Parenting Behavior (PPB) variable. The mentor form also gathered data on if there was additional ECD opportunity available (e.g. Fundanathi group), and if they attended it. If so, there was typically only 1 opportunity available (range = 0–2).

The mentor form was created by the program leadership, based on their primary goals of promoting reading and playing among caregivers and desire to engage and empower caregivers. Mentors received extensive training to ensure reliability and minimize social desirability bias. Mentors practiced filling out forms on each other and did visits together (team leader and two mentors) to establish interrater reliability. Additionally, the team leader did multiple visits with each mentor and they would complete the same form and thoroughly discuss it. Since the intervention is based on relationships, and mentors have extensive experience with children, they could discern variability in how familiar the children were with the toys and books and assess, through conversations and interactions, how often caregivers played with and read to their children.

Psychometric Analyses

Initial psychometric analyses suggested a four-factor structure across the items in the survey: Special person, friends, hope, playing and talking, and reading. Further, this tentative factor structure (estimated using weighted least squares due to a violation of multivariate normality among items) provided better fit (i.e., fewer cross-loaded items) when oblique factor rotations were used; both promax and geomin rotations were employed and yielded better fit than unrotated or varimax (orthogonal) rotations. Further, all four-factor solutions provided better fit with fewer cross-loaded items than three-factor solutions, as was suggested by results from parallel analysis, eigenvalue, and optimal coordinates scree plots alongside Velicer’s (1976) minimum average partial method.

When determining whether to collapse factors, exploratory factor models were employed alongside standardized Cronbach’s alpha coefficients. Items from conceptually related factors were collapsed into single factors and yielded improved reliability statistics. The communication factor was formed by combining reading and talking with a resulting α = 0.83; support was formed by combining special person and friends with a resulting α = 0.91. Hope and play each retained their own factors with alpha coefficients of α = 0.56 and α = 0.84, respectively.

When further verifying the factor structure as suggested by conceptual similarity and reliability analyses, a confirmatory analysis with correlated factors—play, hope, communication, and support—yielded factor loadings ranging from 0.49 to 0.786. The confirmatory factor model yielded an SRMR of 0.075, RMSEA of 0.048, and Comparative Fit and Tucker-Lewis indices of 0.893 and 0.879, respectively, when employing a diagonally weighted least squares estimator to adjust for a violation of multivariate normality among the items. These fit indices and factor loadings suggest the model fit the data well, thus supporting this four-factor structure.

Statistical Analysis

Relationships between program involvement and parenting behavior variables measured were initially assessed by way of Pearson’s r correlation coefficients in order to determine between which constructs notable hypothesized multivariate relationships may exist. Hierarchical linear modeling (HLM; Raudenbush & Bryk, 2002) was selected as the most viable analytical framework to understand the predictors of positive parenting behaviors. HLM is flexible to individual variations, start and stop dates and missing data and has the ability to conduct longitudinal analyses, nesting time points within homes. As described in the Methods section, the Communication and Support variables were constructed in similar manners due to the high correlations between their respective constituent constructs. That is, a preliminary exploratory factor analysis of all the above-described scales indicated that Reading and Talking belief items loaded onto the same factor (thus called ‘Communication’), and the Friends and Special Person items loaded onto the same factor (thus called ‘Support’). Consequently, sum scores per person across both constellations of the above-stated items were used for analysis purposes.

A final consideration was the effect of the pandemic. While the correlations between calendar date and all other constructs ranged from nonexistent to moderate (likely due to rolling admission and differing baselines) the true effect of the shutdown was obscured. Therefore, a set of exploratory scatterplots were conducted separately, treating the onset of the pandemic as a discontinuity point.To isolate the effects of actual program activities, several program- and family-level characteristics were identified with only the number of visits selected as the sole “time” variable in the HLM. Consequently, the resulting full model constructed to examine the effects of program and family characteristics on PPBs can be identified as

$$\begin{array}{l}PPB_{it} = \pi _{00t} + \pi 0_{1t}Visits + \pi _{02t}InvolvementCaregiver\\\qquad\qquad +\, \pi _{03t}OpportunitiesAttended + e_{it}\\ \quad\,\pi _{00t} = \beta _{0i} + \beta _{1i}Support + \beta _{2i}Play + \beta _{2i}Comms + \beta _{3i}Hope + r_{0i}\\\quad\, \pi _{01i} = \beta _{1i} + r_{1i}\\\quad\, \pi _{02i} = \beta _{2i}\\\quad\, \pi _{03i} = \beta _{3i}\end{array}$$

where PPBit = Predicted positive parenting behavior at time point t for participant iπ00t = Grand mean/Intercept π0pt = Coefficient estimate for Level-1 predictor p at time point teit = Residual error β0i = Intercept for family iβpi = Coefficient estimate for Level-2 predictor p for family iroi = Variance estimate of the intercept for family ir1i = Variance estimate of the slope of Visits estimate for family i.

While support, play, communication, and hope beliefs were all measured at each of the time points, the intraclass correlations (ICCs) for these measures ranged from 0.988 to 0.999, thus indicating nearly all the variability in these values was situated at the family level rather than the temporal (visit) level. Consequently, these measures were specified as level-2 (family level) fixed effects in the HLM. Conversely, the number of visits, involvement of the caregiver, and opportunities attended were all measured at the temporal level and yielded ICCs ranging from 0.22 to 0.47, thus indicating that a substantial proportion of variability was situated at the temporal level (level 1 predictors). Additionally, due to the sheer variability in trajectories of PPB, “visits” was specified as a random effect. To assess the relative efficacy of each predictor, both metric and standardized coefficient estimates were obtained. While estimates of variance accounted for are not universally established, Nakagawa’s R2 has been shown to provide a reasonable estimate of both marginal (fixed effects only) and conditional (mixed effects) variance accounted for and was used in the present analysis (Nakagawa et al., 2017).

Assumptions of normality, homoscedasticity, independence, and linearity were assessed for both fixed and random effects residuals. Multicollinearity was also assessed with all tolerance values above 0.5 (RangeTol. = 0.501–0.986) and VIF values lower than 2.0 (RangeVIF = 1.018–1.996).

The full HLM was sequentially constructed by beginning with a null model:

$$\begin{array}{l}y_{it} = \pi _{00t} + \mu _{it}\\ \pi _{00t} = \beta _{0i} + \varepsilon _{0i}\end{array}$$

with blocks of predictors included in subsequent models in order to determine the most optimal model. A total of five models were estimated and compared to one another using the Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and deviance/-2 Log-likelihood (-2LL). Additionally, because the models were nested in nature—that is, all predictors of previous models were included in later models—a χ2 deviance test was also used to determine whether each more complex model provided better fit than the model preceding it. Table 1 illustrates the parameter estimates, significance tests, fit statistics, and model information for all five models.

Table 1 Results from HLM models predicting positive parenting behaviors

A sensitivity analysis was conducted by employing a Bayesian general linear mixed effects model using Markov Chain Monte Carlo estimation (MCMCglmm) to internally replicate the results of the frequentist HLM and mitigate any potential method effect (Hadfield, 2010). The MCMCglmm functions in a manner similar to a standard HLM, but in lieu of traditional coefficient estimates, MCMCglmm probabilistically creates a distribution of estimated values (the posterior) for each specified parameter given a hypothesized prior distribution of parameter estimates. A final element to note in the present study was the impact of the Covid-19 pandemic on the constructs of interest. While a model containing a priori specified effects for Covid-19 would prove to be computationally intensive and logically convoluted, the influence of the pandemic could not be ignored. Therefore, exploratory plots were considered to examine this effect with a predetermined discontinuity point of April 1, 2020, which separated all visit data based on the program’s shutdown.

Results

Descriptive statistics prior to the pandemic, after the initial lockdown, and overall were conducted and can be found in the Supplementary Materials. Results from the principal HLM analyses indicate that all time-varying predictors—caregiver involvement, opportunities attended, and visit number—were significant positive predictors of PPB, as were level-2 beliefs about hope and support. Caregiver involvement was the strongest of all the predictors with a standardized coefficient estimate of 0.497 (Coef.Unstd. = 0.946; p < 0.001) followed by visit number (Std. = 0.199; Coef.Unstd. = 0.026; p < 0.001), support (Std. = 0.177; Coef.Unstd. = 0.105; p < 0.001), hope (Std. = 0.112; Coef.Unstd. = 0.320; p < 0.05), and opportunities attended (Std. = 0.081; Coef.Unstd. = 0.142; p < 0.001). Further, when considering the random effects variance components, 95% bootstrapped confidence intervals were utilized to determine significance. The random intercepts component was significant (CI [0.473–1.013]) as was the random slopes component for visit number (CI [0.0013–0.0035]). The final model, “model 3 with random slopes”, was deemed the best fit based on performance indices utilized. Parameter estimates and model fit statistics are shown in Table 1, below.

Standardized coefficient estimates for the full model are shown in Table 2, below, in order to compare predictor importance.

Table 2 Standardized coefficients for model 3 with random slopes

To approximate the proportion of variance in PPB accounted for, Nakagawa’s R2 was employed for both the marginal (fixed effects only) and conditional (mixed effects). Using only the fixed effects, the marginal R2 was estimated to account for 10.6% of the variance in PPB, while the inclusion of the random effects (both random intercepts and random slopes for visit number) accounted for approximately 22.2% of the variance in PPB. Of particular note is the increase in the conditional R2 from 0.179 to 0.222 when allowing the slope for visit number to be random. The sensitivity analysis was conducted to verify if a different statistical approach could be taken and reveal similar results. The results shown below are consistent with the HLM results in that the same variables were considered significant and in the same direction. Therefore, we can be confident that the findings presented in this paper are internally valid and robust. The results of the analysis can be found in the Supplementary Materials.

Discontinuity Plots to Analyze the Pandemic Effects

The data in this manuscript came from March of 2019 to May of 2021, and no individual data was collected between April-October 2020. To understand the effect of the shutdown on caregivers and the program, exploratory regression discontinuity plots were conducted on each variable. The X axis is the number of visits, not the date. This was done so that the rolling admissions would not pull the fit line down. The exploratory plots created illustrate the effect of visit number on each of the predictors and outcomes in the main analysis both before and after the shutdowns due to the pandemic.

As can be shown in Fig. 1, below, beliefs about Communication and Play as well as proportion of engagement opportunities attended all had a strong positive linear relationship with the number of visits prior to the shutdown. It is important to note that Play beliefs, Communication beliefs, and Support were all only collected one-time post-shutdown. Thus, any perceived linear relationship after the discontinuity line is a cross-sectional relationship based on the number of visits they had received by that time. Regarding Play and Communication beliefs, those who had approximately 25 visits prior to shutdown (approximately one year in the program) seemed to maintain positive beliefs, while those who did not have as many visits tended to report lower beliefs about the importance of playing with, and reading and talking to their children.

Fig. 1
figure 1

Trajectories of constructs across visit number before and after Covid-19 shutdowns

Caregiver involvement and PPB had several time points after shutdown and, surprisingly, had a stronger positive linear relationship with the number of visits after the shutdown. Reasons for this are speculative and discussed in the next section, but results suggest the possibility for positive ECD change even as communities battle with the pandemic. The linear relationship between the proportion of opportunities attended (i.e. opportunities attended divided by the ones available during that time) seemed to change after the shutdown and settle lower than the previous fit line and assessments among those with a high number of visits was noticeably higher post-shutdown.

Support and Hope beliefs both yielded unorthodox patterns that generally trended downward both before and after the shutdown. These plots cannot convey statistical significance. They only provide a reasonable indication of general trajectories and linear relationships between variables both before and after the shutdown.

Discussion

Previous research has called for robust translational research that supports integration and application of developmental science (Osher et al., 2020). Luthar and Eisenberg (2017) specifically called for translational intervention science with low-resourced samples in real-world community settings in order to understand the systemic supports needed to foster program success. The current study sheds light on the programmatic and individual difference precursors in a low-resourced community-based intervention. Results highlight the importance of individual engagement, social support, and length of intervention, in and how these elements can foster growth even in a pandemic.

The most important predictor in this study was how engaged the caregivers were in learning from the mentors and relating to their children at a specific point in time. The strength of the caregiver engagement over other variables indicates the importance of accounting for individual differences of engagement when understanding the effectiveness of ECD interventions, but also realizing this individual difference can be leveraged as caregivers gain trust and lean into the program. This importance of caregiver engagement converges with the conclusions of a recent longitudinal study in a low-resourced Columbian home-stimulation intervention. That study found that the intervention’s impact was accounted for by parental involvement (Attanasio et al., 2020). It is important to remember that the current study measured caregiver involvement as a changeable individual difference that was best accounted for at the time-varying level. Thus, interventions and ECD professionals should strive to earn the caregivers’ trust and increase their engagement as a way to maximize the intervention impact. The strength of this predictor also highlights the inherently relational nature of home visiting programs. Additionally, the slope of the positive linear relationship between caregiver engagement and number of visits actually increased after the onset of the pandemic. The reason for this is unknown, but it may be due to the overall reduction of support and relational opportunities due to social distancing, which may have accentuated the value of the home visitation intervention as the pandemic continued. If so, this highlights the value of mentor relationships as a key factor to weather the pandemic with strong ECD stimulation.

The strength of caregivers’ self-reported support system was also a leading predictor of positive parenting behaviors. Prior research has suggested that when parents’ support is high, they provide higher quality learning environments (Chang, 2017) and family-driven interventions that provide social support help parents engage in more responsive care (Osher et al., 2020). Caregiver engagement and social support should be at the forefront of program development and evaluation and should be prioritized in ECD. These two predictors together showcase the importance of strong relationships. Existing research on relational interventions suggests it can improve early developmental trajectories (Valentino, 2017), and leading ECD researchers are unanimous on the importance of involving and supporting caregivers to promote child resilience (Luthar & Eisenberg, 2017). The number of visits was a strong predictor of positive parenting behaviors, suggesting the efficacy of the program in changing reading and playing behaviors. The rolling admissions nature of this program made it most important to look at the number of visits, instead of calendar date. It is likely that the long-term vision this program had for ECD behavior improvement is one of its major strengths. It takes time to change habits, and allowing participants to receive bi-weekly visits for two years is a strong model for lasting change. The regression discontinuity plots suggested that those who had been in the program longer prior to the shutdown had a more positive profile overall, especially in beliefs about ECD and those beliefs are known precursors to positive early child development (Erkel et al., 2019; Ihmeideh, 2019).

Beliefs about communication, play, and hope for the child’s future were the last variables added to the predictive model of positive parenting behaviors. This was done in order to understand if the effects of early childhood beliefs added value above and beyond other programmatic and individual differences. Reported beliefs about communication (talking and reading), and playing with children, did not significantly improve the prediction of reading and playing behaviors. This might be because of the high mean values of these variables, it is likely that caregivers already had positive beliefs and these did not act as barriers to ECD in this sample. It also may be that the variance of these beliefs is already accounted for by the caregiver engagement variable. The regression discontinuity plots did reveal that the caregivers who had been in the program around a year or more before the shutdown sustained higher beliefs of the importance of reading, talking, and playing after the pandemic shutdown. This may point to the intervention as a potential reason for the resiliency of these beliefs in the face of hardship.

Hope beliefs did significantly improve the model above and beyond the effects of caregiver involvement, number of visits, and opportunities attended. This effect should be taken lightly because of its tenuous psychometric properties. However, this finding does align with research showing that parents who believed they could prepare their children for school were more likely to read with them (Weigel et al., 2006) which is comparable to this study’s operational definition of hope. Thus, we recommend further investigation on the role of caregivers’ hope in predicting the efficacy of an ECD intervention. Furthermore, a study from caregivers shortly after the onset of the pandemic suggested that parenting self-efficacy may mediate the detrimental effects of pandemic parenting distress on ECD (Morrrelli et al., 2020). While the effect of the shutdown on all three belief variables (play, communication, hope) was difficult to determine in this study, it does seem like caregivers who had been in the program approximately 25 visits were more likely to report positive beliefs compared to those with fewer cumulative pre-pandemic visits. In other words, if the pandemic hit after almost a years’ worth of visits, caregivers’ positive ECD beliefs were more resilient during the pandemic.

Several studies have been published since the onset of the pandemic noting the rising parental stress (Horiuchi et al., 2020; Russell et al., 2020; Roos et al., 2021) and social development concerns in ECD, especially in situations of low support and resources (Pitchik et al., 2021). However, the current study reveals that there can be positive programmatic change in the midst of a global pandemic, even without large-scale funding and even in a high risk and low-resourced community. The purpose of this study is not to understand how the pandemic impacted ECD, but to shed light on how a community ECD interventions was affected by it and which aspects made it most resilient and effective.

Overall, research has suggested that, of the five components of the Nurturing Care Framework (WHO, 2018), the one with highest effect on positive ECD is opportunities for early learning (i.e. reading and playing behaviors) (Pierce, 2021). There is substantial longitudinal evidence that increasing playing and reading behaviors in the family unit can lead to long-lasting improvements in language development (Attanasio et al., 2014; Cabrera et al., 2017; Knauer, et al., 2019; Vally et al., 2015; Wesileder et al., 2018) and social development (Cabrera et al., 2017; Colwell & Lindsey, 2005; Vally et al., 2015). The current study suggests that a long-lasting, relationally-driven home visitation program can improve positive parenting behaviors and this change can happen even in a pandemic.

Methodological Discussion

One novelty of the present study is the use of a naturalistic setting for data collection and program implementation. This stands in contrast to the existing studies on community-based intervention efficacy, which utilize randomized or cluster randomized control trials. While these approaches are methodologically rigorous, they do not mimic the budgetary and staffing constraints of community interventions, which recruit on a rolling basis and do not select participants.

Accounting for individual variability is paramount, especially in low resourced interventions where programs do not have the funds to incentivize recruitment of similar people. HLM is very relevant for rolling admissions, where the baseline (intercept) and the speed of improvement (slope) will vary between each unit. The model presented in this manuscript does not presume anything about the consistency of the individual families or their start and stop time. Lastly, the current study utilized regression discontinuity plots as a way to explore the effect of the initial pandemic-related shutdown on variables of interest, as will be the case in the growing number of longitudinal studies addressing these breaks in data and life experiences.

Limitations & Future Directions

This paper presents data from an ongoing ECD intervention that has longitudinal data from 157 households between 2019–2021. This intervention has slowly established itself in the community gaining the trust of caregivers and increasing enrollment every year. It is important to remember that participation in the intervention is a self-selecting process, thus vulnerable to an ascertainment bias, and results should be interpreted accordingly. The results around the effects of the pandemic and the patterns of ECD data after the shutdown are exploratory in nature. Engagement in the monthly ECD gatherings (Fundanathis) were subject to many variables that were not measured, such as weather, ages of children and timing of their naps, how far it was from each household. Additionally, this study did not gather much demographic data such as the age of the caregiver, financial stability or mental health. However, the nature of the statistical analysis does not presume consistency and is flexible for individual differences of baseline and change patterns.

Lastly, the measure of hope was psychometrically tenuous, including only two items with low internal consistency. It was included because it significantly improved the model fit even after accounting for caregiver engagement, visit number, opportunities attended, and social support. Thus, future work should construct a stronger measure to illuminate the role of hopeful expectations in caregiver reading and playing practices. Future research should expand partnerships between academia and existing local interventions to create symbiotic relationships providing high quality data-analysis, useful programmatic feedback, and valuable translational data on how to best promote high quality, low cost ECD interventions.

Conclusion

This translational study elucidates the importance of relationally-based interventions that focus on implementing high-impact ECD practices over one to two years of continual support. Caregiver engagement was the strongest predictor of reading and playing behaviors, followed by the number of visits received, and the caregivers’ felt support system. Engaging in additional opportunities outside the home visitation system and reporting high levels of hope in their child’s future also significantly added to the model. This study shows data prior to and throughout the pandemic, indicating that it is possible for ECD programs to continue and even improve despite the global crisis. All told, this study provides field evidence of the importance of caregiver ownership and engagement, and that real-world ECD improvements come through supporting caregivers. OSF repository: https://osf.io/4x2hu/?view_only=8e50bbfd2fa94d078c7f16718388926f