Significance

In-person home visiting programs are known to improve maternal and child health outcomes and promote school readiness. However, the impact of conducting home visits through a virtual platform is unknown. This paper adds knowledge to the home visiting literature base of the impact of virtual home visiting on key programmatic outcomes.

Introduction and Background

Perinatal and early childhood home visiting (home visiting) programs are voluntary programs in which trained professionals enter the homes of families with young children and provide education and links to community resources (Howard & Brooks-Gunn, 2009). Evidence-based home visiting programs have been shown to improve maternal and child health outcomes, decrease child abuse and neglect, and promote school readiness (Duffee et al. 2017; Michalopoulos et al. 2017). Despite their positive impact, in-person home visiting programs are time- and resource-intensive, which can create barriers to participation for families (Foulon et al. 2015). Virtual service delivery can help mitigate these challenges for families (McGinty et al. 2006). Promising evidence shows that virtual delivery of home visiting programs is feasible and virtual home visits can adhere to model fidelity (Baggett et al. 2010; Breitenstein et al. 2013; Traube et al. 2020), but little is known about the programmatic impact of virtual home visits.

The Welcome Baby (WB) program, is a voluntary, universal home visiting program designed for families living in Los Angeles (LA) County. WB provides hospital and home-based services to pregnant women and women who have just given birth. The main goal of WB is to work with families to enhance the parent–child relationship, maximize the health, safety, and security of the baby, and to link families to support services when needed. WB is free to all persons giving birth or planning to give birth at one of 14 participating hospitals in LA County.

LA Best Babies Network (LABBN) is the oversight entity of the WB Program. LABBN provides the training, technical and programmatic assistance, and data and communications management for the 14 WB sites in LA County.

In March 2020, in response to the COVID-19 pandemic, all WB sites began providing home visits virtually, either by phone or via video conferencing. Extensive support in the form of virtual meetings, webinars, consult sessions with content experts, written protocols and guides, and resources was provided by LABBN to all WB program managers in order to assist them in shifting their home visiting services to a virtual format. To assess the impact of the COVID-19 pandemic and shift from in-person to virtual home visiting, we used a retrospective review of WB program engagement outcomes prior to and during the COVID-19 pandemic.

Methods

Setting

Welcome Baby is a voluntary home visiting program administered across 14 sites in Los Angeles County, serving approximately 12,000 clients annually. Clients enrolled in WB receive up to seven in-home visits at various points prenatally and postpartum, and one visit in the hospital at bedside after delivery. Clients are enrolled in the program prenatally through community outreach or postpartum in the hospital following birth. Table 1 displays the schedule of WB engagement points. At each engagement point, the home visitor provides education related to pregnancy, labor and birth, self and infant care, and bonding and attachment. Home visitors also provide breastfeeding education and support, perform a reputable screening to assess child development milestones, and screen clients for postpartum depression and refer to services if necessary.

Table 1 Schedule of Welcome Baby engagement points
Full size table

When enrolling in the program, clients provide written informed consent to allow visit data to be inputted into a local database managed by the LABBN Data Team, the Stronger Families Database (SFDB). After each home visit, WB staff input case data and notes into the SFDB. As the oversight entity, LABBN is able to view data reports populated by de-identified source data and analyze these reports for trends across all WB sites.

Program Engagement Outcomes

LABBN analyzed the impact of virtual home visits on the WB program engagement outcomes of: enrollment and outreach, visit service type, missed and completed visits, visit length, assessment completion, and overall program completion.

Analysis

LABBN reviewed reports with summarized data collected between March 2019 and December 2020 to assess programmatic outcomes within matching timeframes prior to the COVID-19 pandemic (March-December 2019) and during the COVID-19 pandemic (March-December 2020). Variables were compared using Microsoft Excel. Data was analyzed for frequency, distribution, and percentage change.

To track monthly prenatal enrollment (enrolled prenatal clients) and outreach numbers (referred clients) specific to each site, LABBN distributed a monthly survey to all WB program managers. Each manager responded with data specific to their site’s outcomes; reports were reviewed for trends. Completion rates for the LSP, GAD-7, and PHQ assessments were each measured by looking at the presence of the assessment completion date and dividing that number by the total number of assessments expected. LABBN measured program completion by looking at clients that ended service with status reason "completed program" and dividing that number by the total number of clients that ended service. LABBN measured average length of visit by summing the visit time in minutes of every completed visit and dividing by the total number of visits. The SFDB is subject to continuous data quality monitoring and an oversight team reviews data entry for inconsistencies.

All data used in this paper comes from de-identified reports. No patient data nor clinical study data was used; therefore, no ethics committee review was required.

Results

Figures 1 and 2 summarize changes in program engagement outcomes from pre-pandemic (March-December 2019) to pandemic (March-December 2020).

Fig. 1
figure 1

Pre-Pandemic vs. Pandemic engagement outcomes: prenatal enrollment, missed visits, completed visits, and length of visit average monthly values. Prenatal enrollment decreased by 50.9% from 2019 (pre-pandemic) to 2020 (pandemic). Missed visits increased by 5.4% from 2019 (pre-pandemic) to 2020 (pandemic). Completed visits increased by 10.4% from 2019 (pre-pandemic) to 2020 (pandemic). Length of visits decreased by 25.7% from 2019 (pre-pandemic) to 2020 (pandemic)

Full size image
Fig. 2
figure 2

Pre-Pandemic vs Pandemic engagement outcomes: LSP, GAD-7, PHQ, and overall program completion average monthly rates. LSP Completion increased by 0.4% from 2019 (pre-pandemic) to 2020 (pandemic). GAD-7 Completion decreased by 0.1% from 2019 (pre-pandemic) to 2020 (pandemic). PHQ-9 Completion decreased by 0.2% from 2019 (pre-pandemic) to 2020 (pandemic). Overall Program Completion decreased by 0.6% from 2019 (pre-pandemic) to 2020 (pandemic)

Full size image

Enrollment and Outreach

At the beginning of the COVID-19 pandemic, sites reported difficulties with conducting community outreach. Previously, WB outreach specialists would hold community events, give presentations at community centers, and work with local healthcare providers to refer families to WB. As the primary methods for getting clients enrolled in WB prenatally, the results of these efforts are reflected in each site’s monthly prenatal enrollment numbers. During the pandemic, nearly all sites reported drops in prenatal enrollment, compared to pre-pandemic. The monthly average number of clients enrolled prenatally decreased by 50.9% during the pandemic (pre-pandemic: 315.2, pandemic: 160.5). Without traditional outreach methods available, outreach specialists had to use novel methods for conducting outreach: through social media, virtual community presentations, and mailed newsletters. Despite some successes using novel recruitment methods, the virtual program created challenges with outreach and recruitment.

Visit Service Type

Prior to March 2020, the vast majority (over 95% on average) of home visits were in-person visits. On March 19, 2020, home visitors suspended in-person home visits due to the pandemic. Sites responded by conducting primarily phone visits for the remainder of March. Virtual home visits using HIPAA-approved video software began in April 2020.

The use of video for home visits increased steadily over time. In April 2020, only about 10% of visits conducted across all sites were done via video; by May, this percentage rose to closer to 30%, and to 44% by June. Starting in July and extending through December 2020, virtual visit completion was consistently over 50% and reached as high as 80% in December 2020.

All WB sites were encouraged to maintain a minimum video completion rate of 75%. The LABBN Technical Assistance Team supported sites in achieving and maintaining this rate through a Virtual Home Visiting Support Response (Bock et al. 2020).

Missed and Completed Visits

At the beginning of the pandemic, the average number of missed visits across sites dropped by roughly 30%. This drop in missed visits was sustained through the months of April and May 2020. During this time, home visiting programs were conducting the majority of home visits via phone versus virtually. WB staff reported that clients were feeling anxious and were looking for guidance and support. Additionally, during this period, LA County was under the strictest Safer-at-Home orders and unemployment rates were at their highest (Employment Development Department 2020). For these reasons, home visitors reported that clients were more apt to answer phone calls.

As businesses in LA County began phased reopening, the number of missed visits rose across all 14 WB sites and remained higher than seen before the onset of the COVID-19 pandemic. Pre-pandemic, the monthly average of missed visits was 1,086 visits; during the pandemic this average rose to 1,148 visits. Program managers reported that clients were under significant stress which was likely a factor in the higher rates of missed visits. Families reported feeling overwhelmed with managing the disruptions in their daily lives; parents were busy assisting children with their virtual schooling and often did not have additional time or devices to use to participate in virtual visits.

Despite the increase in missed visits, the number of completed home visits overall increased during the pandemic. The WB Program Framework encourages home visitors to complete 32–40 visits per month. This completion rate accounts for the time it takes to document the visit notes in the database, and for the time it takes to travel to clients’ homes. Prior to the COVID-19 pandemic, the average monthly visit completion rate across all WB sites was 2,811 visits. During the pandemic, this monthly average rose to 3,138 visits. Many factors influence the number of visits a home visitor can complete each month, and it is unclear exactly what led to the increase in completion rates during the pandemic. However, program managers attributed this increase in part to the fact that virtual home visits made traveling to clients’ homes unnecessary, and WB staff reported being able to complete more visits on average every week with their saved time.

Visit Length

Trends in the average duration of home visits also shifted during the COVID-19 pandemic. Prior to the pandemic, home visits lasted 72.9 min on average. When sites began conducting phone and virtual visits, the average duration of home visits across all WB sites dropped to 54.2 min. The length of a home visit appears to be impacted by delivery method (i.e. phone or video). In March and April, when visits were being conducted primarily by phone, the visits were the shortest; as the percentage of visits conducted via video increased, so too did the average length of visit. During the pandemic, 16% of visits conducted via video were less than 45 min, compared to 28% of phone visits that were less than 45 min. Qualitative data from program managers supports this observation; phone visits made it difficult to establish a personal connection with the client and clients were distracted and less engaged over the phone than they were on video, making them more likely to end the visit quickly.

Assessment Completion

Life Skills Progression

The Life Skills Progression (LSP) is a summary tool that home visitors use to gather and organize information about families’ competencies obtained from visits, screening tools, and observations of the family. The LSP is not intended to be administered via interview, parent self-report, or during a home visit. Instead, a home visitor scores the LSP items by considering in-depth information about the family that has been collected through referral information, interviews and conversation, observations of family functioning, formal assessments, and selected screening tools. This information is used to develop a profile of family strengths and needs, service plans, and monitor progress in outcomes.

The WB Program Framework requires home visitors to complete the LSP and record the results in the SFDB following the 28–38 week prenatal, 2-week postpartum, and 9-month postpartum home visits. The shift to virtual home visits did not affect the LSP completion rate (% completed out of the total number due). Prior to the pandemic, the average completion rate for all sites was 99.4%. During the pandemic, the average LSP completion rate for all sites was 99.8%.

PHQ-9 and GAD-7

WB home visitors are required to screen clients for prenatal/postpartum depression using the PHQ-9 questionnaire, and for anxiety using the Generalized Anxiety Disorder (GAD-7) questionnaire. According to the WB Program Framework, home visitors are required to administer the PHQ-9 at intake and during at least 95% of all subsequent engagement points. The GAD-7 is required to be administered at the 3rd trimester prenatal visit, the hospital visit, at 2 and 9 months of infant age for 90% of all clients.

Transitioning to virtual visits did not have an effect on the completion of PHQ-9 or the GAD-7 questionnaires. Across all WB sites, the average completion rate (% completed out of the total number due) for the PHQ-9 before the COVID-19 pandemic was 99.2%, and during the pandemic, the average completion rate was 99.0%. Similarly, the GAD-7 average completion rate was 98.7% before, and 98.6% during, the pandemic.

Managers requested support around maintaining client confidentiality and best practices for administering assessments virtually. LABBN provided significant technical assistance to WB program managers to help maintain assessment completion rates during the pandemic through a series of protocols, webinars, and consult sessions.

Overall Program Completion

Overall client completion of the WB program was not negatively affected by the pandemic. Program completion rate is determined by dividing the number of clients that attended the final engagement point visit by the total number of clients enrolled. This measure only includes clients who completed at least one postpartum home visit. The overall program completion rate prior to the COVID-19 pandemic was 68.0%. During the pandemic, the overall program completion rate was 67.4%. This indicates that clients were still engaging with the program during the pandemic to the extent seen prior to COVID-19.

Discussion

The COVID-19 Pandemic forced Welcome Baby sites to quickly start conducting home visits virtually, which had implications on the programmatic outcomes. Virtual WB home visits in LA County proved to be feasible and to provide some programmatic benefits.

Importantly, families that enroll in WB are typically considered low-risk, due to the eligibility requirements of the program. This is important for the generalizability of these findings, as other, in-person home visiting programs may serve high-risk populations. These findings are also limited to the impact of virtual delivery on the program engagement outcomes that we were able to analyze with existing data.

Despite challenges, WB home visitors were still able to reach families and provide services virtually. Even with initial difficulties with community outreach and a decrease in prenatal enrollment, completed visit rates rose across all WB sites with the introduction of virtual delivery. Additionally, the rates of assessment completion for questionnaires used to determine support needs for clients were not affected by virtual delivery. Finally, overall client program completion remained the same virtually. These findings suggest that virtual home visits are feasible, and the increased visit completion rate suggests that clients may be able to engage with the program more readily when the program is virtual.

However, the decrease in overall prenatal enrollment rates and the decrease in visit length seen with the shift to virtual visits may suggest that the overall quality of virtual home visits is not as favorable as the in-person visits. The question remains of whether virtual visits are as effective as in-person home visits and provide the same long-term benefits to families. Home visiting programs conducting virtual visits may need to focus efforts on improving outreach to new clients, establishing rapport between home visitors and their clients so as to increase visit length, and maintaining the quality of home visits. More research on the impact of virtual home visits, with regards to programmatic and client health outcomes, is needed.