Introduction

The COVID-19 pandemic overwhelmed healthcare systems disrupting healthcare for patients with chronic diseases, including cancer [1]. Nearly 80% of patients in active cancer treatment reported a delay in healthcare [2]. Persons with cancer had a higher risk of developing serious COVID-19-related complications, including admission to a hospital or intensive care unit (ICU), and even death [3, 4]. Additionally, the pandemic struck the Hispanic community with significant force. Through February 2022, Hispanics had elevated COVID-19 incidence and mortality rates, compared to other minority populations in the United States (US) [5]. When compared to non-Hispanic whites, Hispanics had higher rates of hospitalization and ventilator use [6].

Despite strained access to healthcare, the pandemic presented an opportunity to utilize technology for remote cancer education of people affected by cancer [7]. Educational webinars provided an opportunity to reach diverse audiences in disbursed geographic locations. Participants could use smart phones, tablets, or computers to learn from subject matter experts who were in distant locations and health systems. Furthermore, webinars helped mitigate patient barriers to healthcare, such as lack of mobility or time, travel distance to healthcare, lack of funds, or long waiting times for appointments [7].

However, educational webinars often lacked an effective method to evaluate participant experience and outcomes [8]. Indeed, few studies have focused upon evaluation of cancer-related webinars. For example, one study assessed the attitudes of cancer patients towards vaccinations solely through pre- and post-webinar questions focused upon intent to receive a COVID-19 vaccine [9]. While these data may be informative, they did not evaluate important educational dimensions, such as reach and participant engagement [10].

Evaluation frameworks, such as RE-AIM, have been developed to evaluate community-based interventions and dissemination, including reach and engagement [11]. To our knowledge, RE-AIM has not been used as a framework to evaluate a cancer-related Spanish webinar series. Therefore, the objectives of this study were to (1) apply the RE-AIM model to evaluate a longitudinal webinar series in Spanish and (2) use these findings to inform future educational webinars.

Methods

The Penn State Cancer Institute (PSCI) is an academic cancer center in central Pennsylvania that fosters community outreach and engagement to reduce cancer burden and disparities in its 28-county catchment area. The PSCI established the Hispanic/Latino Community Cancer Advisory Board (CAB) in 2018 to be a bridge between the Hispanic community in central Pennsylvania and the PSCI. The CAB, which includes the Spanish American Civic Association (SACA) with its radio station WLCH Radio Centro, initiated “Vamos a educarnos contra el cáncer” (English Translation: Let’s get educated against cancer) webinar series in 2021 as a response to the pandemic’s disruption in cancer prevention, control, and care. SACA had approximately 1000 followers on social media. The webinar series was also considered to be a method to increase participation and engagement within the CAB, as depicted by a model adapted from Oh and Sundar (Fig. 1) [8].

Fig. 1
figure 1

Webinar series model. To increase participation and engagement, we used a model adapted from Oh and Sundar. Community-based organizations provide input and work together with the Penn State Cancer Institute and speakers to deliver the webinar. Within the webinar, there is content which is tested with pre- and post-questionnaires, ensuring good technology for participants to maintain interest and attention during the webinar. The outcomes are to have strong engagement, increased knowledge, and self-efficacy. The impact is that participants disseminate the webinar by interacting with others, as well as taking personal action to improve their health. Dissemination will provide feedback to increase the input for future webinars.

Full size image

Webinars

The CAB, in collaboration with the PSCI, identified topics and Spanish-speaking subject matter experts for monthly webinars. Webinars used the secure video conferencing platform, ZoomTM. Topics included patient navigation, palliative care, spiritual support for cancer patients, COVID-19 vaccination, and cancer, as well as the prevention of colorectal, breast, liver, and HPV-related cancers (Supplemental Table). All 12 webinars were open to the public and conducted at 6 PM on the 2nd Tuesday of each month between January 2021 and December 2021.

Two to four weeks prior to the scheduled webinar, a study team member reviewed the webinar objectives and the clinical content with the speaker(s). Technical requirements, such as Zoom link, FacebookTM live streaming functions, and poll features, were also tested. A promotional flyer tailored to each webinar was posted on the SACA Facebook page and a radio spot run on Radio Centro. Promotional ads were distributed to community partners, institutional websites, and social media platforms. Additionally, a resource document was developed by the study team for distribution to webinar participants and for later posting to websites.

Each webinar was 60 min. in length, composed of a 9-min. opening period for introductions and poll administration, followed by a 20- to 30-min. educational presentation and a 20-min. session for questions and a final poll. All webinars were video- and audio-recorded and later posted on the CAB’s public website.

Evaluation

Evaluation was guided by the RE-AIM framework of reach, effectiveness, adoption, implementation, and maintenance. Session-specific objectives were established prior to the launch of the webinar series and guided the evaluation. Data were collected through three separate anonymous and optional polls launched at different time points during each webinar, as well as data from social media platforms. Each of the three polls consisted of 10 questions, written in Spanish. The polls were answered only by individuals who connected through Zoom, not by those who viewed the webinar through Facebook live or the recordings. Evaluation also included document review and anecdotal comments from participants.

Reach was defined as the (1) session attendance (i.e., the number of persons who viewed the webinar through WLCH Radio Centro’s Facebook live or connected to Zoom using the meeting link on their own device or on a shared a device) and (2) the number who viewed webinar-related material prior to or after the webinar on social media platforms of Facebook and YouTube. The objective of 240 (20 per session) attendees per session was chosen prior to initiation of the series based on a pilot webinar prior to the series. We also monitored the number of engagements, such as likes, shares, and comments, with the seminar materials and recordings. Effectiveness was defined as participants’ evaluation during the polls of each webinar. Participants were asked, “Overall, how would you evaluate this webinar?” Also, participants’ comments written in the Zoom chat or on the Facebook live stream during the webinar were used to evaluate effectiveness. In addition, participants were asked about technology performance for effectiveness.

Adoption was defined as the participant’s response to the following questions, “As a result of this webinar, what actions do you plan to take for your own health?” and “As a result of this webinar, what actions for someone else’s health do you plan to take?” Implementation was defined as participant’s level of engagement during webinar. Participants were asked the following question, “How engaged did you feel during this webinar?” Implementation was also measured by participants’ intention to share the knowledge learned in webinars by talking to others (e.g., friends, coworkers, family members), as well as intention to take personal action to improve their health. Lastly, maintenance was defined by institutional and community commitment and new resources to continue the webinar series after the initial period of 2021.

Data from the polls were extracted from the Zoom platform. Data analysis used the SAS software© 2014. For categorical data, frequencies and percentages were calculated. Participant comments and feedback written in the session chat and comments from partners were also recorded as qualitative data. This study was approved by the Penn State College of Medicine’s Internal Review Board (STUDY00016788) and The Human Research Protection Program.

Results

For reach, session attendance was 297 live views (Table 2), exceeding the pre-established objective of 240. In total, 23% of attendees completed the demographic survey and 29% completed the evaluation polls. Most respondents reported being female and Latino(a)/Hispanic and most participants had a family member or friend diagnosed with cancer (Table 1). A majority reported speaking both Spanish and English. Almost 50% viewed the webinar from beyond the PSCI catchment area. Within the catchment area, 30% were from Lancaster County, 10% from Dauphin County, 5% from Berks County, 4% from Lebanon County, and 3% from York County. In addition to live views, participants were reached through promotional flyers posted on the radio station’s Facebook page four-weeks prior to each webinar. Pre- and post-webinar reach totaled 11,111. Also, 750 people engaged with the promotional flyer and 1003 with the post-webinar recording.

Table 1 Socio-demographic characteristics of participants in “Vamos a educarnos contra el cáncer” webinar series 2021, Penn State Cancer Institute
Full size table

Overall, 90% of the respondents rated the effectiveness of the sessions as good or excellent (Table 2). In addition, participants shared positive feedback through the Zoom chat. The following comments reflect upon the effectiveness of the webinars:

Excellent seminar! Thank you for [discussing] such an important topic.

This [webinar] is spectacular.

Thank you for everything. [The webinar is] very educational and informative...

Table 2 RE-AIM evaluation of “Vamos a educarnos contra el cáncer” webinar series 2021, Penn State Cancer Institute
Full size table

For adoption, participants were asked about their personal planned actions as well as planned actions for others. Overall, 86% of respondents planned to adopt or improve a cancer-related behavior, while 90% reported planning to improve someone else’s cancer-related action (Fig. 2). During the webinar on mental health among cancer patients, attendees were asked to participate in a mindfulness exercise. One of the participants shared the following comment about the exercise:

The mindfulness exercise from yesterday was very interesting. I really enjoy the webinars because [there is] new information and we only have access to this information only by you and your webinars.

Fig. 2
figure 2

Results on adoption. Overall percentage of participants who planned actions for self and others (n = 85) from all of the webinars. *Includes prostate, cervical, breast, and colon cancer screening

Full size image

Furthermore, the implementation dimension was measured as participants’ level of engagement during the webinar. Overall, 92% of respondents reported feeling engaged during the webinar series. However, one technical problem of posting on Facebook live was encountered during the first two webinars but resolved for the final 10 sessions. We partnered with community-based organizations to implement the webinar series; one comment was:

This is fabulous work and the partnership with SACA and Radio Centro have made a huge difference and impact on penetration in the community.

Finally, maintenance was supported through the development of an operations manual to be used by community leaders and organizations for a transition of the educational webinars to community ownership. In addition, a participant commented specifically on the usefulness of the resource documents:

... a resource sheet is exactly what we need in our Latino community, especially one that we can understand

Discussion

We found that “Vamos a educarnos contra el cáncer” webinar series had a positive outcome in each dimension of the RE-AIM model. We exceeded our target for live views and as of March 2023, there were 3382 views of the recordings on Facebook and YouTube. A majority of participants reported the webinars engaging and influential in their plans to improve their own and others’ cancer-related behaviors. In addition, the study was community-based and created resources which have formed a foundation for the maintenance of the webinar series. These results suggest that overall, our monthly Spanish webinars had a positive impact on Hispanic people affected by cancer.

Although previous studies have shown webinars reach new audiences and disseminate educational material [7], few studies have evaluated webinars [9, 10], and to our knowledge, none has used the RE-AIM framework. Evaluation is an essential tool for gathering evidence on how well webinar series function and produce positive outcomes [12, 13]. Our evaluation examined the reach, effectiveness, adoption, implementation, and maintenance of the series in a systematic approach through the RE-AIM framework. This model provided an excellent component-based framework to evaluate various aspects of the webinar series. Our findings highlight the importance of systematically planning, delivering, and evaluating longitudinal cancer educational webinars for Hispanic/Latino communities.

With the growing demand of online education, our evaluation results support a standardized, longitudinal webinar series as an acceptable and effective alternative to in-person education. Not only did the number of attendees exceed our objectives, the webinar series had participants from a wide geographic area; approximately half of the participants were from geographic locations outside of Pennsylvania. This approach represents cost-savings in transportation, speaker fees, overnight accommodations, printed materials, meals/food expenses, and facility rental. Additionally, participants exhibited a high level of satisfaction with the modality as evidenced by an overall rating of 90% good or excellent. To enhance standardization and sustainability, the study team created an operations manual intended to be used by community organizations to guide future cancer educational webinars.

A few limitations should be recognized. First, not all webinar participants completed the evaluation surveys. This may be due to technological challenges. For example, if a participant joined through Facebook live, they would be unable to participate in the Zoom polls. Second, the majority of participants were female (71%); thus, it is possible these results may not be generalizable to males. Third, we were unable to determine the number of people who attended more than one webinar because the webinars did not collect personally identifying information. Despite these limitations, there are several strengths: (1) utilization of the RE-AIM model as an evaluation framework; (2) being community-based which helped assure acceptability and maintenance; (3) being culturally appropriate for Spanish speakers.

Conclusion

In summary, “Vamos a educarnos contra el cáncer” webinar series had positive reach, effectiveness, adoption, implementation, and maintenance. Most participants reported the webinars engaging, content satisfying, and influential in their willingness to share what they learned with others, and to adopt or improve the cancer-related behaviors promoted. Furthermore, high numbers of people who attended the webinar and recording views demonstrate the interest of the Hispanic/Latino population in cancer-related topics. The webinar series was a novel approach to planning, delivering, and evaluating health education webinars in a culturally appropriate manner. Lastly, our novel evaluation process can serve as a model for future health education webinars.