Introduction

The unprecedented global impact of COVID-19 led to the rapid development and deployment of vaccines against the virus, including vaccines using novel mRNA technology. However, despite the promising safety profile and effectiveness of mRNA vaccines in preventing severe outcomes of COVID-19, there has been a notable global public reluctance to be vaccinated1,2. For example, in the United States, nearly 20% of the population has received no COVID-19 vaccine doses3. As of May 4, 2024, 27.7% of Israel’s population had not received any COVID-19 vaccine. Among the elderly aged 65 and above, the coverage rates for the first and second COVID-19 boosters were 82.6% and 47.8%, respectively. Influenza vaccine coverage was 15.3% nationwide, while 52.3%–58.6% of the elderly population had been vaccinated against influenza4,5. A key reason for COVID-19 vaccine hesitancy is concerns about the safety of the vaccine6,7.

One strategy to address such concerns is by comparing the side effects of the mRNA vaccines to the side effects of the seasonal influenza vaccine, which has been administered for decades with a solid safety record and a high level of public acceptance. Hundreds of millions of vaccines for seasonal influenza and COVID-19 have been administered to date8,9 but information comparing the safety of these vaccines is limited.

Currently, information on vaccine side effects is primarily collected through self-reporting methods, but these methods may be subject to bias and underreporting. To address this issue and provide a more comprehensive assessment of vaccine safety, extensive, continuous, and detailed monitoring of physiological changes in vaccinated individuals is necessary. Wearable devices, such as smartwatches, offer a promising solution. They enable continuous, detailed monitoring of physiological changes in vaccinated individuals, which can help identify vaccine-associated adverse events more effectively than self-reporting alone.

Wearable sensors have been shown to detect subtle medical conditions, such as atrial fibrillation, based on irregular heartbeats10. Several studies have shown that heart metrics, including heart rate, heart rate variability, and resting heart rate, can indicate COVID-19 infection in the pre-symptomatic stage and thus can be used for real-time detection11,12,13. These heart metrics have been previously reported to correlate with subjective symptoms after the COVID-19 vaccine14,15. In the context of COVID-19 vaccination, a number of studies using wearables have observed short-term changes in heart metrics following vaccination, even when such changes were not apparent to patients when self-reporting effects of vaccination16,17,18,19,20.

In this study, our primary objective is to assess the safety profile of mRNA COVID-19 booster vaccines compared to seasonal influenza vaccines. To achieve this, we leverage both prospective and retrospective data sources. The prospective data include self-reported information and physiological measurements captured by Garmin Vivosmart 4 smartwatches, while the retrospective data are derived from electronic health records (EHRs). We focus on evaluating short-term (up to 42 days) self-reported and biomarker reactions following the administration of the BNT162b2 (Pfizer-BioNTech) mRNA COVID-19 booster vaccine, and compare these with reactions observed after the seasonal influenza vaccine. Our analysis is based on a prospective cohort comprising 5079 participants enrolled in an observational trial. To further investigate potential short-term severe adverse events within a 28-day window post-vaccination, we also examine a retrospective cohort comprising 250,000 randomly selected members from Maccabi Healthcare Services (MHS), one of Israel’s largest healthcare providers, serving ~25% of the population. Through this comprehensive approach, we aim to provide a thorough comparison of the safety profiles of both vaccine types.

Methods

Cohorts

Prospective cohort

We studied a cohort of 5079 participants from a prospective observational trial of individuals in the PerMed study16,17,18,19,21 who received a third or fourth BNT162b2 mRNA COVID-19 vaccine and a seasonal influenza vaccine between July 31, 2021 and September 30, 2022 (Fig. 1a, b; see Supplementary Note 1—study protocol, pp. 2–12). Participants filled out a daily questionnaire about clinical symptoms and wore a smartwatch that measured, among other factors, heart rate and heart rate variability-based stress.

Fig. 1: Trial profile.
figure 1

a, b Prospective study, c retrospective study.

Full size image

Upon enrollment in the study, we gathered information on the participants’ sex, age, and pre-existing clinical risk factors. These underlying medical conditions included diabetes, hypertension, heart disease, chronic pulmonary conditions, weakened immune system, cancer, kidney failure, and a body mass index (BMI) >30 (BMI is determined by dividing a person’s weight in kilograms by the square of their height in meters).

We paired observations so that we could compare vaccine reactions in each individual. Among the 5079 participants in the prospective cohort, a total of 4334 COVID-19 vaccine doses and 2639 influenza vaccine doses were administered. For both types of vaccines, we had hourly smartwatch data on heart rate and a Garmin-computed stress measure based on heart rate variability (heart rate: 1877 COVID-19 vaccine doses, 901 influenza vaccine doses; stress measure: 1845 COVID-19 vaccine doses, 878 influenza vaccine doses) (see Supplementary Table 2). From these observations we extracted paired vaccine observations; these are observations for individuals who received both the COVID-19 vaccine (first or second booster) and at least one influenza vaccine.

Retrospective cohort

For the retrospective cohort, we examined anonymized EHRs of 250,000 randomly selected members of MHS (Fig. 1c; see Supplementary Note 1—study protocol, pp. 2–12). Eligible members were individuals 18 years and older who received both a third or fourth BNT162b2 mRNA COVID-19 vaccine and a seasonal influenza vaccine between July 31, 2021 and March 1, 2023. Individuals who were not members of MHS throughout the entire study period were excluded.

Study design

Prospective cohort

For the prospective cohort, participants were asked to complete a daily survey via the PerMed mobile application21. This survey compiled self-reported clinical symptoms from participants, using a predefined list of reactions observed in the BNT162b2 mRNA COVID-19 vaccine trial22, while also allowing participants to freely report any additional symptoms they experienced. The survey was crafted based on potential symptoms that may follow infections with infectious diseases and respiratory illnesses, drawing on the International Classification of Diseases, Ninth Revision, and Clinical Modification (ICD-9) codes related to influenza, influenza-like illnesses, acute respiratory infections, respiratory syncytial virus (RSV), group A streptococcus, and COVID-19.

Throughout the study, from the moment they were recruited until the end, participants wore a Garmin Vivosmart 4 smartwatch. The smartwatch data were used to estimate the effects of vaccination on physiological measures such as heart rate and stress levels. Stress levels, computed by Garmin, range from 1 to 100 and are classified into four categories: resting (1–25), low (26–50), medium (51–75), and high (76–100)23. A higher stress level correlates with lower heart rate variability24,25. Heart rate data (beats per minute) were collected in intervals of 15 s while stress measurements were documented every 3 min.

To minimize participant attrition and discomfort, several preventive measures were implemented, thereby enhancing the quality, continuity, and reliability of the collected data. First, participants who did not complete their daily questionnaire by 7:00 p.m. received a reminder through the PerMed application. Second, a dedicated dashboard allowed the survey company to identify participants who repeatedly failed to complete the daily questionnaire or wear their smartwatch. These participants were contacted via text or phone call and encouraged to adhere to the study protocol. Third, to engage participants, a weekly summary report was provided in the PerMed application, and a monthly newsletter featuring recent findings and smartwatch tips was sent out. At the conclusion of the 2-year study, participants received all their insights and were gifted the smartwatch.

Further information regarding the recruitment procedure, choice of smartwatch data analyzed, data collection architecture, and PerMed dashboard is provided in the Supplementary Materials (see Supplementary Notes 1–3) and elsewhere18.

Retrospective cohort

For the retrospective cohort, we examined anonymized EHRs of the patients. These records are automatically collected from various clinics and medical facilities nationwide and updated monthly in MHS databases. The data are coded, anonymized, viewed, stored, and processed within the research room of the MHS. MHS uses the ICD-9 classification with procedures coded using Current Procedural Terminology codes. We obtained demographic details for each patient, along with diagnoses linked to 28 possible adverse events, as specified by ICD-9 codes.

Outcomes and statistical analysis

Prospective cohort

Though the majority of participants only filled out the survey once daily, if there were multiple entries by a single participant within a day, only the final entry was considered. Since questionnaires cannot be updated after submission, participants were instructed to submit a new response in case of filing errors; we assumed that the last entry provided a more accurate representation of the participant’s entire day.

We defined a “baseline period” for each participant using their data (self-reported questionnaire and smartwatch) 7 days prior to vaccination. A participant’s “baseline” refers to the last questionnaire they submitted during the baseline period and smartwatch data for the entire baseline period. If a symptom was reported following the vaccination and was not noted during the baseline period, we interpreted this to be a side effect of the vaccine. Participants who did not complete the questionnaire during the baseline period were omitted from our analysis because we could not determine whether their symptoms existed prior to vaccination. We compared the baseline period to 7 and 14 days after vaccination inclusive of vaccination day (“post-vaccination” period), for discrete and continuous metrics, respectively. Since the US Centers for Disease Control and Prevention (CDC) advises that vaccine side effects typically disappear after a few days, our analysis concentrates on the first 72 h post-vaccination26.

We included participants who (1) submitted the questionnaire at least once during the baseline period, (2) completed the questionnaire at least once within 72 h after vaccination, (3) provided wearable device measurements during the same day-of-week and time-of-day during their post-vaccination and baseline periods, and (4) received both the seasonal influenza and COVID-19 vaccine.

We excluded participants who provided data for only 1 day (either during the baseline or post-vaccination periods), with an exception if the participant provided data for the same hours-of-day and days-of-week during the baseline and post-vaccination periods.

We differentiated participants based on their self-reported intensity of symptoms as recorded in a questionnaire during the 72 h following each vaccination. Participants were grouped into “No Reaction,” “Mild Reaction,” or “Severe Reaction” categories, based on their most severe symptom reported in the 72 h following each vaccination. Consequently, if a participant experienced a single severe symptom for a day, and mild symptoms for the remaining 3 days post-vaccination, they were designated as having a severe reaction. The severity categorization could vary for participants after each administered vaccine dose.

In alignment with the CDC27 and the Pfizer clinical trial22, we categorized symptoms as follows:

Mild symptoms: abdominal pain, feeling hot, back or neck pain, feeling cold, muscle pain, weakness, headache, dizziness, vomiting, sore throat, diarrhea, cough, leg pain, ear pain, loss of taste and smell, swelling of the lymph nodes, fast heartbeat, and hypertension; severe symptoms: chest pain, dyspnea (shortness of breath), fever, confusion, and chills.

For participants who reported feeling hot and recorded their temperature, we divided them into two categories: above 38.9 °C (fever) or below 38.9 °C (feeling hot); in cases where the participant did not record their temperature, we assumed it to be below 38.9 °C.

From the questionnaire data, we computed the proportion and corresponding 95% confidence interval (CI) of participants who reported experiencing each side effect in the post-vaccination period. The 95% CI for each side effect was calculated using a binomial distribution Binom(n,p), where p = the proportion of participants reporting the symptom, and n = number of participants eligible to be included in the self-reported questionnaire analysis. In cases where participants received more than one dose of the COVID-19 or the seasonal influenza vaccine during the study period, and were eligible to be included for both inoculations, each self-reported event (i.e., specific symptoms or no symptoms) was considered as:

$${{{{\rm{number}}}}\; {{{\rm{of}}}}\; {{{\rm{events}}}}}=\frac{1}{{{{\rm{number}}}}\; {{{\rm{of}}}}\; {{{\rm{vaccine}}}}\; {{{\rm{episodes}}}}}\left[{{{\rm{event}}}}\right]$$
(1)

to avoid overcounting.

For the continuous Garmin smartwatch data, we compared measurements from the post-vaccination period to the corresponding day-of-week and hour-of-day measurements from the baseline period at the individual level. For example, we compare a participant’s Wednesday average 9 a.m. heart rate with their previous Wednesday 9 a.m. heart rate. If these data were not recorded (e.g., if a participant did not wear the smartwatch in the same period before and after vaccination), we excluded the participant from this analysis. Then, we aggregated each hour’s differences divided by all participants to calculate a mean difference and the associated 95% CI. We present this analysis starting a week before and after vaccination. In cases where participants received more than one dose of the COVID-19 or the seasonal influenza vaccines during the study period, and were eligible to be included for both inoculations, we calculated the average differences over all inoculations.

We also compared the difference of physiological changes between COVID-19 vaccination and seasonal influenza vaccination among individuals who received both vaccines. For each participant, we first calculated the daily mean changes in heart rate between the post-vaccination period and the baseline period. We did this separately for the mRNA COVID-19 booster vaccine and the seasonal influenza vaccine. Then we calculated the difference between these two mean values for each participant and each of the 7 days after inoculation. This is equivalent to a two-sided Welch’s t-test, which does not assume equal population variance.

Retrospective cohort

For each individual in the retrospective cohort who received both vaccines, we noted the existence in the EHRs of 28 potential adverse events (see Supplementary Table 1) during the 28-day period before and after each vaccination. This set of adverse events was composed based on a previous large-scale study that examined the safety of the first and second (primary series) COVID-19 vaccine doses28.

We calculated the number of adverse events shown in patients’ EHRs before and after each vaccine. Consistent with a previous study29, we chose a time interval of 28 days to evaluate the potential short-term effects of each of the vaccines separately. Similar to large cohort studies18,30, we evaluated the risk differences using a self-as-control method that compared adverse events in the same patient in two periods: a baseline period of 28 days (from 35 to 8 days before vaccination) and a post-vaccination period of 28 days (0–27 days after vaccination). We used a 7-day buffer period between the baseline and post-vaccination periods in accordance with the guidelines of the Israeli Ministry of Health31, which recommend that an individual should not receive inoculation if suffering from any apparent infection with severe symptoms, including a fever higher than 38 °C. This 7-day buffer period was also consistent with the assumption made by a large-scale safety study in which non-vaccinated individuals were not eligible to serve as controls if the person experienced any health event in the week before the follow-up period28.

During outpatient visits in Maccabi’s clinics, past diagnoses also appear as part of the current visit’s diagnoses. Thus, in line with the previous study, for each adverse event, we omitted individuals who were previously diagnosed with the same event in the year prior to the baseline period28.

We conducted a pairwise comparison for each individual, calculating the risk difference between pre- and post-vaccination values, which we denote \({Y}_{{diff}}^{i,j}\) for person i and event j. If an adverse event was recorded in the individual’s EHRs in the post-vaccination period (i.e., 0–27 days post-vaccination) but not in the pre-vaccination period (35–8 days pre-vaccination), then the event is potentially associated with the vaccine or a random event, and we set:

$${Y}_{{diff}}^{i,j}=1$$
(2)

If the converse was true and an adverse event appeared before vaccination but not after vaccination, the event is potentially a random event, and we set:

$${Y}_{{diff}}^{i,j}=-1$$
(3)

If a specific event was reported in both the pre- and post-vaccination periods, we assumed the event is not associated with the vaccine, and we set:

$${Y}_{{diff}}^{i,j}=0$$
(4)

and thus the individual was excluded from the analysis of event j. For a fair comparison between the baseline and the post-vaccination period, if an individual was found to be positive for COVID-19 during the post-vaccination period, we compared only the events recorded in the period between the inoculation and the week prior to the positive test date and matched this period with the same time interval in the baseline period. Likewise, as individual cannot die during the baseline period but can die during the post-vaccination period, if an individual died during the post-vaccination period, we compared only the events recorded in the period between the inoculation and the recorded time of death and matched this period with the same time interval in the baseline period.

The risk difference for event \(j\) is the mean value of \({\bar{Y}}_{{diff}}^{\cdot ,j}\) calculated over all vaccinated individuals. This approach mirrors the standard estimation of risk differences in exposed and unexposed groups32, while taking into account the paired nature of the samples. To calculate the 95% CI for the difference without imposing any unknown distribution, we applied a non-parametric percentile bootstrap method with 10,000 repetitions, similar to previous safety studies28,30. In case an individual received more than one inoculation of COVID-19 vaccine or more than one inoculation of the seasonal influenza vaccine, for each repetition we chose randomly one of the inoculations for each vaccine type.

Additionally, we assessed the relative risk between COVID-19 and influenza vaccines by analyzing the incidence of adverse events over a 28-day period following each vaccination. We employed a non-parametric percentile bootstrap method with 10,000 repetitions to calculate the 95% CI. We repeated the aforementioned analyses for hospitalizations due to adverse events, stratified by age group.

Ethical approval

The prospective study was approved by MHS’ Helsinki Institutional Review Board, protocol number 0122-20-MHS. All participants gave written informed consent to participate in the study and were advised both orally and in writing of the nature of the study. This study is part of a larger study (an observational clinical trial funded by a European Research Council grant) and is in accordance with the European Union General Data Protection Regulation: a cohort of 5000 participants are recruited, download a dedicated mobile application, receive smartwatches, grant access to their medical records, and are followed for 2 years. Since the retrospective data were pseudonymized, the Helsinki Institutional Review Board approved the use of this cohort data without requiring specific consent from MHS members (protocol number 0122-20-MHS).

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Results

Prospective cohort

Cohort characteristics

A total of 2115 participants received either the COVID-19 vaccine or the influenza vaccine during the study period (Table 1). Among these participants, 1905 received the COVID-19 vaccine, while 899 received the influenza vaccine. In the paired sample of 689 participants receiving both vaccines, 355 (51.5%) were female, and age ranged from 20 to 85 years, with a median of 58. This is significantly higher than the median age in Israel, which is 30.5 years33. Among the 689 participants, 357 (51.8%) reported that they had an underlying medical condition and 177 (25.7%) had a BMI of 30 or greater.

Table 1 Cohorts characteristics
Full size table

When we paired observations for individuals who received both COVID-19 and influenza vaccines, we obtained 799 paired COVID-19 vaccine doses and 692 paired influenza vaccine doses for heart rate; and 779 paired COVID-19 vaccine doses and 672 paired influenza vaccine doses for the stress measure (see Supplementary Table 2). These numbers are representative of 577 and 544 individuals with heart rate and stress data, respectively, who received both vaccines. Similarly, we had 881 paired daily self-reports on symptoms for 7 days pre- and post-COVID-19 vaccination and 727 pre- and post-influenza vaccination paired self-reports, corresponding to 621 individuals.

Questionnaire analysis

For the COVID-19 vaccine, 39.7% [95% CI 36.4%–42.9%] of individuals reported no systemic reaction; for the influenza vaccine, this fraction was 66.9% [95% CI 63.4%–70.3%] (Fig. 2a). The most common reactions reported for the COVID-19 vaccine were weakness (16.1% [95% CI 13.6%–18.5%] of vaccinated individuals), headache (13.1% [95% CI 10.9%–15.3%]), muscle pain (12.1% [95% CI 10.0%–14.3%]), fever (7.0% [95% CI 5.3%–8.7%]), and chills (5.7% [95% CI 4.2%–7.3%]). The most common reactions reported for the influenza vaccine were weakness (7.3% [95% CI 5.5%–9.2%]), headache (5.3% [95% CI 3.7%–7.0%]), muscle pain (3.6% [95% CI 2.3%–5.1%]), feeling cold (3.5% [95% CI 2.2%–5.0%]), and sore throat (3.0% [95% CI 1.8%–4.3%]).

Fig. 2: Extent of reported reactions following vaccination.
figure 2

a Reported symptoms following COVID-19 and influenza vaccination. Error bars represent 95% confidence intervals based on a binomial distribution. b Comparison of self-reported reaction severity for COVID-19 vaccination and influenza vaccination among 621 individuals receiving both vaccines: number and percent of individuals reporting various combinations of no reaction, mild reaction, and severe reaction to the COVID-19 vaccine and influenza vaccine.

Full size image

We compared the severity of symptoms for the 621 individuals who received both a COVID-19 and influenza vaccine and who self-reported symptoms for both (Fig. 2b). Using Spearman’s rank correlation test we found a weak, albeit significant, correlation in the severity of the symptoms for the two vaccines (r = 0.185, p value < 0.001). Specifically, individuals who reported a more severe reaction after influenza vaccination tended to likewise report a more severe reaction after COVID-19 vaccination. Of the 621 individuals, 278 (44.8%) reported no reaction to either vaccine, 173 (27.8%) reported no reaction to the influenza vaccine and a mild reaction to the COVID-19 vaccine, and 36 (5.8%) reported no reaction to the influenza vaccine and a severe reaction to the COVID-19 vaccine. Similarly, 58 individuals (9.3%) reported a mild reaction to both vaccines, 38 individuals (6.1%) reported a mild reaction to the influenza vaccine and no reaction to the COVID-19 vaccine, and 25 individuals (4.0%) reported a mild reaction to the influenza vaccine and a severe reaction to the COVID-19 vaccine. Only five individuals (0.8%) reported a severe reaction to both vaccines. Of the 621 individuals, 321 (54.9%) reported the same level of symptom severity for both vaccines, and 259 (41.7%) reported a higher level of symptom severity for the COVID-19 vaccine than for the influenza vaccine. Only 46 individuals (7.4%) reported a lower level of symptom severity for the COVID-19 vaccine than for the influenza vaccine.

Wearables analysis

We examined mean pre- and post-vaccination differences in hourly heart rate and stress data for participants who received either the COVID-19 or influenza vaccine (Fig. 3). Following the administration of the COVID-19 vaccine, we identified a statistically significant increase in heart rate during the first 3 days compared to the baseline period. This increase peaked 22 h after vaccination, with a mean difference of 4.48 (95% CI 3.94–5.01) more beats per minute compared to baseline. By the 6th day post-inoculation, heart rate levels returned to baseline. A similar trend was observed for stress data: the increase in stress measure was statistically significant with a peak 22 h after vaccination and a mean increase of 9.34 (95% CI 8.31–10.37) units in the stress measure. For influenza vaccination, no statistically significant changes in heart rate or the stress measure were observed.

Fig. 3: Physiological reaction following vaccination.
figure 3

Mean difference in a, b heart rate (in beats per minute [bpm]) and c, d heart rate variability-based stress measure (in points) between the post-vaccination and baseline periods after a, c COVID-19 and b, d influenza vaccinations. Shaded regions represent 95% confidence intervals. bpm refers to beats per minute.

Full size image

To directly compare the effects of the two vaccines we conducted a pairwise comparative analysis in which we examined daily mean changes in heart rate and the stress measure for individuals receiving both vaccines (Fig. 4). For each such participant, we calculated the mean change in the indicator (either heart rate or stress) associated with COVID-19 vaccination compared to an individual’s baseline levels minus the mean change in the indicator associated with influenza vaccination compared to the individual’s baseline. Figure 4 shows that the increase in both heart rate and the stress measure for each participant was higher for COVID-19 vaccination than for influenza vaccination in the first 2 days after vaccination, and that this increase is statistically significant (using two-tail paired t-test: HR: p value = 0.0011, effect size = 0.2400; HRV: p value < 0.0001, effect size = 0.3247). However, these differences were small: for example, on the 2nd day after vaccination, mean heart rate was 1.5 (95% CI 0.68–2.20) beats per minute higher after COVID-19 vaccination than after influenza vaccination, compared to an individual’s baseline, and mean stress was 3.8 (95% CI 2.27–5.22) units higher. Moreover, these differences disappeared by the 3rd day after vaccination: for the third through 7th days after vaccination, there were no statistically significant differences in changes in these two indicators for the COVID-19 vaccine compared to the influenza vaccine.

Fig. 4: Paired analysis.
figure 4

Daily mean changes in the smartwatch indicators for heart rate and the stress measure. For each participant, this was calculated as the mean change in the indicator (either a heart rate or b heart rate variability-based stress measure) associated with COVID-19 vaccination compared to an individual’s baseline minus the mean change in the indicator associated with influenza vaccination. Error bars represent 95% confidence intervals. bpm refers to beats per minute.

Full size image

Retrospective cohort

Cohort characteristics

The retrospective cohort included 31,297 Maccabi members who received both vaccines during the study period. Of these individuals, 16,956 (54.2%) were female (Table 1). Age ranged from 12 to 103 years, with a median of 59. An underlying medical condition was listed in the EHR for 14,437 (45.8%) individuals, and 5069 (16.2%) had a BMI of 30 or greater.

EHR analysis

We found no elevated risk of hospitalizations for any of the 28 adverse events following the administration of either vaccine (Supplementary Data 1). Among the 31,297 individuals receiving both vaccines, 37 hospitalizations for adverse events occurred in the post-vaccination period compared to 51 in the baseline period for the COVID-19 vaccine and 32 hospitalizations occurred in the post-vaccination period compared to 30 in the baseline period for the influenza vaccine, corresponding to risk differences of −4.5 (95% CI: −10.2 to 1.3) and −0.6 (95% CI: −5.8 to 4.2) event/10,000 vaccinated individuals, respectively.

We found no statistically significant increase in the risk of the examined adverse events for COVID-19 vaccination (Supplementary Data 1). Influenza vaccination was associated with an increased risk of Bell’s palsy (1.3 [95% CI 0.3–2.6] additional events/10,000 people), but was not associated with any other increase in risk of the examined adverse events.

We examined the risk difference for adverse event hospitalizations stratified by age group and found no increase in such hospitalization risk after either vaccine across different age groups (Table 2).

Table 2 Adverse events following vaccination stratified by age group
Full size table

Discussion

Our analyses of data from patient questionnaires and smartwatches (in a prospective study) and from EHRs (in a retrospective study) comparing side effects of COVID-19 booster vaccination and seasonal influenza vaccination further support the safety of the first and second BNT162b2 booster vaccinations in eligible populations. In the patient questionnaires, more individuals reported mild or severe effects after the COVID-19 vaccine than after the influenza vaccine (60.3% vs. 33.1%). We found a weak but significant positive correlation in the severity of the symptoms for the two vaccines. From the smartwatch data, we identified a small but statistically significant increase in heart rate and the stress measure in the 3 days after COVID-19 vaccination, with levels returning to normal within 6 days after vaccination. For influenza vaccination, no statistically significant changes in these measures were observed. Although our paired analysis of reactions to COVID-19 and influenza vaccination suggested that the differences are significant, they were small and do not indicate lack of safety. Importantly, for the retrospective cohort, our analysis of EHRs—which are based on diagnoses of physicians in clinics and hospitals—identified no significant change in risk of adverse events after either vaccine, except for an elevated risk of Bell’s palsy after the influenza vaccine, a rare event. This finding is in line with a recent large-scale study that also found this positive association34. Despite this finding, the benefits of influenza vaccination in preventing severe outcomes far exceed this risk.

At both the aggregate level where we compared average changes in smartwatch measures pre- and post-vaccination for COVID-19 vs. influenza vaccination, and at the individual level where we compared each patient’s changes in these measures, we found an increase in heart rate: 4.48 (95% CI 3.94–5.01) beats per minute higher at the aggregate level, and 1.5 (95% CI 0.68–2.20) beats per minute higher 22 h after vaccination when we performed a pairwise comparison. Previous studies have found that even small long-lasting increases in heart rate are associated with an increased risk of death35,36. However, we found that the increases in heart rate disappeared within 6 days after vaccination. We also found increases in the heart rate variability-based stress measure. Some details are available on how the Garmin stress measure is calculated37,38, but the exact algorithm is proprietary and is not fully disclosed. Although the stress measures returned to baseline within 6 days after vaccination, the clinical significance of the short-term increases in the stress measure is unknown.

The integration of both retrospective and prospective data methodologies in our study enables a comprehensive understanding of the safety profiles of mRNA COVID-19 booster vaccines and seasonal influenza vaccines. Prospective data collection, including self-reported information and physiological measurements, provides real-time insights into participants’ experiences following vaccination, allowing for the detection of immediate reactions and short-term outcomes. This approach enables us to capture a detailed snapshot of individual-level responses, facilitating the identification of reactions with a high level of granularity. Retrospective data derived from EHRs offer a broader population-level perspective, encompassing a larger sample size. By analyzing historical healthcare data, we gain insights into the occurrence of adverse events over time, including those that may manifest beyond the immediate post-vaccination period. However, each data source has its limitations: prospective data collection may be susceptible to biases inherent in self-reporting and wearable devices, while retrospective data may be subject to incompleteness or inaccuracies in healthcare records. Integration of these complementary datasets helps to overcome individual methodological shortcomings and enhances the robustness of our findings: our prospective data analysis provides granular insights into short-term reactions, while the retrospective analysis offers a broader perspective on vaccine safety. The synergistic combination of the two approaches enables a comprehensive evaluation of vaccine safety that can guide evidence-based decision-making for public health policy and vaccination strategies.

Our analysis has several limitations. First, we considered only BNT162b2 (Pfizer-BioNTech) mRNA vaccines. Second, the median cohort age was greater than the median age in Israel, reflecting the fact that older individuals (55+ years age group) are more likely to receive both influenza vaccination and a COVID-19 booster shot than younger individuals (12-54 years age group). As we analyzed only individuals who got vaccinated in both vaccines, the total population vaccinated in each vaccine separately might be different. Third, the Garmin smartwatches are not medical-grade wearable devices, although previous studies have demonstrated the accuracy of smartwatches in measuring heart rates39,40 and our analysis considers relative heart rate values pre- and post-vaccination rather than absolute heart rate values.

The composition of the cohorts may affect the extent to which our findings can be generalized to the broader Israeli population and to other populations. The Participants in our prospective cohort were mainly recruited through social media advertisements and word-of-mouth referrals, resulting in a convenience sample. The study’s rigorous requirements, such as the continuous use of smartwatches and daily questionnaire completion over a 2-year period, may have discouraged certain segments of the population from participating. Moreover, the retrospective cohort comprised members of MHS and may not fully represent all ethnic minority groups. However, it is essential to acknowledge that we do not anticipate differential adverse reactions based on ethnicity41.

Another limitation involves the underreporting of short-term vaccine reactions in the EHR. There are several reasons why people might not report these reactions: mild or expected symptoms often go unreported; some individuals may not recognize that their symptoms are potential side effects of vaccination; others may attribute their symptoms to pre-existing health conditions or unrelated illnesses; and some have limited access to their general physician. The Israeli Ministry of Health addresses this gap by encouraging individuals to report any short-term reactions using a 3-min online survey42. Thus, individuals might find it sufficient to report adverse events via the online system rather than contacting their general physician. However, these data are not shared with the individual’s care provider (in this case, MHS). To mitigate this limitation in our retrospective cohort, we focused only on severe adverse events, which are more likely to be appear in the EHR because of individuals seeking care.

Finally, our analysis does not necessarily represent the entire population eligible to receive COVID-19 booster doses or seasonal influenza vaccine according to the Israeli Ministry of Health guidelines, only those who received them. For example, the observation that individuals who had a severe adverse response to the COVID-19 primary vaccine series or influenza vaccine (see reports on adverse events after the primary series28) might opt not to receive the COVID-19 first or the second boosters or the seasonal influenza vaccine and would therefore not be represented in our analysis.

Participants in the prospective study are also members of MHS, so future research could investigate whether the participants’ self-reported adverse events following immunization (AEFI) were documented in the EHR. This comparison could help quantify the extent of underreporting of AEFI, providing valuable insights into vaccine safety surveillance and potential strategies for improvement.

Our analysis demonstrates the power of examining patient data from multiple sources. Although more individuals experienced side effects after COVID-19 vaccination than after influenza vaccination, as reflected both in patient self-reports and in smartwatch data, the differences in examined side effects detected by the smartwatches were small, the differences disappeared by 3 days after vaccination, and heart measures returned to their pre-vaccination baseline within 6 days after vaccination. Moreover, no statistically significant increase in the risk of adverse events, as reflected in EHRs, was associated with COVID-19 vaccination. Taken together, these findings support the safety of the first and second BNT162b2 (Pfizer-BioNTech) mRNA COVID-19 booster shots.