Introduction

Vaccination is recognised as a major contributor to the improvement of public health and a notable achievement in medicine (Vasileiou et al., 2021). Globally, it remains one of the most cost-effective medical treatments, credited with controlling the effects of deadly diseases including the COVID-19 pandemic (Agrawal et al., 2021). The ongoing COVID-19 vaccination programme has led to a significant reduction in COVID-19 related infection rates, symptoms severity, and deaths (Agrawal et al., 2021). However, research has showed a significant rate of COVID-19 vaccine hesitancy, even in developed countries where vaccine uptake tended to be higher (Adu et al., 2023a; Guillon & Kergall, 2021). In the specific context of the current study in New Zealand, a concerning proportion (26%) of respondents from a representative sample indicated unwillingness to accept COVID-19 vaccination (Thaker, 2021), with trust in scientific experts serving as a buffer against COVID-19 vaccine hesitancy. As of February 7th, 2024, in New Zealand, the total confirmed COVID-19 cases stood at approximately 2.6 million, resulting in approximately 4000 deaths attributed to the pandemic, with daily reports documenting fatalities (Health New Zealand, 2024).

Typically, a mathematical model in New Zealand indicated that achieving a near 100% vaccination rate within the population is necessary to reach the population immunity threshold for COVID-19. Recognising the practical challenges associated with attaining this level of vaccination, the study recommended ongoing efforts to maintain and increase vaccination coverage. The emergence of new variants of COVID-19 also pose threat to achieving the global aim of transitioning to the endemic phase of the pandemic. Such challenges associated with the eradication of the COVID-19 pandemic necessitate the need for continuous research and developing efforts, including promoting COVID-19 vaccine uptake. This approach is crucial not only for safeguarding against COVID-19 but also for providing protection against potential emerging variants of the virus (Steyn et al., 2022). Thus, a timely examination of COVID-19 vaccination attitudes with New Zealand context-specific valid, reliable, and standardized instruments could enable better understanding and targeted strategies towards improving COVID-19 vaccination (Gjersing et al., 2010). The majority of the available studies assessed COVID-19 vaccination attitudes using a single-item scale, which is limited in reliability and susceptible to biases in meaning and interpretation due to its reduced ability to capture the diverse overarching aspects of the underlying construct (Loo, 2002; Prickett et al., 2021). On the other hand, a multi-item scale, the recent Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S) assesses diverse cognitive dimensions related to COVID-19 vaccine acceptance attitudes.

The MoVac-COVID19S employed in this study originates from the Motors of Influenza Vaccination Acceptance Scale (MoVac-Flu Scale) originally developed by Vallée-Tourangeau et al. (2018). Adaptation of the MoVac-COVID19S involved adjustments that replaced instances of “flu” with “COVID-19” (Chen et al., 2021), employing a Chinese sample for the validation process. This modification was necessary due to the similarities in symptoms and features between COVID-19 and influenza. COVID-19 can arguably be perceived as a more severe version of the flu (Czubak et al., 2021). This scale, grounded in the Cognitive motivation Empowerment (CME) model underscores the pivotal role of cognitive processes and motivation in driving behaviour. According to the CME model, behaviour is shaped by the interplay between cognition and motivation, with individuals’ perceptions of their control, abilities, and self-perceptions serving as influential factors (Thomas & Velthouse, 1990). This model, widely applied in studying health behaviours for decades, posits four cognitive processes that empower individuals toward a particular action: the value attributed to a specific behaviour, perception of the impact of an action, knowledge about an action, and the sense of autonomy over the behaviour. Bellali et al. (2023) reported that individuals who scored higher on the components of the CME model were more likely to accept influenza vaccination. Notably, the scale’s flexibility allows for future adjustments to accommodate emerging vaccination programs, underscoring its adaptability and relevance in addressing evolving public health challenges (Chen et al., 2021). So far, the one-factor model of the 9-item version of the MoVac-COVID19S has exhibited promising psychometric among Malay and Indonesian respondents (Pramukti et al., 2022). The four-factor structure of both the 9-item and 12-item structures also indicated better fit indices in India, Ghana, Taiwan, China, and Afghanistan compared to their one factor structures (Chen et al., 2021, 2022).

Nevertheless, the limitations found in previous literature were: the psychometric properties of the MoVac-COVID19S have been predominantly tested in specific Asian countries and one country in sub-Saharan Africa, Ghana, primarily among university student samples. While these populations are classified as collectivist cultures (Hofstede et al., 2005), the English version of this scale has been validated in Ghana only. Such limited psychometric evidence regarding the MoVac-COVID19S restricts the generalizability of the scale to other populations, especially considering the differences in cultures. The previous studies’ estimation of a four-factor structure from the 9-item version lacked sufficient items for meaningful factors (Kuss & Lüdtke, 2011). Researchers failed to provide adequate information on the dimensionality of the MoVac-COVID19S (Reise et al., 2013). For instance, how does specific sub-scales reflect a particular construct? Overall, there was a lack of rigorous psychometric evidence for the MoVac-COVID19S, potentially limiting its applicability across diverse cultures and sub-groups. Therefore, we aimed to assess the psychometric properties, including the internal consistency, dimensionality, internal, convergent, and divergent validity of the previously validated English version of the MoVac-COVID19S using a sample from the general population of New Zealand. By so doing, we offer a culturally specific instrument for health researchers in New Zealand to investigate COVID-19 vaccination attitudes, enabling the effective design and implementation of optimal vaccination interventions against COVID-19 and future viral and bacterial threats in New Zealand. To assess convergent and divergent validity, we hypothesised a positive relation between the scores of a single measure of COVID-19 vaccination willingness scale and educational levels, respectively, and the scores of the MoVac-COVID19S (Adu et al., 2023a; Guzman-Holst et al., 2020).

Method

Participants

This study utilized data collected from the general population in New Zealand. The total sample comprised 413 participants, with ages ranging from 18 to 89 years (Mage = 46.4; SD = 18.1). Regarding ethnicity within the New Zealand sample, 275 participants (67%) identified as Europeans, 35 (9%) as Asians, 29 (7%) as European/Māori, 23 (6%) as Māori, 12 (3%) as Pacific, and 4 (1%) as Middle Eastern, Latin American, and African (MELAA); additionally, 1 participant (0.2%) identified as European/Asian. See Table 1 for details of participants’ demographic information.

Table 1 Socio-demographic characteristics of participants (n = 413)
Full size table

Power analysis

Maintaining a minimum sample size is recommended to reduce non-convergence risk and ensure accurate estimates and standard errors in Confirmatory Factor Analysis (CFA) (Kyriazos, 2018). Our CFA power analysis aimed to achieve an effect size of 0.10 with a confidence level of 80% and a significance level of 0.05, resulting in a minimum sample requirement of 200 cases. Our current sample size surpasses this threshold, enhancing the power of our results. Hence, the present sample size is deemed sufficient for conducting the CFA on the MoVac-COVID19S (Kyriazos, 2018).

Procedure

Due to the utilization of a previously developed scale in our study, our pilot test did not involve gathering primary data. Instead, we administered the survey questions to few expert academic staff and students to evaluate aspects such as the clarity of instructions, the quality of items, the acceptability of formatting, and the ease of administration (Hertzog, 2008). Data collection for our study was facilitated through the Qualtrics data collection company. The authors’ institution funded this recognised and credible data recruitment company to engage the general population of New Zealand aged 18 years and above for participation in the current study. Participants were rewarded by the Qualtrics data collection company for their participation in the present study. Qualtrics has been instrumental in data collection for many studies over decades (Douglas et al., 2023). It’s essential to note that while online data collection is gaining popularity (Douglas et al., 2023), our decision to opt for this method was primarily driven by the COVID-19 protocols in place during our data collection period. The online procedure emerged as the most viable alternative, considering the safety measures necessitated by the pandemic. Participation in the current survey was voluntary. Before proceeding to the main measures, participants provided responses to demographic variables (Table 1). On average, participants took approximately 15 min to complete the entire questionnaire.

Measures

The Motor of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S; Chen et al., 2021) was used to assess COVID-19 vaccination attitudes. The original scale demonstrated strong internal consistency, with reliability ranging from 0.90 to 0.92 for both versions of this scale. All fit indices, including Comparative Fit Index (CFI) = 0.98, Tucker-Lewis Index (TLI) = 0.97, Root Mean Square Error of Approximation (RMSEA) = 0.09, and Standardized Root Mean Square Residual (SRMR) = 0.06 were also found to be acceptable. The scale further showed known group validity with measures of COVID-19 preventive behaviours (Chen et al., 2021). Essentially, the scale measures the four basic traits of the CME model, with each trait encompassing three items, amounting to a total of 12 items for the full scale. These items are as follows: value (“It is important that I get the COVID-19 jab”, “The COVID-19 jab plays an important role in protecting my life and that of others”, and “The contribution of the COVID-19 jab to my health and well-being is very important’’); impact (“Vaccination is a very effective way to protect me against the COVID-19”, “Vaccination greatly reduces my risk of catching COVID-19’’, and “Getting the COVID-19 jab has a positive influence on my health’’); knowledge (“I know very well how vaccination protects me from the COVID-19’’, ‘I understand how the flu jab helps my body fight the COVID-19 virus’’, and ‘How the COVID-19 jab works to protect my health is a mystery to me’’); and autonomy (“ I get the COVID-19 jab only because I am required to do so’’).

Notably, two items under autonomy and one under knowledge are phrased negatively (Table 2); these three items were excluded from the 12-item version, resulting in the creation of a 9-item version of the instrument. This self-report instrument uses a 7-point scale ranging from 1 = strongly agree to 7 = strongly disagree. Lower scores on this scale denote reduced positive attitudes towards COVID-19 vaccination, while higher scores indicate the opposite. Participants further responded to a single-item measure of COVID-19 vaccination willingness (i.e., “When COVID-19 vaccination becomes available to all, would you take it? ’’), the three response options were: 1) “Yes, as soon as I can get it’’; 2) “No, I will wait and review the safety profile’’; and 3) “I will not get the vaccine’’.

Table 2 Item scores of the Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S) among New Zealanders (n = 413
Full size table

Data analyses

The main analyses were conducted using the newly developed extensions within Jamovi statistical software version 2.3, which encompass packages such as lavaan, semtools, and semplot packages (Epskamp et al., 2019; Jorgensen et al., 2019; Rosseel, 2012; The jamovi project, 2022). These packages are derived from pertinent R packages and are deemed appropriate for the analyses conducted. The collected online questionnaire data were transferred from the Qualtrics online survey platform software to the Statistical Package for the Social Sciences (IBM SPSS; version 28) for data preparation. Within SPSS, the dataset was screened for missing values, and the Missing Completely at Random (MCAR) test (Little, 1988) was conducted, indicating the absence of any missing values in the dataset. Descriptive statistics was conducted for demographic variables to obtain background information about participants, and this was followed by the computation of participants’ items’ scores including means, and standard deviations and frequencies and percentages for each item (Table 2).

Further, factor structures of the scale were analysed with CFA, and parameters were estimated with the Diagonally Weighted Least Squares (DWLS), which provides test-statistics and standard errors that are based on the second-order correction (Muthén et al., 1997). Specifically, the study evaluated the 9-item and 12-item versions of the MoVac-COVID19S for their fitness with the collected data. For the 9-item version, only a one-factor structure was assessed. One and four-factor structures were examined for the 12-item version, representing impact, value, knowledge, and autonomy factors. The bifactor analysis was then conducted with the assumption that the general factor was uncorrelated with the specific factors (Hagan et al., 2022). Although both Exploratory Factor Analysis (EFA) and CFA are used to explore the theoretical constructs or factors that may be reflected by a set of items, we did not conduct EFA analysis because, while CFA is preferred when measurement models have a well-established underlying theory, such as the current scale with hypothesized patterns of loadings, EFA is typically employed to uncover the factor structure of a measure (Hurley et al., 19,907).

Confirmatory factor analysis parameters

CFA is a valuable statistical method that provides valid insights into multi-item constructs. It assesses the relations between items measuring similar and distinct constructs, offering a “goodness of fit” evaluation by comparing data parameters to a proposed measurement model. A model’s validity is determined by acceptable goodness of fit parameters. To gauge data-model fit in the present study, commonly used fit indices were examined: CFI, TLI, RMSEA, SRMR, GFI, and Chi-square test of independence. Acceptable model fit is indicated by CFI, GFI, and TLI values exceeding 0.95, while RMSEA and SRMR values should be below 0.08 (Hu & Bentler, 1999). In cases of even slight model misspecification, statistical power to reject the model is notably strong, leading to a higher likelihood of obtaining a significant outcome. Therefore, we addressed the sensitivity of the chi-square test by applying a commonly employed technique which involved using the chi-square to degrees of freedom ratio (χ2/df), where values less than 5 typically indicate an acceptable fit. When this ratio is lower, it suggests that the difference between the observed data and the model’s predictions is relatively small, indicating a better fit (Kline, 2015).

Bifactor model (confirmatory factor analysis) parameters

We conducted a bifactor CFA for the 12-item version of the MoVac-COVID19S to assess the scale’s dimensionality. The bifactor model offers valuable insights into the multifaceted nature of a scale, revealing the extent to which total scale scores reliably reflect variation in a single underlying variable. It provides essential psychometric parameters that explain how sub-scales serve as indicators of a specific construct (Reise et al., 2013), estimating the proportions of observed variances accounted for by both the general and specific factors. In simple terms, the bifactor model assesses whether a scale can be used as a unidimensional or multidimensional measure. To establish the scale’s dimensionality, various parameters were employed, such as intercorrelation between factors. High intercorrelation between factors signifies that the scale measures a single unified construct. Furthermore, three different omega coefficients were computed to determine the extent to which variance in composite observed scores is explained by common sources of variance (Reise et al., 2013). These included Total Omega (ω), Subscale Omega (ωs), and Hierarchical Omega (ωH), with a ωH value exceeding 0.80 indicating satisfactory support for the underlying factor(s). Strong factor loadings and Explained Common Variances (ECV) signify that observed variables effectively capture specific factor(s). It is important to consider multiple parameters when determining the dimensionality of a scale (Reise et al., 2010).

Internal consistency (reliability) estimates

While psychometric experts have underscored McDonald’s Omega (ω) as a preferable alternative to Cronbach’s Alpha (α). In this study, we adopted both α and ω to estimate the internal consistency of the MoVac-COVID19S for the full and sub-scales, offering complementary evidence. Unlike α, which offers reliability assessment under the restrict assumptions of congeneric measures and tau-equivalence, ω provides reliability estimation while relaxing these assumptions (Trizano-Hermosilla & Alvarado, 2016). We estimated the convergent and external validity of the scale by computing Pearson’s correlations between both the latent factors of the scale and education and the COVID-19 vaccination willingness scale.

Results

Content and construct validity testing

Confirmatory factor analysis

The CFA results in Table 3 show that the one-factor structure of the 9-item MoVac-COVID19S was superior to both the one and four-factor structures of the 12-item version of the MoVac-COVID19S. The structures of the 12-item version of the MoVac-COVID19S achieved acceptable fit indices after model modification. Overall, CFI, and TLI values were higher than 0.95; RMSEA and SRMR values were all less than 0.08 for all the factor models. The Chi-square to degrees of freedom ratio were below 5 for all models in our sample. Finally, factor loadings of the 9-item version of the MoVac-COVID19S were found to be strong ranging from 0.50 to 0.97 for the one factor structure of the 9-item version of the MoVac-COVID19S. In the case of both the one and four-factor structures of the 12-item version of the MoVac-COVID19S, factor loadings exhibited a range from 0.21 to 0.97. Notably, item 10 displayed relatively lower factor loadings within the one-factor and four-factor structures. However, the remaining items demonstrated acceptable factor loadings. (Table 4; Supplementary Figure S1, and Figure S2).

Table 3 Confirmatory factor analysis testing the structure of the Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S) (n = 413)
Full size table
Table 4 Factor Loadings of the 9-item and 12-item versions of the MoVac-COVID19S, including factor Loadings of the General and Specific factors, and Explained Common Variance (ECV) of General Factor in the Bifactor Model for the MoVac-COVID19S (n = 413)
Full size table

Reliability estimates

The full versions of the MoVac-COVID19S demonstrated excellent reliability of both the 12-item (α = 0.91; ω = 0.91) and the 9-item (α = 0.94; ω = 0.95) versions. Each construct in the multidimensional scales has its own unique implications for intended outcomes (Levine, 2005). Hence, the internal consistency of each of the measured traits of the 12-item MoVac-COVID19S in the present study were: values (α = 0.95; ω = 0.95), impact (α = .89; ω = 0.89), knowledge (α = 0.55; ω = 0.61), and autonomy (α = 0.74; ω = 0.70).

Convergent validity

To assess the convergent validity of the MoVac-COVID19S, we expected a positive relation between the latent factors of the MoVac-COVID19S and a single-item measure of COVID-19 vaccination willingness. Results from the Pearson’s correlation coefficient revealed a positive relation between the COVID-19 vaccination willingness scale and value (r = .56, p < .001); impact (r = .50, p < .001); knowledge (r = .34, p < .001); and the full scale of 12-item version of the MoVac-COVID19S (r = .51, p < .001). There was also significant positive relation between all the subscales and the full scales. However, there was no significant relation between the COVID-19 vaccination willingness scale and the autonomy and the 9-item version of the MoVac-COVID19S, (a larger sample is perhaps required to make such inferences). Convergent validity was further exhibited through a positive relation between education and both the full scale of the 9-item (r = .14, p < .001), and the 12-item versions of the MoVac-COVID19S (r = .15, p < .001). Biological sex had no significant relation with the scores of the MoVac-COVID19S.

CFA bifactor model

Results from the bifactor model (Table 4) showed that nine items, representing 75% of the total items in the MoVac-COVID19S loaded strongly on the general factor compared to the specific factors. Particularly, item 1 loaded 0.71 onto the general factor as against 0.23 on the autonomy scale. Item 4 loaded 0.68 on the general factor and 0.25 on the impact subscale (Supplementary Figure S3). In terms of ECV, the general factor accounted higher percentage (from 54% to 91%) of the ECV among 9 items, this represented 75% of the total items in the MoVac-COVID19S. For instance, the general factor explained 94% of the ECV shared by item 3, and 89% for item 6 (Table 4). The hierarchical omega reliability coefficients for the specific subscales were as follows: value subscale (0.12), impact subscale (0.05), knowledge subscale (0.00), and autonomy subscale (0.35), while the general factor’s coefficient was higher (0.41). Despite this, none of these reliability coefficients met the designated threshold for establishing the scale’s dimensionality. Considering these findings, we employed these parameters to ascertain the unidimensionality of the MoVac-COVID19S (Supplementary Figure S3).

Discussion

This study validated the MoVac-COVID19S scale among the general population of New Zealand, showing strong psychometric properties, including construct, convergent, and divergent validity, as well as high reliability. Chi-square to degrees of freedom ratios were below 5 for all models, indicating good fit between observed and predicted data patterns, enhancing reliability. The best fit indices obtained in the current study imply that both the 9-item and 12-item versions of the MoVac-COVID19S aligned with the CME model (Thomas & Velthouse, 1990). Despite displaying relatively lower factor loadings within the one-factor and four-factor structures, item 10 was not removed from the scale for several reasons. First, both items retained content relevance to the construct being measured, suggesting their importance in capturing essential aspects of the latent variable. Second, while individual item loadings are crucial for construct validity, removing items solely based on lower factor loadings could compromise the scale’s overall reliability, especially for already developed scales where the interrelatedness of items has been established. Additionally, the inclusion of item 10 aligns with findings from previous research, ensuring consistency with existing literature (Pratscher et al., 2019; Raykov, 2008).

Specifically, after adjustments, the 12-item version demonstrated acceptable fit indices for both one and four-factor structures. Modifying the 12-item MoVac-COVID19S, which includes both positive and negative items, to achieve acceptable fit indices suggests potential room for further revision to enhance its measurement precision. Past research has highlighted how negatively worded items can impact the reliability and validity of psychometric measures (Lee et al., 2016). It is conceivable that the 12-item version of the MoVac-COVID19S, containing negative items, might have been influenced by such factors compared to the 9-item version with only positive items. Notably, the negatively worded items exhibited cross-loading to achieve acceptable fit indices for all the factor structures. Future investigations could explore the influence of these negative items using alternate techniques such as cross-validation (Jaccard & Turrisi, 2003). Chen et al. (2021) have demonstrated improved model fit indices in the 12-item of the MoVac-COVID19S by addressing wording effects, underscoring the potential impact of negatively worded items in this version. However, the 9-item version emerged as psychometrically superior, given its stronger fit as a one-factor structure. Its brevity also makes it more suitable for quicker assessement of attitudes toward COVID-19 vaccination.

Yeh et al. (2021) found the four-factor structure of the 9-item version as a better measure compared to the 12-item version of the MoVac-COVID19S. The majority of the previous research has provided evidence for the superiority of the 9-item version of this scale (Chen et al., 2021; Yeh et al., 2021), supporting our findings; but contrasting to those reported by Pramukti et al. (2022). Chen et al. (2022) reported better fit indices for the four-factor structures of the MoVac-COVID19S compared to the one factor structures across diverse cultural contexts including Ghana, China, Taiwan, India, and Afghanistan, inconsistent with our findings Our study provided evidence on the factor structure of the MoVac-COVID19S in the culturally diverse context of New Zealand (New Zealand Statistics, 2019). The traditional CFA supported both the unidimensional and multifactorial versions of the scale, indicating consistent and effective measurement of attitudes toward COVID-19 vaccination through single and subscale scores of the MoVac-COVID19S. These results successfully replicated earlier proposed factor structures.

Additionally, the reliability coefficients of the two versions of the scale were excellent, suggesting the scale’s consistency within the sample, agreeing with the study of Chen et al. (2021). The sub-scales of the 12-item MoVac-COVID19S had slightly lower reliability, possibly due to wording effects again, especially in the knowledge traits, which had two negatively worded items. Despite this, confidence still remains in using the overall score of the MoVac-COVID19S, especially the 9-item version, in the New Zealand general population for assessing COVID-19 vaccination attitudes. The strong positive correlations among subscales validate the CME model’s interconnected traits driving purposeful behaviour (Thomas & Velthouse, 1990). This suggests that the current scale could facilitate interventions aimed at promoting COVID-19 vaccination. Indeed, Bellali et al. (2023) found that individuals who scored higher on all components of the CME model were more inclined to accept the influenza vaccine.

Additionally, the MoVac-COVID19S positive correlation with the COVID-19 vaccination willingness scale confirms its convergent validity. Consistently, the literature has depicted a positive association between educational levels and vaccination attitudes, aligning with our study’s findings (Guzman-Holst et al., 2020). This observation affirms the external validity of the scale. The results from the CFA bifactor model consistently indicated that the general factor accounted for a greater proportion of the ECV in the items than the specific factors. The majority of the items exhibited strong loadings on the general factor in contrast to the specific factors. Moreover, the notably high intercorrelations among the sub factors (e.g., 0.95), the relatively low reliability of the knowledge subscale, imply that the four factors lack differentiation and should be effectively treated as a single factor. The negative loading of item 10 on the knowledge subscale also indicated problem for the proposed four factor solution of this scale. Collectively, these findings underscore that the general factor of the MoVac-COVID19S more effectively captures COVID-19 vaccination attitudes in the sample than the specific factors, confirming the unidimensional nature of the MoVac-COVID19S. Consequently, individual single scores provide a better explanation of the underlying latent structure of the MoVac-COVID19S than the multidimensional structure. As a result, we contend that treating the MoVac-COVID19S especially the 9-item version as a unidimensional measure offers a more precise assessment of COVID-19 vaccination attitudes compared to the multidimensional scale.

Strengths and limitations

This study represents the first effort to assess and validate the psychometric properties of the MoVac-COVID19S in New Zealand. It also presents a novel exploration of MoVac-COVID19S performance within a diverse population, a unique contribution compared to prior assessments. Moreover, we employed a more accurate method to gauge the dimensionality of MoVac-COVID19S than the traditional CFA, addressing methodological limitations identified in previous research to enhancing the study’s methodological rigor. Notably, this was a high powered study, as our sample size aligns well with CFA recommended thresholds, as determined by experts in the field (VanVoorhis & Morgan, 2007).

However, our study does possess certain limitations. First, while online surveys have become prevalent, especially amid the COVID-19 pandemic, using this method means that participation is disproportionately skewed toward those with internet access and electronic devices. Our sample may not be representative of the general population of New Zealand, as researchers employed a convenience sampling method by relying on readily available individuals. These limitations could introduce selection bias and limit the generalizability of the study. Secondly, employing self-report measures could introduce a potential response bias, as individuals might align their responses with societal expectations (Grimm, 2010). Thirdly, while CFA offers reliable assessments, recent criticisms have arisen. For instance, parameters such as reliability and factor loadings are often sample-dependent, necessitating large samples, which can be challenging to achieve in psychological research. Due to the non-equivalence of our sample composition across personal factors, researchers did not find evidence supporting measurement invariance across these factors. We also understand the difficulties arising from the absence of other reliable and valid measures for validating COVID-19 vaccination willingness, prompting us to initially utilize a single-item approach. Nonetheless, we acknowledge that this approach may limit the scope of validation given the complexity of the construct.

Implications future research

Findings from this study hold practical and theoretical implications for COVID-19 vaccination and other vaccination initiatives. The MoVac-COVID19S can effectively gauge vaccination attitudes, as this scale can facilitate healthcare professionals, policymakers, and researchers understanding of public attitudes and aid in devising strategies to enhance positive attitudes towards vaccination. Caution is advised when using the 12-item version due to wording effects, prompting stakeholders to consider individuals’ single scores of the 9-item version as a more reliable alternative. Notably, the 9-item version of the MoVac-COVID19S appeared to be more robust, and time-saving due to the small number of items, making it suitable for urgent situations or resource-limited research. Future vaccination programs can adapt this scale for new vaccines by modifying the disease names (Chen et al., 2021). Future research should consider employing robust methodologies, such as Rasch analysis, to comprehensively assess the psychometric properties of MoVac-COVID19S, thereby overcoming the limitations of traditional CFA (Bartholomew et al., 2023). Additionally, evidence of measurement invariance across diverse populations, including translated versions of the MoVac-COVID19S is essential to enhance the generalizability of the MoVac-COVID19S. To achieve this, expanding studies to different countries and microcultures is recommended. Further, utilizing a more representative sample and employing relatively robust data collection procedures, along with multi-measure instruments, will provide additional evidence regarding the validity and reliability of this scale.

Conclusion

We examined the psychometric propertiesy of the MoVac-COVID19S. Best fit was observed for the one-factor structure of the 9-item version, while model modifications yielded acceptable fit indices for both the one-factor and four-factor structures of the 12-item version. The two versions of the MoVac-COVID19S exhibited excellent reliabilities among the sample. The bifactor model highlighted the dominance of the general factor, endorsing the MoVac-COVID19S as a unidimensional measure. Consequently, single scores of the 9-item version of the MoVac-COVID19S emerged as the optimal approach for assessing COVID-19 vaccination attitudes. The instrument’s adaptability for new vaccines enhances its utility for future vaccination programs. Employing the Rasch method across diverse populations or sub-groups could further enhance the effectiveness of the scale. Ultimately, this study offers a valuable tool to explore attitudes towards new vaccines, enriching healthcare policy and practice decision-making.