Introduction

Mental health has traditionally been conceptualized from a pathogenic model that considers mental health as simply the absence of mental illness or disease (Keyes, 2005). This bipolar model implies that mental health and mental illness are opposite ends of a single continuum, leading psychologists and psychiatrists to focus on reducing mental illness symptoms to increase mental health. Despite these efforts, the focus on illness treatment has not reduced the prevalence of psychopathology in the last decade (Insel & Scolnick, 2006). Although the initial assumption was that the absence of mental illness does not constitute a criterion for mental health (Jahoda, 1958), it was not until two decades ago that the two continuums of mental health (i.e., well-being as a positive indicator and psychopathology as a negative indicator) were integrated within the same study (Greenspoon & Saklofske, 2001). Since the publication of these findings, numerous studies have supported a more integrative concept of mental health: the complete state model (Keyes, 2005), also known as the dual-factor model (Suldo & Shaffer, 2008). Under this model, complete mental health represents the presence of high levels of emotional, social, and psychological well-being and low levels of mental illness symptoms (Keyes, 2005).

Mental health in adolescence

Adolescence is an evolutionary stage often characterized by vulnerable and challenging situations that can put adolescents’ mental health at risk, increasing the probability of suffering from a mental illness (Burger & Samuel, 2017). Middle adolescence is especially important in the Spanish educational system because it is a transitional stage in which adolescents must decide on their next academic steps (e.g., to choose between an academic or a labor career). In this transition to young adulthood, high levels of well-being and relatively minor symptoms of psychopathology help them cope with the challenging demands of this life stage (O’Connor et al., 2017).

Adopting the dual-factor model has important implications for mental health care, especially considering that depression, the most prevalent mental illness worldwide, is diagnosed in 4.4% of the global population and typically begins at age 15 (World Health Organization, 2017). For instance, the dual-factor model has implications for designing and determining the efficacy of psychological interventions. In positive psychological interventions, defined as programs involving a particular set of strategies that target and promote different components of well-being (Sin & Lyubomirsky, 2009), the dual-factor model can help elucidate what strategies are most effective for promoting (reducing) indicators of well-being (psychopathology) in adolescents (Tejada-Gallardo et al., 2022). Accordingly, research has revealed that adolescents’ high levels of well-being, as opposed to psychological distress, predicted positive outcomes, such as social support, physical health, and academic performance (Suldo & Shaffer, 2008). According to this line of reasoning, monitoring the unfolding of adolescents’ well-being and psychopathology in public mental health is necessary to assess the evolution of mental health and to ensure adequate health care.

Properties of the Mental Health Continuum—Short Form

The MHC-SF is a self-report measure comprising 14 items increasingly implemented in the general population to assess mental health. This short version was developed from the original 40-item mental health continuum long form (Keyes, 2005) and included two widely recognized approaches to well-being: hedonia and eudaimonia. The hedonic approach involves the dimension of emotional well-being, which focuses on life satisfaction and affective states in an individual’s life (Diener, 2000). The eudaimonic approach includes the two dimensions of (1) social well-being, which reflects how an individual functions and thrives in society (Keyes, 2005), and (2) psychological well-being, which focuses on the optimal intrapersonal and interpersonal functioning of an individual (Ryff & Keyes, 1995). During adolescence, females tend to score higher in eudaimonic-related aspects, such as appreciating small things, but they also experience more significant declines in well-being than males do (Crous et al., 2018).

The psychometric properties, construct and structural validity, and reliability of the MHC-SF have been widely tested across diverse cultural contexts and populations (e.g., Echeverría et al., 2017; Rogoza et al., 2018). However, the structure of this measure is still not clearly defined, and there is ongoing academic debate within the psychological field regarding the most appropriate conceptualization of this measure. Discrepancies regarding the underlying structure are especially relevant in adolescents, as some studies have either supported a three-factor structure (Luijten et al., 2019), a bifactor structure using confirmatory factor analysis (CFA) (Piqueras et al., 2022), or a bifactor structure using exploratory structural equation modeling (ESEM) (Rogoza et al., 2018). Recent findings have suggested that ESEM offers a compelling alternative to the conventional three- and bifactor models to capture the structure of the MHC-SF (van Zyl & Ten Klooster, 2022).

Among the different structures of the MHC-SF proposed in the literature, the three-factor structure has been the most widely supported (e.g., Luijten et al., 2019; Petrillo et al., 2015). However, some researchers have considered this structure problematic from a theoretical and empirical perspective due to its acceptable marginally fit indices (de Bruin & du Plessis, 2015). Previous studies have suggested that to overcome this limitation, a bifactor model using CFA (de Bruin & du Plessis, 2015) or a model based on ESEM (Rogoza et al., 2018) might provide a better fit than the three-factor structure.

Even so, when a bifactor model is tested using CFA, some limitations should be acknowledged. The general factor can be overestimated due to the absence of specified cross-loadings on the specific factors of the measure, and reliability estimates might also be inflated among the factors (Morin et al., 2016). An alternative to the drawbacks that the bifactor CFA holds is to implement the ESEM technique using target rotation, which consists of a combination of CFA, allowing for the structure of the measure, and an exploratory factor analysis enabling the presence of cross-loadings among the specific factors (Asparouhov & Muthén, 2009). ESEM using target rotation might be an appropriate alternative to CFA because it is better suited for exploring the a priori hypothesized structure of a measure with multiple dimensions in which items pertaining to highly correlated factors are expected to load on another factor.

A recent validation of the MHC-SF in Spanish adolescents found that this scale was best represented by a bifactor structure (Piqueras et al., 2022). Nevertheless, these results were tested using CFA, an overly restrictive technique for assessing multidimensional models. Implementing ESEM using target rotation (i.e., adding the confirmatory assumption when assessing the structure and validity of the scale) was a missed opportunity. From a psychological standpoint, items explaining psychological constructs are rarely restricted to a theoretically assigned factor (Morin et al., 2016). Thus, it seems suitable to investigate the psychometric properties of a multidimensional scale by selecting the appropriate techniques according to the construct being measured.

On this basis, the present study’s main aim was to provide an in-depth examination of the psychometric properties of the MHC-SF in Spanish adolescents by introducing a bifactor model based on ESEM and considering its advantages over CFA. Furthermore, the study attempted to test this scale’s structural and convergent validity, measurement invariance, dimensionality, and internal reliability. We hypothesized that the bifactor ESEM model with a general positive mental health factor would fit the data better than the three-factor ESEM model and the bifactor CFA model.

Methods

Participants

The sample comprised 465 Spanish adolescents. The sample mean age was 15.17, and the SD was 0.64 (age range: 14–17), 44.5% of which were females. The structural validity of the MHC-SF was assessed with the general sample (n = 465), but 83 adolescents did not complete the measures of life satisfaction, depression, anxiety, and stress. Therefore, the construct validity of the scales was analyzed with 382 adolescents. A sample of approximately 500 participants is considered adequate to assess the psychometric properties of a psychological self-report (Comrey & Lee, 1992); therefore, the sample size of the current study might be deemed suitable.

Instruments

The Mental Health Continuum–Short Form (MHC–SF; Spanish adaptation of Echeverría et al., 2017) assesses how often in the last month individuals felt in a certain way regarding their emotional (3 items; e.g., “How often did you feel happy?”), social (5 items; e.g., “How often did you feel that you had something important to contribute to society?”), and psychological (6 items; e.g., “How often did you feel that you liked most parts of your personality?”) well-being. This scale includes 14 items, which participants respond to using a 6-point Likert scale ranging from 1 (never) to 6 (every day). The Cronbach’s α reliability estimates of the MHC-SF for the present study were 0.81 for emotional well-being, 0.74 for social well-being, and 0.82 for psychological well-being. The model-based reliability (i.e., McDonald’s ω coefficient; Sijtsma, 2009) following the retained model from the analysis is reported in the “Results” section.

The Satisfaction with Life Scale (SWLS; Spanish adaptation by Atienza et al., 2000) assesses the degree of satisfaction with life. This scale includes five items (e.g., “in most ways my life is close to my ideal”), and participants are asked to rate their satisfaction with life on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree). The Cronbach’s α (and McDonald’s ω) reliability estimates of the SWLS for the present study were 0.82 (0.81).

The Depression, Anxiety, and Stress Scale (DASS-21; Spanish adaptation by Daza et al., 2002) assesses the levels of symptomatology associated with depression (7 items; e.g., “I could not seem to experience any positive feeling at all”), anxiety (7 items; e.g., “I was worried about situations in which I might panic and make a fool of myself”), and stress (7 items; e.g., “I find it hard to wind down”) over a week. This scale comprises 21 items, with responses based on a 4-point Likert scale ranging from 0 (did not apply to me at all) to 3 (applied to me very much, or most of the time). The Cronbach’s α (and McDonald’s ω) reliability estimates of the DASS-21 for the present study were 0.86 (0.86) for depression, 0.77 (0.78) for anxiety, and 0.76 (0.77) for stress.

Procedure

Adolescents voluntarily participated in a program on personal development offered to three high schools by the University of Lleida (Social Innovation Chair). Data were obtained as part of the assessment of the program, which aimed to identify adolescents’ ability to improve their well-being. Participants completed the measures via Google Forms and received an individualized report with their results at the end of the survey. To identify patterns of careless responding, we included attention check questions among the items (e.g., “Select the same answer as in the previous question”). From the general sample, 34 adolescents did not select the correct answers to these attention checks and were excluded from the data analysis. Adolescents were included in the study only if informed consent from parents or a legal guardian was returned. We provided information about the study’s aims and the terms of confidentiality and anonymity in the survey. The University Ethics Committee approved the present study under the code CEIC-2157.

Data analysis

Structural validity

We compared four competing models using theory and previous studies. We employed robust weighted least squares (WLSMV) due to its superiority over maximum likelihood (ML) estimation for analyzing ordered-categorical indicators (Beauducel & Herzberg, 2006). Although the WLSMV estimator is typically employed for scales with five or fewer response categories, recent studies have indicated that it performs comparably with ML estimation and reports superior performance in estimating factor loadings for scales with more than five response categories (Brauer et al., 2023). We included (1) a one-factor model consisting of all items measuring a general factor of well-being; (2) a two-factor model testing the hedonic (emotional well-being) and eudaimonic (social and psychological well-being) approaches to well-being; (3) a three-factor model (Keyes, 2005), in which items 1–3 loaded on the latent variable of emotional well-being, items 4–8 loaded on social well-being, and items 9–14 loaded on psychological well-being; and (4) a bifactor model in which each item loaded simultaneously onto the theoretically assigned lower-level factors (emotional, social, or psychological) and a general factor of positive mental health. The three-factor and bifactor models were analyzed in CFA and ESEM using target rotation. As ESEM using target rotation appeared to better represent the structure of the MHC-SF in previous studies with adolescents (Rogoza et al., 2018), this technique was implemented in the present study.

All models were assessed for fit using several indices, including the WLSMVχ2, Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA), and its 90% confidence interval (Hu & Bentler, 1999). CFI and TLI values greater than 0.900 and 0.950, respectively, and RMSEA values lower than 0.080 and 0.060 indicated adequate and excellent model fit, respectively. Yet the selection of measurement models should also be based on theoretically substantive grounds (Greene et al., 2019).

Measurement invariance

The final retained model underwent tests of measurement invariance across genders (females versus males). These tests followed a sequential invariance strategy (Meredith, 1993) adapted for ordered-categorical indicators (Guay et al., 2015), which included the following steps: (1) configural invariance; (2) metric (weak) invariance (invariance of factor loadings); (3) scalar (strong) invariance (invariance of loadings and thresholds); (4) strict invariance (invariance of loadings, thresholds, and uniqueness); (5) invariance of latent variances-covariances (invariance of loadings, thresholds, uniqueness, and variances-covariances); and (6) invariance of latent means (invariance of loadings, thresholds, uniqueness, variances-covariances, and latent means). In addition to the fit indices used to evaluate the structural validity of the scale, fit improvement was examined using the Mplus DIFFTEST function (MDΔχ2; Asparouhov and Muthén, 2009). However, considering the potential sensitivity of MDΔχ2 to sample size and minor model misspecifications, we also looked for a decrease in CFI of 0.010 or less and an increase in RMSEA of 0.015 or less between consecutive models (Chen, 2007). Meeting these thresholds indicates the adequacy of the model fit.

Internal reliability

The reliability of the MHC-SF from the retained model was analyzed using McDonald’s ω coefficient. Unlike Cronbach’s α, which assumes equivalent indicators with equal factor loadings and unidimensional constructs (Dunn et al., 2014; Sijtsma, 2009), ω is a model-based estimate of composite reliability, particularly useful for multidimensional scales in which items are explained by orthogonal factors (Dunn et al., 2014). The ω estimate is a more accurate measure of scale reliability as it takes into account the estimated parameters, including factor loadings and residual variances. This estimation enables to determine the proportion of variance in the total score that is explained by the common variance of the bifactor (Dueber & Toland, 2021). To determine the general factor’s power, we also assessed the proportion of common variance explained by the emotional, social, and psychological well-being factors using the explained common variance (ECV; Rodriguez et al., 2016). By analyzing both ω and ECV, we obtained a comprehensive understanding of the reliability of the MHC-SF in the context of bifactor ESEM.

Construct validity

Pearson correlations were used to assess the construct validity of the best-fitting model of the MHC-SF in terms of convergent validity with another measure of well-being (life satisfaction) and discriminant validity with a measure of psychological distress (depression, anxiety, and stress). Following Cohen’s (1988) recommendations, a correlation coefficient between 0.10 and 0.29 was considered low, a coefficient between 0.30 and 0.49 was considered moderate, and a coefficient ≥ 0.50 was considered high.

All analyses were conducted in Mplus v.7.2 (Muthén & Muthén, 2012). ECV and the McDonalds ω were performed with the standalone program Omega for bifactor models. We employed a bifactor ESEM invariance tool for generating the corresponding syntax (Beer & Morin, 2022). The data and syntax are available at the Open Science Framework (OSF): https://osf.io/ckzm4/.

Results

Structural validity

The six tested models (four CFA representing the one-, two-, three-factor, and bifactor structure, and two ESEM models representing the three-factor and bifactor structures) are presented in Table 1. The fit indices for the one- and two-factor models indicated a poor fit. In the three-correlated factor structure, the ESEM better represented the data than CFA. Likewise, the bifactor ESEM yielded a better fit to the data than the bifactor CFA. To further validate the model selection, factor correlations were used to compare coefficients between the three-factor CFA and three-factor ESEM (detailed results are available in the OSF). The reduced correlations in the three-factor ESEM supported the notion that an underlying general factor was present and therefore reinforced the selection of the bifactor ESEM model (Asparouhov et al., 2015). Hence, based on the fit statistics, information criteria, and factor correlations, the bifactor ESEM model better represented the data and was thus retained to investigate the measurement invariance across genders.

Table 1 Goodness-of-fit Statistics for the Alternative Measurement Models of the MHC-SF
Full size table

Gender measurement invariance

The results from the measurement invariance analysis using the bifactor ESEM solution are presented in Table 1. The goodness-of-fit indices indicated a satisfactory model fit at each stage. When equality constraints were imposed, the changes in goodness-of-fit did not decrease beyond the recommended guidelines. In fact, the CFI, TLI, and RMSEA demonstrated an improvement in fit across most of the steps.

Dimensionality and internal reliability

The standardized factor loadings of the bifactor ESEM model, the McDonald’s ω reliability estimate, and the percentage of ECV in the general factor are presented in Table 2. The factor loadings of all items were significantly higher for the general positive mental health factor than for the specific factors. Although factor loadings were acceptable, some items were marginally explained by their theoretically assigned factors: feeling happy (item 1; emotional well-being), belonging to a community and the belief that one has something to contribute to society (items 4, 5; social well-being), having warm and trusting relationships, and having experiences that challenged one to become a better person (items 11, 12; psychological well-being). These items might better capture general positive mental health.

Table 2 Standardized Factor Loadings of the Bifactor ESEM and Reliability Estimates
Full size table

McDonald’s ω coefficients indicated very good reliability for the general positive mental health factor (0.92), as well as for the emotional (0.83) and psychological (0.84) factors. The social factor exhibited good reliability (0.78). The general factor accounted for 74% of the variance, indicating that specific factors of well-being made moderate independent and additional contributions to overall positive mental health. These results also suggest the presence of a general positive mental health factor on the scale.

Construct validity

The positive mental health factor was positively related to life satisfaction (r = .76, p < .001), supporting convergent validity, and negatively related to depression (r = − .71, p < .001), anxiety (r = − .54, p < .001), and stress (r = − .41, p < .001), supporting divergent validity. The correlations between the specific factors and life satisfaction, depression, anxiety, and stress were not significant.

Discussion

This study aimed to extend the investigation of the factorial MHC-SF structure in a sample of Spanish adolescents. Although previous studies have examined the psychometric performance of this instrument (e.g., Luijten et al., 2019; Piqueras et al., 2022), statistical methods that were appropriate for assessing the underlying structure of the MHC-SF were omitted, such as ESEM (Rogoza et al., 2018; van Zyl & Ten Klooster, 2022), which allows for confirming a hypothesized factorial structure while enabling cross-loadings to examine complex psychological structures (Marsh et al., 2014). Our results extend the findings of this inquiry by revealing that the bifactor ESEM was the most suitable model for representing the structure of the MHC-SF in Spanish adolescents. Reliability, convergent and divergent validity, and measurement invariance further reinforce this finding. Consistent with prior validations of the MHC-SF, we found evidence of scalar invariance across females and males (e.g., Luijten et al., 2019; Piqueras et al., 2022; Rogoza et al., 2018). Furthermore, we conducted analyses to examine the strict, latent variance-covariance, and latent mean invariance. The results indicated invariance at the latent mean level, supporting the meaningful comparison of factor mean scores across gender.

In agreement with previous research (e.g., Luijten et al., 2019; Piqueras et al., 2022), mental health in adolescents can generally be described as comprising emotional, social, and psychological well-being, as assessed by the MHC-SF. The bifactor structure has previously been used as an optimal alternative for representing mental health (Luijten et al., 2019; Rogoza et al., 2018). Consequently, MHC-SF items typically load on a general factor of positive mental health, which captures commonality among items, and load on specific factors that capture group variance (Rodriguez et al., 2016). Notwithstanding, the introduction of the bifactor ESEM revealed that some MHC-SF items represent the positive mental health factor more than their specific factors. Items 1 (emotional well-being), 4–5 (social well-being), and 11–12 (psychological well-being) barely loaded on their specific factors. In our sample, this finding indicated that feeling happy, having a sense of contributing to society, feeling part of a community, reporting warm and trusting relationships, and perceiving personal growth are more significant markers of general positive mental health than of the independent elements for which they were theoretically designed. Moreover, the results show that the item of social integration might be more representative of psychological well-being than of its own dimension. We devised potential explanations for these results.

First, the meaning of “happiness” can be ambiguous, either reflecting a sense of compliance and contentment about one’s life or a more nuanced evaluation of the current affective state (Vittersø, 2016). Second, the items of social contribution and personal growth would be better suited for capturing a notion of positive mental health in adolescents because experiencing a sense of self-realization and personal fulfillment might be a function of evolutionary processes governed by external rather than internal resources. As adolescents increasingly commit to self-standards in forming their identity during the transition toward adulthood, these indicators could become more representative of their specific dimensions (Cote, 2006). Third, as social relationships and the need for affiliation are crucial during this developmental stage (Oberle et al., 2011), our results could imply that the items of social integration and positive relationships, which carry interpersonal information, contribute to adolescents’ sense of positive mental health. Overall, the ESEM results suggested that these items may not be specific indicators of the three well-being dimensions but offer significant information about adolescents’ general positive mental health.

Another noteworthy finding was that the item of social integration emerged as an indicator of psychological well-being instead of social well-being, which might comply with the previous line of reasoning. That is, engaging in social activities and groups that provide social support during adolescence is essential (Deci & Ryan, 2000). Combining these social integration elements (e.g., feeling part of a group of friends) might contribute to adolescents’ search for identity. In fact, the transition toward young adulthood reflects an increasing reliance on and engagement with close friends and partners for identity development (Meca et al., 2015), aspects closely related to psychological well-being (Ryff & Keyes, 1995).

Practical implications

The MHC-SF items captured individuals’ perceptions of positive mental health in a general sense. Nonetheless, some items also provided specific information on emotional, social, and psychological well-being. In other words, each factor captures something unique and, simultaneously, something common with the rest of the items. However, considering the proportion of variance explained by the general factor of positive mental health and that all items loaded more onto the general factor than onto the specific factors, the MHC-SF could be suitable for assessing an adolescent’s positive mental health as a general construct. We, therefore, recommend that certain items should not be used as unique indicators of their assigned dimensions, which opens new possibilities for using them as general markers of adolescents’ positive mental health. The subscale scores of the MHC-SF demonstrate a significant amount of variance beyond the general factor of positive mental health (ω > 0.50; Perreira et al., 2018). Researchers and scholars interested in examining the specific performance of emotional, social, and psychological well-being are encouraged to employ appropriate statistical procedures that carefully represent the structure of positive mental health and capture both general and specific factor variances. According to the literature on the applications of the dual-factor model of mental health, providing common measurement tools with appropriate psychometric performance is the first step toward building a comprehensive approach to mental health in psychological research. The overdemanding expenditure on mental healthcare poses a serious threat to governments, including educational facilities. Equipping researchers and practitioners with widely validated questionnaires that easily assess mental health can contribute to integrating mental health care models and alleviating economic burden (Wood & Tarrier, 2010).

Although this study presents a robust analytical procedure, some limitations should be acknowledged. First, the generalizability of the results might be compromised since the sample was limited to school-based adolescents and used self-reported measures. Second, the sample is not representative of Spanish adolescents because participants were recruited exclusively from Catalonia. Future studies are also necessary to validate the MHC-SF in adolescent clinical samples. For future research, we recommend that studies include a longitudinal method to examine the stability of the general positive mental health factor and specific factors.

Conclusion

Bifactor ESEM could be an appropriate model to reflect and explain the multidimensional representation and factor structure of the MHC-SF in Spanish adolescents. This model offers a comprehensive framework for understanding the MHC-SF scale, emphasizing the central role of the general positive mental health factor. To fully capture the unique contributions of the specific factors (emotional, social, and psychological), employing appropriate statistical techniques is essential. Using our findings to conduct a nuanced analysis could provide valuable insights into the individual role of MHC-SF items in representing positive mental health. This study aligns with a growing number of publications supporting the adoption of the dual-factor model of mental health to assess mental health in adolescents and design cost-effective programs aimed at promoting mental health (Le et al., 2021).