Background

According to the Global Burden of Disease Study, in 2019 approximately 792 million people presented mental health problems, a number that rises to 970 million people if substance use is included [1]. In recent years, several studies [2, 3] have reported an increase in the prevalence of these problems. The estimated prevalence rates of mental health problems and substance use vary widely by region and country: for example, in the US, the estimated rate was 16.9%; in Portugal, 18.5%; in eastern countries, rates tend to be lower (e.g., Thailand, 12.0%; China, 11.3%; India, 13.7%; Indonesia, 10.7%) [4]. The COVID-19 pandemic led to a global increase of mental health problems, particularly stress, depressive and anxious symptoms [5, 6], suggesting an increased need for mental health assessment as well as the provision of services and efficacious interventional programmes.

The increasing prevalence of mental health problems globally suggests the need to adopt preventative measures, as early detection of signs and symptoms increases the probability of success in terms of treatment outcomes [7,8,9,10,11,12,13]. One relevant resource for the prevention of mental health problems is Mental Health Literacy (MHL), a multifactorial construct encompassing: (1) knowledge related to the prevention of mental health problems; (2) recognition of signs and symptoms (i.e., being able to identify the onset of problem development); (3) identification of available options and treatments; and (4) knowledge of tailored self-help strategies and first-help skills to support others who are developing and/or present with mental health problems [9]. Low levels of MHL have been found to be related to difficulties in recognizing mental health problems (whether in oneself or in others) [12], delay in seeking help, inadequate use of resources and action strategies, as well as communication difficulties with health professionals and lower adherence to treatments [2, 14,15,16,17].

Research has examined the role of variables that potentially influence MHL levels, namely gender [7, 18], proximity to someone with mental health problems [3, 19,20,21,22] and culture [23, 24]. The results of these studies indicate that young female adults [3, 25,26,27,28] and people with proximity to people with mental health problems [7, 15, 18, 29,30,31,32] tend to present higher levels of MHL. With regard to culture and its influence on MHL, factors such as personal beliefs, religion, language, cultural diversity and subjective experience seem to influence knowledge about mental health problems [9, 23, 33, 34]. Studies that compared the MHL levels of participants from different countries and regions found that participants from western and developed regions presented higher levels of MHL, in comparison to participants from developing regions [8, 35].

Several instruments have been developed to assess MHL (see reviews [27, 36]), some of them focusing on specific dimensions (e.g., knowledge; stigmatizing perceptions) or specific mental health problems or diagnoses (e.g., schizophrenia; depression) [7]. Taking into account the updated construct of MHL [9], and limitations of previous measures (e.g., use of time-consuming vignettes, measures limited to specific mental health problems), a new instrument to provide a more up-to-date assessment of this construct was developed for assessing MHL in young people (Mental Health Literacy questionnaire – young people form; MHLq-YP; [7]). In 2018, Dias and collaborators adapted this questionnaire for young adults - the Mental Health Literacy questionnaire-young adult form; MHLq-YA. The MHLq-YA includes 29 items, rated on a five-point Likert-type scale, organized into four dimensions: (1) knowledge of mental health problems, (2) erroneous beliefs/stereotypes, (3) first aid skills and help seeking behaviour, and (4) self-help strategies. The preliminary study of this instrument’s psychometric properties showed appropriate levels of validity and internal consistency [18].

Given the increasing need to assess MHL among young adults and given time constraints when undertaking such assessments, the current study aimed to expand the analysis of the psychometric properties of the MHLq-YA to provide a revised and shorter version. In addition, we examined the psychometric properties of this short version in student samples from six different countries (China, India, Indonesia, Portugal, Thailand, and United States).

Methods

Participants

The study involved 2180 senior school and undergraduate students, aged between 17 and 25 years old, from six different countries – China, India, Indonesia, Portugal, Thailand, and the United States.

The sample from China included 496 undergraduates (63% female), aged from 17 to 22 years (M = 19.00; SD = 0.84). The sample from India included 284 undergraduate students (63% female), aged between 18 and 25 years (M = 20.85; SD = 1.85). The Indonesian sample included 197 undergraduate students (63% female), aged between 18 and 23 years (M = 19.08; SD = 1.28). The Portuguese sample included 382 undergraduate students (53% female), aged between 17 and 25 years (M = 21.21, SD = 2.15). The Thai sample included 385 undergraduate students (70% female), aged between 18 and 24 years (M = 20.26; SD = 1.30). The United States sample included 436 undergraduate students aged between 18 and 25 years (M = 20.60; SD = 1.99; 60% female).

Procedure

Prior to data collection, the MHLq-YA was translated into the predominant language of each country, following the guidelines for translating and adapting items [37]: (1) translation from English or Portuguese by a bilingual translator; (2) back-translation by a different bilingual translator; (3) pilot testing of items, through think aloud procedures (Thailand) or comprehension rating using a 3-point scale questionnaire (China); (4) semantic comparison of the translation and back-translation; (5) review by experts in psychometrics and linguistics (India, Indonesia, and Thailand); (6) analysis of the translated version by the Portuguese research team.

Participants’ recruitment occurred through in person contact with students (Portugal, US, and Thailand) and through online dissemination (China, India, Indonesia, Portugal, Thailand, and US).

The study followed ethical guidelines in each country with all participants providing written informed consent. The sociodemographic and MHLq-YA forms were self-administered using online platforms (India, Indonesia,Thailand), in-person paper-and pencil questionnaires (China, US), or both (Portugal). Data collection occurred both prior to (Portugal, Thailand, and US) and during (China, Thailand, India and Indonesia) the COVID-19 pandemic.

Instruments

The protocol included a sociodemographic section, comprising self-reported questions about gender and age, and the MHLq-YA.

Mental Health literacy questionnaire – young adult form (MHLq-YA)

The original version of MHLq-YA [18] included 29 items, developed to measure MHL on four dimensions: (1) knowledge of mental health problems (e.g., “A person with depression feels very miserable.”; “People with schizophrenia usually have delusions.”), (2) erroneous beliefs/ stereotypes (e.g., “Mental disorders don’t affect people’s behaviors.”; “People with mental disorders belong to low-income countries.”, (3) help-seeking and first aid skills (e.g., “If I had a mental disorder, I would seek my relatives’ help.”; “If someone close to me had a mental disorder, I would encourage her/him to look for a psychologist.”, and (4) self-help strategies (e.g., “Physical exercise contributes to good mental health.”; “Sleeping well contributes to good mental health.”). Participants were asked to rate each item, ticking the option that indicates how much they agree or disagree, using a five-point scale (ranging from 1 = Strongly Disagree to 5 = Strongly Agree) was used to respond to the items. Cronbach’s Alpha for the total scale in the adaptation study was 0.84.

Data analysis

Considering the preliminary evidence from the MHLq-YA study [18], first we performed a confirmatory factor analysis (CFA) on the original structure with the Portuguese sample. For CFA analyses, we used AMOS software (SPSS Inc, Chicago, IL, version 27.0) and the estimation method chosen was maximum likelihood (ML). Following the theoretical recommendations, to evaluate the goodness of fit of models, we used the following global indeces: Chi-square (X2) and Chi-square difference (X2/gl); Non-Normed Fit Index (NNFI) and Comparative Fit Index (CFI); and Root Mean Squared Error of Approximation (RMSEA). The standard value for CFI is equal to or greater than 0.95, and lower or equal to 0.08 for RSMEA [38,39,40]. For the analysis of local adjustment and item elimination, in addition to factor loadings, we also considered the magnitude of the Squared Multiple Correlation Coefficient (R2), the variances and covariance, and the amount of error associated [38, 39]. Taking into account the limitations in several items identified in the exploratory study [18], and the above mentioned local adjustment indices, elimination of items would be considered, resulting in a shortened version of MHLq-YA to be tested with the samples from China, India, Indonesia, Thailand, and the United States.

For the calculation of total score, items from “erroneous beliefs/stereotypes” dimension were reverse-scored. Internal consistency was analysed in all samples through Cronbach’s Alpha and McDonald’s Omega [41], as well as the interrelation among subscales, using SPSS (SPSS Inc, Chicago, IL, version 27.0). McDonald’s Omega coefficient provides more accuracy to the approximation of internal reliability, when compared to Cronbach’s Alpha. Standard values stipulate that McDonald’s Omega values above 0.70 are acceptable, and, for Cronbach’s Alpha, a coefficient of 0.60 is the cut-off for a measure to be considered internally reliable [42]. Statistical significance was set at p < .05.

Results

Confirmatory factor analyses

The construct validity of the MHLq-YA was tested by performing a CFA with the Portuguese sample. Considering the previous results of the exploratory study of psychometric properties [18], local adjustment indicators, as well as the global adjustment, 13 items were removed from the model, namely: MHLq2, MHLq3, MHLq4, MHLq5, MHLq9, MHLq10, MHLq12, MHLq11, MHLq14 MHLq18, MHLq21, MHLq23 and MHLq24. Also, the modification indices specified the correlations between the errors of MHLq06 and MHLq13 items, the MHLq08 and MHLq17 items, as well as MHLq01 and MHLq19.

The revised version of MHLq-YA, now named Mental Health Literacy questionnaire- Short Version for adults (MHLq-SVa), fit well to the Portuguese data (X2(95) = 153.17 × 2/gl = 1.62 ; CFI = 0.95; NNFI = 0.95 RMSEA = 0.040). At the level of local adjustment, the factorial weights were globally high, ranging between 0.50 and 0.97 (Table 1).

The quadri-dimensional model showed similar goodness-of-fit indices in US (X2 (95) = 206.547 × 2/gl = 2.17; CFI = 0.97; NNFI = 0.96; RMSEA = 0.052), China (X2 (95) = 186.190 × 2/gl = 1.96; CFI = 0.95; NNFI = 0.94; RMSEA = 0.044), Thailand (X2 (95) = 180.768 × 2/gl = 1.90; CFI = 0.95; NNFI = 0.94; RMSEA = 0.048), India (X2 (95) = 257.482 × 2/gl = 2.71; CFI = 0.88; NNFI = 0.85; RMSEA = 0.072) and Indonesia (X2 (95) = 183.044 × 2/gl = 1.92; CFI = 0.95; NNFI = 0.94; RMSEA = 0.051). The model also showed a good local adjustment in these five countries (Table 1), with items loadings ranging from 0.20 to 0.93 and globally good R2 values (above 0.30). The standardized loadings revealed some concerns in two countries, regarding items MHLq13 (Indonesia), MHLq15, and MHLq27 (India).

Table 1 Standardised Factor Loadings, Standardised Error and Squared Multiple Correlations for China, India, Indonesia, Portugal, Thailand, and US.
Full size table

Reliability

The reliability of MHLq-SVa for each country is reported in Table 2. Both Alpha and Omega coefficients for each dimension and total score were close, ranging from 0.59 to 0.93.

Table 2 Descriptive Statistics of MHLq-SVa Dimensions, Cronbach’s alpha and McDonald’s Omega
Full size table

The correlations between dimensions and the total score of MHLq-SVa (Table 3) ranged from − 0.29 to 0.92. As expected, considering the nature of the items included in dimension 2 (erroneous beliefs/stereotypes), negative correlations were found between this dimension and the other three dimensions and the total score in all countries, except India.

Table 3 Means, Standard Deviations and Correlations Among MHLq Dimensions and Total Score
Full size table

Discussion

The current study aimed to provide evidence of the psychometric properties of a shorter version of the MHLq-YA in six different countries (Portugal, US, China, Thailand, India, and Indonesia).

Following the preliminary study of the psychometric properties of the MHLq-YA [18], where some items presented psychometric issues (e.g., items loading simultaneously in two factors; acceptable, but low loading values), a CFA was conducted with the Portuguese sample, suggesting the exclusion of 13 items. This resulted in a shorter version of the measure, with 16 items organized in four dimensions – Knowledge of mental health problems; Erroneous beliefs/stereotypes; Help-seeking and first aid skills; Self-help strategies – in line with the most recent definition of the MHL construct [9]. This shortened version of MHLq-YA was tested with data from five countries (US, China, Thailand, India, Indonesia). The data from these different countries globally fit the four-factor model tested. The interrelations between factors and total score confirmed the questionnaire’s consistency, with significant contributions of each dimension to the latent construct – MHL. According to the reference values [42], the internal consistency was globally acceptable in each country. The US data shown the highest internal consistency values. This revised version of the MHLq showed a good psychometric structure to be used as an assessment tool of MHL in six countries.

This study has two major strengths. First, it allowed for the development of a shorter version of a questionnaire for assessing MHL in young adults, which will be easier to administer, less time consuming to score, and less burdensome for participants. Second, this was the first time that this questionnaire was tested in different cultures, from distinct regions (Europe, North America, and Asia), in languages spoken in countries with large populations, facilitating future research focused on multicultural comparisons of MHL. There are also limitations. The sample sizes of each country varied significantly, potentially compromising comparative analyses, such as measurement invariance. Data collection procedures (paper-pencil vs. online) and timing (pre and during COVID-19 pandemic) differed between countries. The fact that some data were collected during the pandemic could have influenced participants’ responses, since mental health awareness is thought to have increased during this period (e.g., [43,44]). The difference in data collection procedures could have also affected data comparison between countries, since online and paper-pencil administration could result in different responses, particularly regarding the knowledge dimension, as answers to these items could be found online.

Future studies should address and overcome the limitations stated above, but also contribute to the strengthening and applicability of this instrument. First, considering the concerns regarding the factor loadings of some items, as well as the non-significant correlation between Factor 2 and Factors 1 and 3 in India, new data collection and analyses should be developed in India and Indonesia, in order to explore the need to revise item translation or make cultural adjustments. Second, other instruments should be considered, not only to control social desirability, but also to further study the MHL construct by means of examining concurrent validity (e.g., Mental Health Literacy Scale; [45]). Third, data collection procedures (online vs. paper-pencil) should be compared in order to examine possible differences between them. Fourth, data collection should be extended to more heterogeneous groups of participants (e.g., different age groups, different educational backgrounds, clinical samples), assuring equivalent sample sizes. Fifth, it would be interesting to test the psychometric properties of this measure with samples from other countries and languages, from other regions, such as South America and Africa. Sixth, using larger samples, measurement invariance should also be examined. Finally, a multi-method approach could be used in future research, highlighting the extent to which perceptions are consistent with others’ reports in different cultures.

Conclusion

Preliminary validation of the MHLq-SVa suggests that it is a valid and reliable measure for assessing MHL in young adults from six different countries and languages (Portugal – Portuguese, US – English, China – Chinese, Thailand – Thai, India - Hindi, and Indonesia – Indonesian). Future studies are needed to test measurement invariance and other relevant psychometric properties, allowing multicultural comparisons of MHL.