Background

Self-management is a subjective day-to-day approach including medical management, role management and emotional aspects of their condition, which is engaged to improve health conditions and maintain wellness [1]. This self-management starts with a person addressing their own difficulties and concerns in their daily life [1]. Self-management improves several aspects of life with chronic illness, such as symptoms, self-efficacy and Quality of Life (QOL) [2, 3].

Mental health guidelines and guidance have pointed out the importance of self-management to complement pharmacotherapy and psychotherapy in the treatment of mental illness [4,5,6]. The reported benefits of self-management for mental health include higher self-efficacy, a lower relapse rate, higher sense of coherence, better self-rated health, and fewer comorbidities [7, 8]. Additionally, self-management is essential to promoting “recovery” [8]. Recovery means not only improvement in clinical aspects such as a reduction of symptoms, but also personal aspects such as transformation of attitudes or ways of thinking, enabling patients to live their life more satisfactorily and with hope [9, 10].

To support the recovery of the people with mental health challenges, it is important to assess and support their self-management behavior as the outcome of their self-management. Behavior is all actions, not only those externally observable, but also inner actions like thoughts or recognitions. In the mental health area, most of the scales for measuring self-management are limited by the diagnosis or situation of use such as a psychiatric vocational rehabilitation service. Nowadays, much of the mental health welfare support services within a community are provided to users without the distinction of a diagnosis. Furthermore, a program style that doesn’t require information regarding the diagnosis is not uncommon when the aim is to facilitate recovery. Thus, a trans-diagnostic scale that measures self-management behavior is needed to improve mental health support services in the community. The Patient Activation Measure 13 for Mental Health [11] measures mental health self-management attitudes, and a Japanese version is available [12], but not for self-management behavior. It is desirable to be able to measure not only attitudes, but also behaviors, because motivation and cognitive function are often impaired by mental illness.

The Mental Health Self-management Questionnaire (MHSQ) [13] was developed to measure the use of self-management strategies to recover from mood and anxiety disorders as users empower themselves and take responsibility in their recovery [13]. The items were extracted from qualitative research that clarified strategies to recover from mood and anxiety disorders [14]. MHSQ contains 18 items and consists of three factors (Clinical, Empowerment, and Vitality). Responses are given on a 5-point Likert scale ranging from 0 (Never used) to 4 (Very often used). A higher score indicates more frequent use of self-management strategies over the past two months. The three factors were described as follows: “Clinical” refers to getting help and using resources, “Empowerment” relates to building upon strengths and a positive self-concept to gain control, and “Vitality” refers to an active and healthy lifestyle [13]. MHSQ is reported to have satisfactory reliability, examined by Cronbach’s α (Clinical = .69, Empowerment = .81, Vitality = .75) and the test-retest reliability of each factor using intraclass correlation coefficient (ICC) (Clinical: ICC = .78, 95%CI [.68, .85]; Empowerment: ICC = .76, 95%CI [.65, .84]; Vitality: ICC = .85, 95%CI [.78, .90]) [13]. The validity of the original MHSQ was examined by content validity, concurrent validity, convergent validity, and discriminant validity from the recovery concept [13]. MHSQ is an adequate scale to measure the use of self-management strategy which tends to be highly individualized. Although the original MHSQ focuses on persons with mood and anxiety disorders, it is reasonable to adopt it for other mental diagnosis because of the following points. In the development process of MHSQ, disease-specific items were excluded [13]. It is suggested that the core aspects of self-management for a chronic illness are common, and are as follows: problem solving, decision making, resource utilization, forming a patient/healthcare provider partnership, and taking action [1, 15]. It can be considered that the remaining items were a common strategy to deal with mental health challenges. That is why we recognized that MHSQ can be adopted to other mental diagnoses. Accordingly, it is assumed that if a Japanese version of MHSQ were to be developed, it would contribute to the assessment of mental health self-management behavior in people with mental health challenges.

The aim of this study is to develop a Japanese version of MHSQ (MHSQ-J) and to clarify its psychometric properties among people with mental illness living in Japan. To verify reliability and validity, internal consistency, test-retest reliability, structural validity, hypotheses testing and cross-cultural validity, these components are examined according to the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist [16, 17]. We aimed to verify the following hypotheses. MHSQ-J has a positive correlation with scores of the scales associated with self-management behavior and personal recovery and has a negative correlation with scores of the scales associated with clinical recovery.

Method

Ethical considerations

The study protocol was approved by the Ethical Committee of the Graduate School of Medicine/Faculty of Medicine, the University of Tokyo (11513). Aims, procedures, the voluntary nature of participation, anonymity, and privacy protection were explained using a Participant Information Sheet. In addition, participants were informed that refusal or suspension of participation would not cause any disadvantage. Participants gave their consent by responding to the questionnaire.

Study design and participants

This is a validation study for MHSQ-J, and its psychometric properties were validated according to the COSMIN checklist. An anonymous survey administered to people living in the community with mental illness was conducted from July to October in 2017. We recruited 295 outpatients from outpatient mental health clinic A (site A), B (site B), and the psychiatric mental health outpatient service C (site C) in a psychiatric hospital. All the facilities were in the Kanto region. Clinic A is in a commuter town, and clinic B is in an office building of a large city. Site C has over 600 beds, and is separate from an alcohol use disorder specialty outpatient facility.

Inclusion criteria were: outpatients ≥20 years old that understood Japanese. Outpatients who were regarded as having mental health instability by the professional staff working at the site were excluded. Mental health instability was considered in cases of the following: 1) Patients for whom answering this questionnaire would be an excessive burden and participation would be a hindrance to their treatment because of their symptoms or participation in a clinical trial; 2) the relationship with their doctor was not well established because it was a first doctor visit or had just started recently, or, being prone to problems from the past. At site A, as a first step, the doctor asked an eligible patient to participate in this study and to meet the researcher after the consultation. Subsequently, the researcher provided an explanation about this research to patients that agreed to meet us, and obtained consent for participation. At sites B and C, the researcher confirmed patient eligibility or the conditions which the specialists at the facility considered patients to be ineligible. The researcher asked patients about eligibility conditions (e.g., “Is this your first visit to this clinic?”) when necessary. If the patient was eligible, the researcher asked if they would agree to participate in this study. The researcher or facility staff explained the study to patients using a Participant Information Sheet.

Test-retest reliability was tested among participants who agreed to receive the retest by mail (n = 104).

Measurements

Development procedure of the Japanese version of MHSQ (MHSQ-J)

The following procedure was used to translate MHSQ, based on the principles described by Wild [18].

  1. (1)

    Preparation and forward translation: first, permission to translate MHSQ into Japanese was obtained from its original author. MHSQ was originally developed in French, and the original author provided an English version for Japanese development. Two native Japanese speakers, who were mental health researchers having experience in psychiatric nursing, carried out independent translations of MHSQ from English to Japanese.

  2. (2)

    Reconciliation: five mental health researchers in face-to-face meetings including the author reached a consensus on a draft Japanese translation of MHSQ that best reflected the literal and conceptual content of the English version of MHSQ.

  3. (3)

    Back translation: the draft version was back-translated into English by a professional native English translator, who did not know about the English version of MHSQ.

  4. (4)

    Back translation review and harmonization: the researchers who conducted the forward translations reviewed the back-translation to ensure the literal and conceptual equivalence of the translation. The original author also confirmed the back translation.

  5. (5)

    Cognitive debriefing and finalization: ten Japanese people using community mental health services tested the pre-final Japanese version of MHSQ to confirm whether the items on the questionnaire were subjectively relevant and appropriate to the situation of their self-management behavior. They were also asked what they thought about the items after answering the questionnaire. The authors confirmed the cognitive equivalence of the translated MHSQ-J (Additional file 1).

Scales for verifying hypotheses testing

Five scales were used for hypotheses testing to verify the construct validity of MHSQ-J. The scales assessing correlation with self-management behavior were the Self-management Skill Scale (SMS) [19] and the University of Tokyo Health Sociology version of the SOC scale (SOC-3-UTHS) [20,21,22]. The scales assessing correlation with recovery were the Japanese version of the Self-identified Stage of Recovery Part-B (SISR-B) [23, 24], the Japanese version of the Flourishing Scale (FS-J) [25, 26], and the Japanese version of the 12-item self-administered version of the WHO Disability Assessment Scale 2.0 (WHODAS 2.0) [27,28,29].

  1. (1)

    Scales related to Self-management behavior

Self-management skill scale

Engaging in desirable health behavior is a part of self-management behavior. Cognitive skills such as effective ways of thinking to achieve the behavior are related to health behavior [30]. The Self-management Skill Scale (SMS) is a scale for measuring general cognitive skills that are effective in realizing the behavior that one wishes and that can be utilized in various situations [19, 31]. The relation of the high score of SMS to various health behaviors was previously confirmed [32, 33]. SMS is a 10-item scale including six reverse-scored items. The items are rated on 4-point Likert scales ranging from 1 (Not applicable) to 4 (Applicable). A higher score indicates an abundance of self-management skill. Cronbach’s α coefficient was reported as .75 among university students in Japan [19].

The University of Tokyo health sociology version of the SOC scale ver.1.2

The University of Tokyo Health Sociology version of the SOC scale (SOC-3-UTHS) ver.1.2 is a scale assessing Sense of Coherence (SOC). SOC is a concept that reflects the ability to cope with stress in Salutogenesis theory [34, 35]. As problems in mental health are linked with stress, self-management and the ability to cope with stress was assumed to have a relationship. Positive and significant associations were reported between SOC scores and self-management behavior among people with chronic illness [36]. SOC-3-UTHS ver.1.2 consists of three items. Responses are on a 7-point scale, from 1 (Not applicable) to 7 (Applicable). The items are scored so that a higher score indicates a higher ability to cope with stress. Cronbach’s α coefficient was reported as .84 among a Japanese population [21].

  1. (2)

    Scales related to recovery

Japanese version of self-identified stage of recovery part-B

Personal recovery, an important aspect of recovery from mental illness [9], was assessed using the Japanese version of Self-identified Stage of Recovery Part-B (SISR-B), a 4-item scale to assess the key component of the process of recovery from mental illness [23]. Items are rated on a 6-point scale, ranging from 1 (Disagree strongly) to 6 (Agree strongly). Cronbach’s α coefficient was reported as .82 among people with mental illness living in the community. A higher score indicates a higher level of recovery [24].

Japanese version of flourishing scale

The Japanese version of the Flourishing Scale (FS-J) was used to measure psychological well-being as a state of increased personal recovery [37]. FS-J is an 8-item scale. Items are rated on a 7-point scale, ranging from 1 (Strongly disagree) to 7 (Strongly agree). Higher scores indicate respondents that view themselves in positive terms in diverse areas of human functioning. Cronbach’s α coefficient was reported as .95 among undergraduate students in Japan [26].

Japanese version of the 12-item self-administered version of WHO disability assessment scale 2.0

Because self-management affects clinical recovery, such as through a reduction of symptoms or disability [9], the Japanese version of the 12-item self-administered version of the WHO Disability Assessment Scale 2.0 (WHODAS 2.0) was used to assess the level of clinical recovery. This scale assesses the degree of disability due to various health conditions, with responses on a 5-point scale ranging from 1 (None) to 5 (Extreme/cannot do), where a higher score indicates a higher level of disability [29]. Cronbach’s α coefficient was reported as .92 among preoperative patients in Japan [38].

Demographic variables

Information regarding socio-demographic and clinical characteristics was collected, including age, sex, income, marital status, work status, educational background, cohabitants, diagnoses of psychiatric disorders, period from first visit to psychiatrist, an experience of hospitalization due to mental health problems, the use of mental health support services, frequency of visits with psychiatrists, and severities of depression and anxiety. The severities of depression and anxiety were assessed using the Japanese version of the Patient Health Questionnaire-9 (PHQ-9) [39,40,41] and the Japanese version of General Anxiety Disorder-7 (GAD-7) [41, 42], respectively. PHQ-9 has nine items, and GAD-7 has seven items. Both are based on 4-point scales. Cronbach’s α coefficient of PHQ-9 was reported as .91 among university students [43], and that of GAD-7 was reported as. 87 among university students in Japan [44].

Statistical analysis

Statistical analysis was conducted for respondents who completely answered all items of MHSQ-J.

Validity of MHSQ-J

Since there was no gold standard for assessing self-management strategy, validity was determined by assessing construct validity. Construct validity was confirmed by structural validity, cross-cultural validity, and hypotheses testing.

According to the COSMIN checklist [16], confirmatory factor analysis (CFA) was performed to confirm the fit of the factor structure of the original MHSQ to the data for structural validity. Cross-cultural validity was also examined by CFA. The following indices were used: Adjusted Goodness of Fit Index (AGFI), Comparative Fit Index (CFI), and Root Mean-Square Error of Approximation (RMSEA). AGFI and CFI values equal to or above .95 were considered a good fit [45, 46]. An RMSEA value less than or equal to .06 was considered a good fit, .08 or less indicates reasonable fit, .08 to .10 indicates a mediocre fit, and values above .10 indicate a poor fit [46, 47]. Factorial correlations were also examined.

Exploratory factor analysis (EFA) was performed to verify structural validity along with CFA. The Kaiser-Meyer-Olkin index (KMO) of sampling adequacy and Bartlett’s chi-square test of sphericity with p < .05 were examined to confirm the suitability of the data for factor analysis. KMO was compared to adequacy of standards [48]. The generalized least-squares method with Promax rotation was used for the factor extraction because it was assumed that the factors have correlations with each other. The number of factors was determined based on the Scree test.

Pearson’s correlation coefficients were examined between the scales for hypotheses testing and the total score and each factor of MHSQ-J. The hypotheses for MHSQ-J score were as follows: positive associations with the scores of SMS, SOC-UTHS ver.1.2, SISR-B, and FS-J, moderate negative association was assumed with WHODAS 2.0. Pearson’s correlation coefficients of .40–.70 and > .70 were considered moderate and strong, while .20–.40 and < .20 were considered weak and poor correlations, respectively.

Reliability of MHSQ-J

To evaluate reliability, internal consistency and test-retest reliability were calculated. Cronbach’s α coefficient for total score and each subscale of MHSQ-J were examined to assess internal consistency. Sufficient internal consistency was assumed for a Cronbach’s α greater than .70 [49, 50]. Test-retest reliability was tested among participants who agreed to receive the retest by mail (n = 104). Participants were asked to take a re-administration of MHSQ-J two weeks later after the initial questionnaire and we used data from all responses (n = 82) that were provided within a period of 8–20 days from the initial survey. Test-retest reliability was evaluated by exclusion of responses that reported significant changes in their condition between the initial survey and the test-retest survey. Participants were also asked about changes in their subjective condition (“Were there any changes in your condition since the time you previously answered this survey?”) using an original item on a 7-point Likert scale ranging from “7: Extremely improved” to “1: Extremely worsened” to confirm significant differences in their condition compared with the initial survey. We judged that there was a significant change in condition at the retest if the participants answered the question as 1, 2, 6, or 7. For test-retest reliability, ICC was examined for the total score and each subscale of MHSQ-J using the data with complete answers for MHSQ-J. ICC > .70 considered as excellent agreement [51].

Statistical analyses, except for CFA, were conducted using SPSS 22.0 for Windows. CFA was conducted using AMOS version 22. Values of p less than .05 were considered statistically significant (two-tailed test).

Results

Respondent characteristics and scale descriptions

A total of 266 participants returned the questionnaire (response rate = 90%) and 82 returned the retest (response rate = 79%). For all analyses, 23 respondents were excluded because of incomplete answers in MHSQ-J, and a total of 243 were used. Socio-demographic and clinical characteristics of the respondents are shown in Table 1.

Table 1 Socio-demographic and clinical characteristics of the respondents
Full size table

The mean age was 46.3 (SD = 12.3), and the range was between 21 and 82. Almost half of the respondents were male and not married. There were 44% who lived alone, and 41% had hospitalization experience because of mental problems. The mean frequency of hospital/clinic visits per month was 2.3 (SD = 2.0). Average period from the first visit to the psychiatric/psychosomatic doctor was 11.9 years (SD = 9.8). The mean score of total MHSQ was 38.5 (SD =11.5, range 5–71). A ceiling effect was seen with item 5, and a floor effect was seen with items 4 and 18.

Validity of MHSQ-J

Structural and cross-cultural validity

CFA was performed with all items, based on the structure of the original MHSQ (Fig. 1: Confirmatory factor analysis of MHSQ-J). The goodness-of-fit indices were not optimal, but marginal (AGFI = .830, CFI = .852, RMSEA = .072). The factorial correlation between Empowerment and Vitality was .66. The correlations between Clinical and the other two factors were .30 with Empowerment, and .15 with Vitality.

Fig. 1
figure 1

Confirmatory factor analysis of MHSQ-J

Full size image

The adequacy of data for EFA was confirmed by Bartlett’s test of sphericity (p < .001) and KMO (.842). The goodness-of-fit test for the factor extraction method was adequate (χ2 = 120.381, df = 102, p = .103). The three-factor-structure indicated by the Scree plot was optimal. The three factors of MHSQ-J explained 47.83% of the variance. Whole factor loadings are shown in Table 2. Three items (item 4, 13, and 14) loaded onto two factors with low factor loadings (< .40) and had comparable values over .20. The inspection of kurtosis and skewness indices for the total score and each factor were tested, and the results were: Total: kurtosis −.382, skewness −.018; Clinical: kurtosis −.224, skewness −.033; Empowerment: kurtosis −.347, skewness −.189; Vitality: kurtosis −.596, skewness −.042.

Table 2 Means, standard deviations (SDs), factor loadings, and Cronbach’s α coefficients for the total score and MHSQ-J factors (N = 243)
Full size table

Hypotheses testing

Significant correlations except for the Clinical subscale were confirmed as being in the same direction as the hypotheses (Table 3). The total score, the Empowerment subscale and the Vitality subscale of MHSQ-J showed a significant positive correlation with SISR-B, FS-J, SOC-3-UTHS ver.1.2 and SMS, and a significant negative correlation with WHODAS 2.0. The Clinical subscale showed a significant correlation only with WHODAS 2.0, and the correlation was very weak.

Table 3 Pearson’s Correlation Coefficients between the total and each MHSQ-J factor with the score of hypotheses testing measures
Full size table

Reliability of MHSQ-J

  1. (1)

    Internal consistency

The Cronbach’s α coefficients of the total score and each subscale of MHSQ-J, and the coefficient when an item was deleted, are shown in Table 2. Cronbach’s α of the total score, the Clinical subscale, the Empowerment subscale, and the Vitality subscale were .83, .65, .81, and .75, respectively.

  1. (2)

    Test-retest reliability

In total, there were 82 responses for the retest, and 20 responses were excluded because of the following reasons: incomplete answers for MHSQ-J (n = 7), significant change in condition at the retest (n = 9), not within the period of 8–20 days from the initial survey (n = 3), or the reply date of the retest was unidentified (n = 1). Finally, 62 responses were used to examine test-retest validity, and the mean period of the two tests was 13.9 days (SD = 1.83, range = 11–20).

The ICC scores of MHSQ-J were as follows: Total score: .84, 95% confidence interval (CI) [.75, .90], Clinical: .75, 95% CI [. 62, .84], Empowerment: .81, 95% CI [.70, .88], and Vitality: .62, 95% CI [.44, .75]. Overall, the scores showed moderately good test-retest reliability. The ICC score of each item of the Vitality subscale were also examined. The scores were as follows, item 15: .32, 95% CI [.08, .53], item 16: .66, 95%CI [.50, .78], item 17: .58, 95% CI [.39, .73], and item 18: .65, 95% CI [.47, .77].

Discussion

This study aimed to develop and verify the reliability and validity of MHSQ-J. The results indicated adequate internal consistency, test-retest reliability, structural validity, and cross-cultural validity.

Regarding structural validity, the result for the CFA using the original factorial structure showed almost acceptable factor loadings and a marginal goodness-of-fit index score. The result for the EFA was that the highest factor loadings of items 13 and 14 were for Vitality, although these items of the original MHSQ loaded in Empowerment. But it is reasonable to suppose that they belong to Empowerment as on the original scale, based on the size of the factor loadings. Therefore, it seems reasonable to consider the structure of MHSQ-J as similar to the original MHSQ among the participants in this study. The correlations between factors were moderate and positive between the Empowerment factor and the Vitality factor, but weak between the Clinical factor and the other two factors. This trend was similar to the original results (r = .48 between Empowerment and Vitality, r = .23 between Clinical and Empowerment, and r = .28 between Clinical and Vitality), and the original author indicated that the Empowerment and the Vitality subscales of MHSQ might contain related sub-aspects which differ from the Clinical subscale [13].

Concerning hypotheses testing, the scale related to self-management and recovery showed significant correlations with the total score, the Empowerment subscale, and the Vitality subscale of MHSQ-J in the hypothesized direction. On the other hand, the Clinical subscale showed no correlation with any scales used for hypotheses testing in this study. This seems consistent with the results for the Clinical subscales of the original MHSQ, which showed a significant correlation of smaller than .30 on the scales of symptoms and on the social participation scale [13]. The original author explained that “strategies from the Clinical subscale, such as consulting a mental health professional, might be implemented earlier in the recovery process when symptoms are most severe” [13]. Furthermore, while WHODAS 2.0 was used to assess mental health clinical recovery, it might not be sensitive enough to measure clinical recovery in mental illness. WHODAS 2.0 measures the disability caused from all health problems including physical problems, which might be one of the reasons weakening the correlation between the Clinical subscale and clinical recovery.

In this study, MHSQ-J showed good reliability in terms of reasonable Cronbach’s α coefficients and ICCs. The Cronbach’s α coefficients indicated excellent internal consistency in the total score, the Empowerment subscale, and the Validity subscale. The internal consistency of the Clinical subscale was acceptable, but relatively low. This result was consistent with the results for the original MHSQ (Clinical = .69, Empowerment = .81, Vitality = .75) [13].

The ICCs of the total score, the Clinical subscale, and the Empowerment subscale were satisfactory, and the ICC of the Vitality subscale was acceptable, but relatively low, especially for item 15.

In addition, a ceiling effect on item 5 (the use of medicine), and a floor effect on items 4 and 18 (participation in a group which supports or helps oneself, and exercises to relax, respectively), were seen in this study. Regarding the ceiling effect, most of the participants took medication frequently, because they were all outpatient users. Regarding the floor effect, the groups described in item 4 and the activity for relaxation described in item 18 are not familiar to all people in Japan with mental illness.

This study has limited generalizability. Participants were limited to psychiatric outpatients. And the participants were also limited in terms of who could answer the questionnaire and their symptoms, such that cognitive function and concentration seemed to be high. We did not collect information about the diagnosis from the clinic. This was also a limitation of this study as we could not eliminate a participant that uses psychiatric or psychosomatic outpatient services without psychiatric illness.

This is the first scale to measure the usage of mental health self-management strategies in Japan. In this study, the small difference between the original and the Japanese version of MHSQ in model fit and factor structure probably is based on cross-cultural differences and differences in the participants. Although we developed and analyzed this scale faithfully to the original scale, the results indicate the need to modify some items from a cross-cultural and trans-diagnostic point of view. The present study could not clarify the nature of each factor in MHSQ-J. To provide greater usefulness for MHSQ-J, and an understanding of self-management for mental health, further study is needed to more readily adapt services for people with mental illness living in the community in Japan, and to clarify which factors are more important at various points of treatment, or depending on symptoms.

Conclusion

MHSQ-J is valid and it reliably measures the use of self-management strategies for mental health among people with mental illness living in a Japanese community.