Background

The concept of person-centred care is essential for mental health services [1]. Subjective outcome measures are increasingly popular alongside measures of symptomatic remission or objective functioning [2]. Western mental health policies and studies have focused on personal recovery as an important clinical outcome [3, 4]. Personal recovery has been described as a unique process of changing one’s attitudes, values, feelings, goals, skills, and/or role and developing new meaning and purpose in life beyond the limits imposed by one’s illness [5]. The concept of recovery extends beyond removal of all symptoms or the complete restoration of functionality. Recovery involves minimising the impact of mental illness and maximising well-being (e.g., by developing valued social roles) [6]. A recent systematic review supported a conceptual framework of personal recovery consisting of the following factors: Connectedness; Hope and optimism about the future; Identity; Meaning in life; and Empowerment (abbreviated to CHIME) [7].

Effective recovery assessment tools are needed for better recovery-oriented support. A recent review found that the Recovery Assessment Scale (RAS) has been the most widely used [8, 9]; however, the Questionnaire About the Process of Recovery (QPR) is the only measure whose items all map onto the CHIME recovery framework [10, 11].

Service users, researchers, and clinicians collaboratively designed the QPR; it has high internal consistency reliability and validity [12]. An exploratory factor analysis (EFA) of the original 22-item version identified two factors; these were labelled ‘intrapersonal’ and ‘interpersonal’. Subscales examining these factors showed good internal consistency (intrapersonal: α = 0.94; interpersonal: α = 0.77) and good construct validity and reliability [10]. A 15-item version has also been developed [12] that is less burdensome and slightly more robust [13]. Community-level recovery-promotion intervention research (REFOCUS) and pre-post-evaluation of Recovery College students have used the QPR as a primary outcome measure [14, 15]. The 22-item Chinese version showed good validity and reliability as a measure of perceived levels of recovery [16].

Mental health service reform in Japan has been slower than in the other OECD countries, with psychiatric hospital care being dominant [17,18,19]. Japan requires community-based care and recovery-oriented services. Some person-centred service models were originally developed in Japan [20, 21]; however, no research has examined personal recovery using a validated assessment tool for users participating in a specific mental health service in Japan.

Accordingly, this study aimed to develop a Japanese version of the QPR (QPR-J) and examine its validity and test–retest and internal consistency reliability. We also tested the possible factor structures of personal recovery among Japanese service users using EFA with the QPR-J.

Methods

Study designs

The study was performed in two stages, using a cross-sectional and prospective cohort study design. Specifically, the first stage used a cross sectional design to develop and validate the QPR-J in users of mental health services, while the second stage used a prospective cohort design to evaluate the test-retest reliability of the tool in a subset of the participants.

Participants

We conducted a survey of mental health-related service users using the Japanese version of the QPR (QPR-J). The sample was recruited from 14 community mental health services in the suburbs of Tokyo. All the participants recruited from community mental health services had been diagnosed with mental illness and/or mental disorders by a psychiatrist. In our survey, the participants selected their own diagnosis, including the categories ‘other’ and ‘not known’, by themselves (self-report). Inclusion criteria were as follows: (1) aged ≥18 years, (2) able to participate and give informed consent, and (3) having any mental disorder. We explained the study to the participants. All participants provided a written indication of informed consent. Participation was voluntary. This study was approved by the Ethical Committee of the Faculty of Medicine, The University of Tokyo [approval No. 10890].

Development of the Japanese version of the QPR

The original QPR contains 22 items. Responses were made on a 5-point Likert scale (0 = strongly disagree, 4 = strongly agree). Scores’ possible range is 0–88. The intrapersonal and interpersonal subscales contain 17 and 5 items, respectively (possible score ranges: 0–68 and 0–20). Higher scores indicate greater recovery.

The following procedure was used to translate the QPR [22]. First, we obtained permission to translate the QPR from its original authors. Three native Japanese speakers independently translated the original instrument into Japanese. Eleven mental health professionals reviewed the three translations and collaborated to create a draft translation of the QPR-J. Second, the draft version was back-translated into English by a professional translator who was a native English speaker following the guidelines for the translation and adaptation process for patient-reported outcomes measures [22]. Then, another native professional translator checked the back-translation of the QPR. We refined the draft after we compared the original questionnaire with the back-translation. Third, the authors of the original QPR and the authors of the present study revised the back-translation based on the original concept. Fourth, we conducted pre-tests for users and peer staffs to confirm whether the items on the pre-final questionnaire were subjectively relevant and appropriate to the participants. We asked them what they thought about the items after answering the questionnaire. The participants reported that the items were easy to understand and relevant to their life situation. The survey used the finalised QPR-J; surveying ran from October to December 2015 (Additional file 1).

Validity testing

The Recovery Assessment Scale (RAS) is closely correlated with the original QPR [13]. Quality of life (QOL) is closely correlated with recovery [23]. Accordingly, the RAS and a scale measuring QOL were used to examine the QPR-J’s convergent validity.

The RAS is a user-rating measure examining personal recovery developed in the USA; a 41-item version and a 24-item version exist [9, 23]. Responses use a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). Higher scores indicate greater recovery. The validity and reliability of both versions of the RAS have been supported in the USA (Corrigan 1999, Corrigan 2004). The validity and reliability of the Japanese version of the 24-item RAS have been supported [24].

The Short Form-8 (SF-8) is a self-report measure that assesses health-related QOL; it contains subscales examining general physical health (the Physical Component Summary: PCS) and mental health (the Mental Component Summary: MCS). Higher scores indicate better QOL [25]. The validity and reliability of the SF-8–Japanese Version have been supported [26].

The QPR-J’s convergent validity was examined by calculating the Pearson product-moment correlation between the Japanese RAS and the Japanese SF-8’s mental health subscale (MCS). Values of 0.40—0.70 and >0.70 were considered moderate and strong, respectively.

Reliability testing

The test-retest reliability was assessed in a subsample of participants who completed the survey again after a two-week interval (N = 10). Intraclass correlation coefficients (ICC) were calculated as ICC (2, 1) using a two-way random effects model. We considered ICCs >0.80 to indicate excellent agreement [27].

Internal consistency reliability was examined using Cronbach’s alpha [28] calculated between scores on the full version, intrapersonal and interpersonal subscales. We considered alpha coefficients ≥0.70 satisfactory [29] and those ≥0.60 acceptable [30, 31].

Confirmatory factor analysis

We performed a confirmatory factor analysis (CFA) to test the fitness of the data to the factor structure extracted from the original 2-factor subscales of the English scale. The two factors were “the intrapersonal subscale” (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 18, 19, 21, 22) and “the interpersonal subscale” (14, 15, 16, 17, 20). We reported the goodness of fit index including the comparative fit index (CFI) and the root mean square error of approximation (RMSEA). The acceptability of model fit was judged by the recommended standards: CFI > 0.90 and the RMSEA values of 0.06 or less for a good fit and 0.08 or less for a reasonable fit [32].

Exploratory factor analysis

We conducted an EFA along with the CFA to compare the original factor solution to the one that would emerge from the data. We performed an EFA to explore factor structure of the QPR-J and reported all statistical metrics (Cronbach’s alpha for internal consistency and the test-retest reliability) related to the yielded factors.

The QPR-J’s factorial validity was evaluated using maximum likelihood estimation with Promax rotation. Maximum likelihood estimation gives us the best results when the data are relatively normally distributed. ‘Maximum likelihood estimation allows for the computation of a wide range of indexes of the goodness of fit of the model [and] permits statistical significance testing of factor loadings and correlations among factors and the computation of confidence intervals’ [33].

Eigenvalues ≥1.00 were considered to indicate factors. Assumptions regarding matrix identity and sampling adequacy were evaluated using Bartlett’s Test of Sphericity and the Kaiser-Meyer-Olkin (KMO) test. Inter-factor correlations for the final model were examined to determine the extent of correlations among factors.

All analyses were conducted using IBM SPSS Statistics, version 22.0 (IBM SPSS, Armonk, NY, USA). The CFA was conducted using IBM SPSS AMOS, version 22.0 (IBM SPSS, Armonk, NY, USA).

Results

We developed the QPR-J (cf. Additional file 1) and tested the validity of the QPR-J in a community sample of Japanese individuals. We excluded 21 respondents who did not respond to the QPR-J and analysed data from 197 respondents (90.4% of the initial 218 service users; Table 1). Respondents were mostly females (61.9%), never married (72.6%), living with their families, and diagnosed with schizophrenia (49.2%). Respondents’ average age was 42.0 ± 10.9 years. A subsample completed the QPR-J questionnaire again two weeks later to examine test-retest reliability (N = 10). They were mostly females (70.0%), were never married (50.0%), lived with their families, and had been diagnosed with schizophrenia (80.0%), and their average age was 44.7 ± 6.5 years.

Table 1 Demographic characteristics of the study participants
Full size table

Table 2 presents mean scores for the examined variables. Table 3 presents QPR-J item-level ratings. The QPR-J items 9 and 13 were rated slightly higher than the other items.

Table 2 Sample scores on the study measures (N = 197)
Full size table
Table 3 Item-level rating on the QPR-J (N = 197)
Full size table

Validity testing

The scores on the total QPR-J, intrapersonal and interpersonal subscales were significantly and positively correlated with the scores on the RAS and SF-8 MCS (Table 4). These results indicated adequate convergent validity for each scale.

Table 4 Pearson’s correlation coefficients on the QPR-J intrapersonal (17 items), QPR interpersonal (5 items)
Full size table

Reliability testing

The ICC values for the full version (0.85 [95% CI 0.35–0.97]), intrapersonal subscale (0.85 [95% CI 0.35–0.97]), and interpersonal subscale (0.89 [95% CI 0.55–0.97]) were satisfactory.

Cronbach’s alpha coefficients indicated excellent internal consistency for the full version, intrapersonal subscale (α = 0.91 and 0.90 respectively). The internal consistency of the interpersonal subscale was acceptable but relatively low (α = 0.65).

Confirmatory factor analysis

Goodness of fit index for the original 2-factor model were reasonable fit (CFI = 0.87 and RMSEA = 0.07) (Tables 5 and 6).

Table 5 Results of the confirmatory factor analysis: Goodness-of-fit indices for the two-factor QPR-J models
Full size table
Table 6 Item loadings for the confirmatory factor analysis
Full size table

Exploratory factor analysis

An EFA was conducted using maximum likelihood estimation with Promax rotation. Bartlett’s Test of Sphericity (χ2 = 1, 756, p < 0.001) and the KMO test (0.90) indicated that the correlation matrix was factorable. Five factors had eigenvalues >1.0. Item factor loading was 0.34–0.70 which was not considered as good loadings. In addition, one factor (Factor 5) had only 2 item (Table 7). The five factors accounted for 47.64% of the total scale variance. Factor 1 (Positive relationships and redefining the meaning of life, 8 items) accounted for 17.79%, Factor 2 (Improving the skills of self-assessment and literacy of the treatment, 2 items) accounted for 20.61%, Factor 3 (Accepting the illness and positive decision making, 6 items) accounted for 3.53%, Factor 4 (Support from others and motivation to change, 4 items) accounted for 3.41% and Factor 5 (Self-management, 2 items) accounted for 2.30%.

Table 7 Item loadings for the exploratory factor analysis and statistical metrics (Cronbach’s alpha for internal consistency and ICC values for test-retest reliability)
Full size table

Discussion

This study aimed to develop and examine the validity and reliability of the QPR-J. The results indicated adequate convergent validity. The QPR-J’s full version, and intrapersonal subscale showed good test-retest reliability and excellent internal consistency reliability.

The scores on the total QPR-J, intrapersonal, and interpersonal subscales were significantly and positively correlated with the scores on the RAS-J (r = 0.77, 0.78, and 0.54, respectively), which is slightly above the correlations reported in a previous study reported between the QPR and RAS (r = 0.73, 0.75, and 0.46, respectively) [13]. The scores on the total QPR-J were also significantly and positively correlated with the scores on the SF-8 MCS, which is in line with previous studies reporting correlations between the original QPR and the Personal and Social Performance Scale (including socially useful activities and personal and social relationships) [12], the QPR-Chinese and Schizophrenia Quality of Life scale [16], and the QPR-Swedish and General Quality of Life [34]. These results indicate that the QPR-J has satisfactory convergent validity.

The results also indicated that the QPR-J has satisfactory test–retest reliability (ICC values for the full version: 0.85 [95% CI 0.35–0.97], intrapersonal subscale: 0.85 [95% CI 0.35–0.97], and interpersonal subscale: 0.89 [95% CI 0.55–0.97]). Test–retest reliability of the original QPR (n = 88) was ‘fair to good’ for the total QPR (ICC = 0.74, 95% CI 0.63–0.82) and ‘good’ for intrapersonal (ICC = 0.75, 95% CI 0.64–0.83) and interpersonal (ICC = 0.66, 95% CI 0.53–0.77). The test–retest assessments in our study were performed with a small sample size (N = 10). Therefore, future validation studies of the Japanese version of the questionnaire should better assess the test–retest reliability of the measure. Future validation studies of the QPR-J with large sample sizes sufficient to evaluate test–retest reliability are needed.

Cronbach’s alpha coefficients indicated excellent internal consistency for the full version and intrapersonal subscale (α = 0.91 and 0.90, respectively). The internal consistency of the interpersonal subscale was acceptable but relatively low (α = 0.65). These results were consistent with internal consistency of the original QPR (α = 0.89, 0.90, and 0.49). Previous studies indicated excellent internal consistency for the full version (QPR-Chinese: α = 0.90 and QPR-Swedish: α = 0.91). An independent evaluation of the psychometric properties of the original QPR [13] recommended 15-item versions excluding the QPR interpersonal subscale for use in research and clinical practice because of the poor internal consistency of the QPR interpersonal subscale. Further research to evaluate the psychometric properties of QPR-J with a sufficient sample size is needed.

An EFA of the original 22-item version identified two factors; these were labelled ‘intrapersonal’ and ‘interpersonal’ [10]. The EFA supported a 3-factor structure (Self-Empowerment, Effective Interpersonal Relationships, and Rebuilding Life) [16] in the QPR-Chinese and one-factor structure summarized as intrapersonal factors in the QPR-Swedish [34]. An EFA of the QPR-J identified five factors. When compared with the two factors of the original scale [10], the ‘Interpersonal’ subscale of the original QPR was divided into two factors; ‘Factor 1: Positive relationships and redefining the meaning of life’ (Item 17) and ‘Factor 4: Support from others and motivation to change’ (Item 15 and 16). Further, the attitude of Japanese people toward others and toward their life may be divided into ‘active relationships’ (Positive relationships and redefining the meaning of life) and ‘passive relationships’ (Support from others and motivation to change). Cultural differences in values of social relationships might affect these classifications. Japanese are strongly motivated toward relational harmony and interdependence [35]. An EFA of the QPR-J extracted ‘recovery process through positive relationship with others (Factor 1)’ and ‘recovery process within support from others (Factor 4)’. Japanese may build positive relationships (items 4, 5, 3, and 17) and recognise and define the meaning of life through the relationship (items 6, 10, 18, and 19) [Factor 1]. In addition, belongingness and support from others (item 16 and 13) and support for others (item 15) might motivate them to change (item 9) [Factor 4]. Some of the factor loadings were less than .50, which indicated poor factor loading, and the five factors accounted for 47.64% of the total scale variance. These trends of poor loadings did not change even after limiting the diagnosis to schizophrenia and/or mood disorders (Additional file 2: Table S1 and Table S2). Further research with a large sample size sufficient to allow EFA sufficiently is needed.

The result of the CFA indicated reasonable goodness of fit indexes for the original 2-factor model. Therefore, we adopted the factor structure extracted from the original 2-factor based on our CFA results. Some of the items had a factor loading below 0.5 in the CFA. Item 20 (‘I realise that the views of some mental health professionals is not the only way of looking at things’) had a factor loading 0.22 in the QPR-J and 0.38 in the QPR [13]. It may be too vague and difficult for the participants to think about “some mental health professionals” and “way of looking at things”. Item 13 (‘I am able to access independent support’) also had low factor loadings in the QPR-J (0.27). All the participants were recruited from community mental health services, and most used independent support. A careful review of item wording and diversity of sampling is needed in a replication study.

The first research examining recovery was qualitative and originated in the Western Europe and North America. Some studies that examined recovery have discussed international differences in the conceptualisation of personal recovery [36, 37]. A cross-cultural study examining recovery using the RAS found that “personal confidence and hope” and “reliance on others” varied in conceptualisation between the USA and Japanese samples [38]. Another cross-cultural study examining well-being found that personal control and relational harmony most strongly predicted well-being in the USA and Japan, respectively [39]. A cultural comparison study suggested that patients from Western European and Japanese cultures have different typical recovery needs [40]. Future research should further examine the Japanese conceptualisation of recovery and well-being.

Here are some of the comments we received from the participants: “I think that answering the QPR-J was a good opportunity to think about myself”, ‘I hope the staff will receive recovery oriented-education and provide better services’. Based on such comments, we believe that researchers should assess the process of recovery using the QPR-J and implement the best practices of recovery-oriented services in Japan.

This study has the following limitations. First, we drew respondents from suburbs in specific urban areas and using mental health services; therefore, the sample may not accurately represent the Japanese population. Further research is needed with more diverse samples. Second, Japan and the UK may have importantly different cultural and mental health care contexts. Future research should develop or adapt questionnaires to suit the Japanese cultural context. Finally, as we mentioned above, future validation studies with large sample sizes sufficient to evaluate test–retest reliability and to allow sufficient EFA of the QPR-J are needed.

Conclusions

The QPR-J is valid, and it reliably measures recovery among mental health service users in Japan.