Introduction

Social health is known for being a key tenet of recovery in those with mental illness. Included in a broadening of the concept of health in the middle of the 20th century (World Health Organization 1949), the concept of social health generally focuses on social activities, social well-being, social network quality, interpersonal communication, social support, and social role participation and satisfaction (Castel et al. 2008). Russell (1973) has defined social health as “that dimension of an individual’s well-being that concerns how s/he gets along with other people, how other people react to her/him, and how s/he interacts with social institutions and social mores.” In 1957, Greenblatt proposed five principal areas comprising recovery from mental illness. Four of the five principal areas, vocational capacity, educational capacity, family, and social aspects of the community, address the constructs of social health. More recently, Liberman (2008) suggests recovery from mental illness be defined as a 2-year duration of satisfying peer relationships, cordial family relations, and engagement in productive activity (e.g. work or school), amongst other things. Additionally, Luecht and Lasser (2006) believe recovery from mental illness consists not only of symptom remission, but quality of life and social functioning.

Social functioning is an important aspect of one’s overall social health, referring to an individual’s ability to function in community, social, or occupational domains (Cornblatt et al. 2007). Specific components of social functioning measured in current research include independent living, work, recreation, romantic relationships, family and social relationships, and financial concerns (Keck et al. 1998; Rosa et al. 2007; Tohen et al. 2000; Zarate et al. 2000). Measures of social functioning have been widely used in social health assessment when attempting to evaluate mental health recovery and identify areas that can be improved with targeted mental health services (Barnes et al. 2008; Bauwens et al. 1998; Birchwood et al. 1990; Dijkers et al. 2000; Eisen et al. 1994; Newton-Howes et al. 2008; Siegel et al. 2006; Zarate et al. 2000). Although good social functioning is not synonymous with a reduction in mental health symptoms, severity of depression, or episodes of psychosis (Zanello et al. 2006), deficits in social functioning can inhibit long-term recovery and even predict relapse (Barnes et al. 2008; Bauwens et al. 1998; Cornblatt et al. 2007; Keck et al. 1998; Patterson et al. 1997; Zarate et al. 2000). Strakowski and colleagues studied functional outcomes in psychiatric patients and found that nearly all patients who were in remission 8 months after hospitalization still had persistent impairment in at least one area of social functioning. Less than half of patients achieved good functional outcomes in at least three of the four social functioning areas that were measured (Strakowski et al. 2000).

In addition to identifying areas for further improvement in mental illness recovery, the measurement of social health and functioning has other implications. Impaired social functioning has been shown to predict susceptibility to schizophrenia, allowing care providers to target susceptible adolescents for early intervention (Cornblatt et al. 2007). Social functioning is positively related to quality of life in older adults with psychiatric illnesses (Patterson et al. 1997; Ritsner et al. 2000) and social dysfunction has also been associated with depression severity, lack of full remission from depression, and increased anxiety (Gift et al. 1980; Rytsala et al. 2006; Upadhyaya et al. 2000). Additionally, social health and functioning assessment can provide insight into treating patients with personality disorders (Quinton et al. 1995; Seivewright et al. 2004). Studies investigating differences in social functioning between psychiatric diagnostic categories have generally found that patients with schizophrenia display the most social dysfunction, followed by patients with bipolar, depressive, and affective disorders (Cannon et al. 1997). Patients with anxiety disorders typically display the least dysfunction of the three groups (Sanderson and Andrews 2002).

Several brief assessment tools have been developed to assess social health, including the Social Functioning Questionnaire (SFQ; Tyrer et al. 2005), Functioning Assessment Short Test (FAST; Rosa et al. 2007) and Multidimensional Scale of Independent Functioning (MSIF; Berns et al. 2007). Of these instruments, the 8-item SFQ has been widely used and researched. However, in many clinical and research settings, measures that are solely designed to assess social health have not been used, and adding even the brief 8-item SFQ to existing assessments will increase overall assessment burden on both clients and staff. In settings where provider time with patients is limited and paperwork is already extensive, the use of brief instruments to assess individual changes in social health could help physicians and mental health professionals individualize treatment strategies for their clients’ recovery. Many broader assessment tools that are already being used include items that can also be used to measure social health. Therefore, in an effort to limit assessment burden on clients and staff in both research and clinical settings, and to allow clinicians already using broader assessment tools to evaluate their clients’ social health and functioning, it is useful to evaluate whether items within currently used assessment tools can be used to measure social health and functioning. This would also allow researchers and/or clinicians to conduct retrospective studies evaluating changes in social health in their patients over time.

Our objective was to identify and evaluate a set of social health scales comprised of items from two widely used mental health recovery assessment tools: the California Quality of Life (CA-QOL), used throughout California, and the Mental Health Statistics Improvement Program Consumer Survey (MHSIP) mental health outcomes assessment tool, used across the United States. Since validated, published measures of social health do not currently exist, we identified a social functioning measure that overlaps with the construct of social health to use as a comparison measure in evaluating the new social health scales. The identified items from the CA-QOL and MHSIP were investigated to determine how they compared with the validated Social Functioning Questionnaire (SFQ). Specifically, we sought to compare psychometric properties of the identified CA-QOL Social Health Scale and MHSIP Social Health Scale with those of the SFQ, to determine whether the identified scales appear to measure the same construct as the SFQ, and to determine whether the identified scales would replicate the SFQ’s pattern of results when describing differences in social health between different demographic and diagnosis groups. In addition, we sought to determine whether our proposed scales could accurately describe levels of social health among people receiving services within a large county adult mental health system based on demographic and diagnosis information (e.g. individuals living in supervised living facilities would be expected to have poorer social health than individuals living independently).

Methods

Participants

Data were obtained from a large scale assessment of client reported outcomes in San Diego County Adult and Older Adult Mental Health Services. Twice per year (May and November), adults aged 18 and older receiving psychological and behavioral treatment services complete an anonymous self-report questionnaire in the presence of a clinician at their provider site during a 2 week data collection period. For this study, we used data collected in November of 2007. The questionnaire was distributed in 2 parts: part 1 included demographic information, the MHSIP, and the CA-QOL; part 2 included the SFQ and questions regarding a recent local wildfire disaster. Eight hundred ninety-two surveys were returned. Response rates varied by clinic; however, the overall study response rate for completing any portion of the survey was 77%. However, of the 892 respondents, 112 only answered demographic questions on page 1 and an additional 171 chose not to or were not asked to complete any of part 2 of the survey. Many clients are burdened by these questionnaires so it is not unusual for them to skip part 2, which is referred to as the “supplemental” section and is not required by some clinics. Because this study examined survey items from both sections of the survey, only participants who responded to parts 1 and 2 were included in the analyses. Thus, the study sample size was 609 participants. All participants received the measures in the following order: MHSIP, CA-QOL, and SFQ. Due to incomplete response patterns, final sample size ranged from 566 to 609 for the CA-QOL, MHSIP and SFQ comparisons.

Demographic characteristics for the sample were as follows: 40.7% lived independently, 49.9% were Caucasian, 43.6% were female, and mean age was 42.5 (SD = 14.6). We obtained diagnosis information from the San Diego County Adult and Older Adult Medical Information System, a patient record holding database. Participants fell into the following categories: 36.2% were diagnosed with schizophrenia or schizoaffective disorder, 23.0% were diagnosed with major depressive disorder, 15.0% were diagnosed with bipolar disorder, 8.9% were diagnosed with other depression disorder, 5.8% were diagnosed with anxiety disorder, and 5.3% were diagnosed with an unspecified psychotic disorder.

Table 1 Descriptive statistics and correlation coefficients between subjective CA-QOL items and SFQ total score
Full size table

Measures

California Quality of Life (CA-QOL)

The CA-QOL is a 40-item survey that was developed to assess patient reported outcomes in the California Adult Performance Outcome System. The CA-QOL was modeled after the Quality of Life Interview Short Form (Lehman 1988) which is based upon a conceptual model that incorporates objective life conditions and subjective satisfaction with life conditions (Andrews and Withey 1976; Campbell et al. 1976). The main constructs of the CA-QOL objective scales are family contacts, social contacts, finances, and arrests. Items on the objective scales ask about frequency of family and social contacts with the following response options: at least once a day, at least once a week, at least once a month, less than once a month, not at all, or not applicable. Items asking about adequacy of finances are rated as yes/no. One item asks about frequency of arrest, with 5 response options ranging from 0 to 4 or more arrests in the past month. The main constructs of the CA-QOL subjective scales are satisfaction with life, living situation, family relations, social relations, daily activities, leisure activities, safety, and health. Items on the subjective scales ask about level of feeling/satisfaction with various outcomes on a 7-point scale. Response options for these items are: delighted, pleased, mostly satisfied, mixed, mostly dissatisfied, unhappy, or terrible. Work and religious activities are not included in the questionnaire. We scored all items so that higher numbers corresponded with more dissatisfaction/problems. Reliability of the CA-QOL in the development pilot study was high (Cronbach’s alpha = .93; California Department of Mental Health 1999).

Mental Health Statistics Improvement Program Consumer Survey (MHSIP)

The version of the MHSIP Consumer Survey used in this study consists of 36-items designed to assess the care of persons with mental illness and is widely used in public mental health systems (Center for Mental Health Services 1996). Seven domains are assessed: general satisfaction, perception of access to services, perception of quality and appropriateness of care, perception of participation in treatment planning, perception of outcomes of services, perception of functioning, and perception of social connectedness. Each item is a declarative statement. Response options ranged on a 5-point scale from strongly agree to strongly disagree, where higher numbers corresponded with greater disagreement, and thus greater dysfunction.

Of forty-eight states using consumer surveys in 2001, 45 of these states targeted adults with serious mental illnesses, while 26 surveyed adults with other mental illnesses (NASMHPD Research Inc. 2002). Thirty-seven of these states distribute the surveys state-wide and 11 limit the distribution to select areas or sampling frames. As of 2001, thirty-eight states had implemented a version of the MHSIP consumer survey to assess consumer perception of care, with the number of items on the MHSIP versions ranging from 19 to 40 (NASMHPD Research Inc. 2002). Reliability of the MHSIP was high in a pilot study (Cronbach’s alpha = .95; Minsky and Lloyd 1996).

Social Functioning Questionnaire (SFQ)

The SFQ is an 8-item instrument measuring an individual’s perception of functioning and was adapted from the longer Social Functioning Schedule (Remington and Tyrer 1979; Tyrer et al. 2005). The items cover aspects of work and home tasks, financial concerns, relationships with family, sexual activities, social contacts, and spare time activities. The exact 8 items are: completed my tasks at work and home satisfactorily; find my tasks at work and at home very stressful; have no money problems; have difficulties in getting and keeping close relationships; have problems in my sex life; get on well with my family and other relatives; feel lonely and isolated from other people; and enjoy my spare time. Response options are: 0 (most of the time), 1 (quite often), 2 (sometimes), and 3 (not at all). Items were all coded so that 0 corresponded to no problems with social functioning and 3 corresponded to severe problems with social functioning. The total SFQ score is a sum of the individual item scores, and can range from 0 to 24. Reliability and construct validity has been shown to be good (Remington and Tyrer 1979).

Procedures

We identified items from the CA-QOL and MHSIP based on their conceptual correspondence with the construct of social health. Ten items from the CA-QOL subjective scales assessing satisfaction with family relations, social relations, daily activities, and leisure activities; 4 items from the CA-QOL objective scales assessing frequency of family contacts, frequency of social contacts, and arrests; and 8 items from the MHSIP assessing functioning and social connectedness were retained for further investigation. The items from the CA-QOL and MHSIP are presented in Table 1 and Table 2, respectively. Three scales were created from retained items that produced moderate or large correlations with the SFQ (≥.40; REF). The identified CA-QOL Social Health Scale and MHSIP Social Health Scale were then examined for their relationship with the SFQ total score. This study was approved by the University of California, San Diego Institutional Review Board and the San Diego County Mental Health Services Research Committee. The authors have no known conflicts of interest and certify their responsibility for this manuscript.

Table 2 Descriptive statistics and correlation coefficients between MHSIP items and SFQ total score
Full size table

Statistical Analysis

Data were analyzed using Statistical Package for the Social Sciences (SPSS), Version 16.0 (SPSS Inc. 2008). We examined Pearson correlations for the identified CA-QOL and MHSIP items with the SFQ total score. We then evaluated psychometric properties of the identified scales, including calculating scale reliability coefficients. Finally, we examined Pearson correlations for the identified CA-QOL Social Health Scale and MHSIP Social Health Scale with the SFQ total score. We dichotomized items (strongly agree and agree vs. neutral, strongly disagree, and disagree; delighted, pleased, and mostly satisfied vs. mixed, terrible, unhappy, and mostly dissatisfied) to examine percent of clients reporting agreement with MHSIP items and satisfaction with CA-QOL items. We then examined means for each of the social health scales as a function of demographic and other descriptive statistics, including gender, ethnicity, education, living situation, and psychiatric diagnosis. We used MANOVAs and follow up ANOVAs to test whether means differed significantly, and Bonferroni Post-Hoc tests to determine which groups differed.

Results

Pearson correlations resulting from the identified CA-QOL objective scale single item comparisons with SFQ total scores were weaker, ranging from .07 to .22. Due to these weak correlations with the SFQ, we did not evaluate these items further in our analysis.

Pearson correlations for the identified CA-QOL subjective items with SFQ ranged from .48 to .62, and all were significant at the 0.01 level (see Table 1). Seven of the 10 CA-QOL items produced correlation coefficients greater than .54 when compared to the SFQ. The CA-QOL item assessing satisfaction with family relations demonstrated the weakest correlation with SFQ, while CA-QOL items assessing satisfaction with fun, spare time, and time spent with other people demonstrated the strongest correlations with SFQ.

Pearson correlations identified MSHIP items with SFQ ranged from .42 to .50, and all were significant at the 0.01 level (see Table 2). Four of the 8 MHSIP item by SFQ matches produced correlation coefficients greater than .45. MHSIP items assessing ability to handle situations when they go wrong and having needed support in a crisis demonstrated the weakest correlations with SFQ. MHSIP items assessing feeling of belonging in the community and ability to do desired activities demonstrated the strongest correlation coefficients with SFQ.

All of the identified CA-QOL subjective items and all identified MHSIP items met criteria for inclusion in the scales. Scale 1 (CA-QOL Social Health Scale) was a mean of the 10 CA-QOL items and scores ranged from 1 to 7; scale 2 (MHSIP Social Health Scale) was a mean of the 8 MHSIP items and ranged from 1 to 5; and scale 3 (Combined Social Health Scale) was a mean of all 18 items that were recoded so that items from the CA-QOL and MHSIP were weighted equally and had a possible range of 0–12 (CA-QOL items had possible values of 0, 2, 4, 6, 8, 10, and 12, and the MHSIP items had possible values of 0, 3, 6, 9, and 12).

As shown in Table 3, comparisons of the identified CA-QOL Social Health Scale and MSHIP Social Health Scale with SFQ produced Pearson correlation coefficients of .70 and .56, respectively. The Pearson correlation coefficient for the Combined Social Health Scale with SFQ was .70. Cronbach’s alpha for the scales ranged from .93 to .94.

Table 3 Correlation coefficients between SFQ total score and identified scales from the CA-QOL and MHSIP
Full size table

Means for each of the scales across gender, ethnicity, education, living situation, and diagnosis groups are presented in Table 4. Participants who reported being American Indian, homeless, and having major depressive disorder also reported the most dysfunction on the identified scales and on the SFQ.

Table 4 Descriptive characteristics and Social Health Scale means across gender, ethnicity, education, living situation, and diagnosis groups
Full size table

For the MANOVAs, Box’s M test of homogeneity of covariance was significant (P < .001) for 4 of the 5 models (i.e. gender, ethnicity, education, living situation, and diagnosis). However, Levene’s homogeneity of variance test was not statistically significant for any of the four DVs at the P = .05 level, indicating that the assumptions of homogeneity of variance were not violated. Using Wilk’s criterion (Λ) as the omnibus test statistic, the combined dependent variables resulted in significant main effects for living situation, F(28, 1956) = 1.98, P = .002, partial η2 = .025, and diagnosis group, F(24, 1930) = 3.16, P < .001, partial η2 = .033. There were no overall differences in social health between gender, ethnicity, and education groups.

Follow up ANOVAs revealed an overall effect of education on mean scores for the SFQ, F(8, 556) = 2.24, P = .023, and CA-QOL Social Health Scale, F(8, 556) = 1.98; P = .048, and an overall effect of living situation on mean scores for the SFQ, F(7, 545) = 4.01; P < .001, CA-QOL Social Health Scale, F(7, 545) = 2.53; P = .014, MHSIP Social Health Scale, F(7, 545) = 2.99; P = .004, and Combined Social Health Scale, F(7, 545) = 3.21; P = .002. There was also an overall effect of diagnosis on mean scores for the SFQ, F(6, 556) = 8.35; P < .001, CA-QOL Social Health Scale, F(6, 556) = 4.82; P < .001, MHSIP Social Health Scale, F(6, 556) = 4.41; P < .001, and Combined Social Health Scale, F(6, 556) = 5.74; P < .001. Results from the Post-Hoc tests can be found in Table 4.

The mean SFQ score for this study population, where higher scores indicate more dysfunction, was 9.66 (SD = 4.68). As expected, mean SFQ score was significantly higher than previously reported norms in community samples, Mean = 4.60; SD = 3.6; T(695, 1) = 28.535; P < .001. Mean SFQ score for this study was similar to previously reported norms in hospital patients at the point of discharge and with recurrent psychotic disorders, Mean = 9.80; SD = 5.0; T(695, 1) = −0.784; P = .433, and was significantly lower than previously reported norms in psychiatric emergency patients, Mean = 11.40; SD = 4.4; T(695, 1) = −9.805; P < .001 (Tyrer et al. 2005).

Discussion

Our study describes levels of social health and functioning in a large adult public mental health program and proposes a general framework for the creation of social health scales using items from questionnaires currently administered in public mental health systems (NASMHPD Research Inc. 2002). Since these individual instruments have been widely used for several years, the identification of a social health scale within these instruments allows researchers to conduct retrospective studies evaluating changes in social health in their patient population over time. Furthermore, the availability of a scale within instruments that are already used provides the added ability to individualize client treatment planning according to reported social health outcomes without increasing the time devoted to survey administration or increasing response burden. Measures of social health are also useful at the patient level by allowing the clinician to observe pre- and post-treatment changes when assessing recovery. Thus, this identification of a social health scale within these pre-existing measures allows programs to incorporate social health assessment into current evaluation modalities to track program level changes in outcomes and enhance treatment planning and delivery nationwide.

The social health scales identified in this study demonstrated good psychometric properties and moderate to strong relationships with the validated SFQ. Specifically, the scales demonstrated good internal reliability and face validity. Concurrent validity for the identified CA-QOL Social Health Scale and MHSIP Social Health Scale was supported by acceptable correlations with the SFQ in a large sample of mental health clients. Based on the amount of overlap in variance accounted for by the SFQ, CA-QOL Social Health Scale, and MHSIP Social Health Scale, it is likely that these scales are assessing similar (overlapping) constructs.

Currently, no normative data exists for the identified CA-QOL Social Health Scale and MHSIP Social Health Scale. The few reports that provide normative scores on the MHSIP used a shorter version than the version used in this study. Although this study identified a social health scale from the 36-item version of the MHSIP, it is likely that a similar scale could be identified from a shorter version. The mean SFQ score in this population was nearly the same as that of the SFQ’s norming population of hospital patients at the point of discharge with recurrent psychotic disorders. This indicates that the level of social health in this general psychiatric population is quite low.

Many of the between group differences in social health in our unique sample are not unexpected. The identified items demonstrated similar patterns of results as previous studies with respect to social health and quality of life when describing different demographic groups. Specifically, our sample demonstrated higher social health among those who reside in homes with supervised or assisted living, and worse social health among those who are homeless (Pinikahana et al. 2002), thus adding validity to the scoring of the identified social health scales.

It is important to note the differences in our study’s results from those found in other studies investigating differences in social health between psychiatric disorders. In general psychiatric populations, patients with schizophrenia have reported the poorest social health, while those with depressive, affective, and anxiety disorders have reported the best social health (Cannon et al. 1997; Sanderson and Andrews 2002). On average, our study showed that those with major depressive disorders reported the poorest social health, while those with schizophrenia reported the best social health. The most likely explanation for this discrepancy is the different sampling frame of this study. For individuals with mental illness, previous studies have shown social health varies as a function of illness severity (Kennedy et al. 2007, Koivumma-Honkanen et al. 1996, Lehman 1983), allowing for highly functioning individuals with schizophrenia to express greater social health than poorly functioning individuals with depression. Consequently, the possible range of social health in patients with major depression is thought to vary in severity, such that many patients have high enough social health to maintain a job and have private health insurance. These people would not qualify for Medi-Cal (government assisted health care) or meet the other income eligibility requirements for treatment at the county mental health services and thus would have been less likely than depressed patients with lower social health (i.e. without a job or health insurance) to be included in this study. Also, a patient treated for a bout of depression may only receive treatment for a limited amount of time after remission. Although the range of social health among patients with schizophrenia or psychotic disorders can also vary, these patients often require continual management and may be more likely to receive ongoing treatment at the county mental health services. While only the most severe depression cases would be represented in our sample as explained above, the full range of schizophrenia severity may be present.

Although the construct of social health is not synonymous with that of quality of life, overlapping concepts exist between the two constructs such that a deficit in one is likely to weaken the other. Some of these overlapping domains measured across quality of life and social health instruments include ability to go out, community, employability, family and children, friends, interpersonal relations, partner and sexuality, and relationships/social supports (Korr and Ford 2003). Adding validation for the overlap between these two constructs, the differences we found between diagnoses groups on all scales replicate those found in the research on quality of life in the mentally ill. Atkinson et al. (1997) also found that patients with schizophrenia reported higher quality of life than patients with affective disorders. Pukrop et al. (2003) not only found higher self-rated quality of life greater in patients with schizophrenia than patients with depression, but found a similar pattern of differences within the social health domain of their quality of life measure such that patients with schizophrenia reported greater social health than patients with depression.

Limitations of this study include the lack of validation against objective measures or clinician-rated outcomes and the low response rate among this population. There may be important differences between responders and non-responders in this study which may have affected our evaluation of the social health scales. Also, because of the self-report nature of questionnaire research, psychiatric symptoms such as mood or psychosis could skew perceptions of outcomes (Atkinson et al. 1997; Winter et al. 2007). Further studies are needed to examine the ability of the CA-QOL Social Health Scale and MHSIP Social Health Scale to assess and predict patient recovery in the mental health system.

In summary, social health measures have value in mental health service outcome assessments. However, adding even a brief social health measure to an already lengthy assessment survey is problematic, especially due to time constraints. The present analyses supported items from two widely used mental health outcome assessment measures, the CA-QOL and MHSIP, as valid self-report measures of social health. Thus, when lengthy surveys are not feasible, CA-QOL and/or MHSIP items can be used to assess social health for research and assessment purposes.