Administration and Policy in Mental Health and Mental Health Services Research

, 33:433

The Effects of Organizational Culture and Climate on the Access to Mental Health Care in Child Welfare and Juvenile Justice Systems

Authors

  • Charles Glisson
    • The University of Tennessee
    • Children’s Mental Health Services Research CenterThe University of Tennessee
  • Philip Green
    • The University of Tennessee
Article

DOI: 10.1007/s10488-005-0016-0

Cite this article as:
Glisson, C. & Green, P. Adm Policy Ment Health (2006) 33: 433. doi:10.1007/s10488-005-0016-0

Abstract

This study examined the effects of organizational culture and climate on the access to mental health care for 588 children referred to child welfare and juvenile justice systems in 21 Tennessee urban and rural counties. Cross-level, hierarchical linear models (HLM) analyses indicated that children served by child welfare and juvenile justice case management units with constructive organizational cultures were more likely to receive the needed mental health care. For example, controlling for the child’s need for mental health care and other child and family characteristics, the odds of a child receiving mental health care in a case management unit with the most constructive culture were 11 times the odds of receiving mental health care in a unit with the least constructive culture. Constructive cultures were characterized by organizational norms and expectations that case managers would be mutually supportive, develop their individual abilities, maintain positive interpersonal relationships, and be motivated to succeed. These findings suggest that efforts to improve access to mental health care for children referred to child welfare and juvenile systems should include the development of constructive organizational cultures in case management units responsible for the children’s care.

Keywords:

organizational cultureorganizational climatechild welfarejuvenile justicechildren’s mental health care

Introduction

Studies of child welfare and juvenile justice systems confirm that most of the children referred to these systems who need specialty mental health care do not receive the needed care (Burns et al., 2004; Lyons, Baerger, Quigley, Erlich, & Griffin, 2001; Webb & Harden, 2003). This is an important service deficit because there is evidence that the majority of the three million children served each year by child welfare and juvenile justice systems nationwide are at risk of serious emotional or behavioral problems that can follow them into adulthood (Burns et al., 2004; Garland et al., 2001; Hazen, Hough, Landsverk, & Wood, 2004; MacKinnon-Lewis, Kaufman, & Frabutt, 2002; U.S. Department of Health and Human Services, 2003). The high risk of chronic behavioral and mental health problems faced by these children means that adequate child welfare and juvenile justice services must include timely access to appropriate mental health care in the early stages of the children’s contacts with these systems (Garbarino, 1999; Lindsey, 1994; MacKinnon-Lewis et al., 2002; U.S. Department of Health and Human Services, 2004). This study examines the access to mental health care among children served by one state’s child welfare and juvenile justice system as a function of several child, family, and service system characteristics.

Although previous studies documented the need for improved access to mental health care for children referred to child welfare and juvenile justice systems, few studies identified the service system factors that contribute to the children’s access to mental health care. That is, there is limited information about the characteristics of child welfare and juvenile justice systems that explain why some systems do a better job than others of providing access to mental health care for the children who need it most. These characteristics must be identified if access to mental health care is to be improved in these systems. This study is based on the assumption that the children’s access to specialty mental health care within these systems results in part from the work norms, values, expectations, and perceptions that characterize the case management units that serve the children. These characteristics are captured by the concepts of organizational culture and climate (Glisson, 2002; Glisson & James, 2002).

Previous studies found that organizational culture and climate affect service quality and outcomes independently of the education, training, and experience of the service provider and of the characteristics of the children and families receiving the services (Glisson & Hemmelgarn, 1998; Glisson & James, 2002; Hemmelgarn, Glisson, & Dukes, 2001). The present study specifically examines the roles that the organizational culture and climate of case management units play in the access to mental health care for children with behavioral and mental health problems who are referred to child welfare and juvenile justice systems.

Children’s Service Sectors and Mental Health Care

Although studies of children at risk of chronic mental health and behavioral problems tend to focus on a specific service sector, there is evidence that over time the children who are most at risk are served by more that one service sector and that the child welfare and juvenile justice systems serve a particularly large number of children who need services from the specialty mental health care sector (Burns et al., 2004; Garland et al., 2001; Hazen et al., 2004; Scott, Snowden, & Libby, 2002). Although child welfare services are traditionally associated with parental maltreatment and juvenile justice services are associated with children’s illegal behavior, there is a great deal of overlap in the populations served by these systems in both the risk factors and mental health needs of the children and their families (Garland et al., 2001). In some states, child welfare and juvenile justice systems are separate systems and in others, such as the state in this study, child welfare and juvenile justice services are provided by the same statewide system. In this state, Tennessee, child welfare and juvenile justice services are provided by the Department of Children’s Services (DCS).

The overlap between the populations served by child welfare and juvenile justice systems has created confusion and disagreement in the research literature for at least a half-century about the similarities and differences between the children served by these systems (Maas & Engler, 1959). The discussions with judges and caseworkers reported by Maas and Engler (1959) in the middle of the last century revealed differences in views and practices nationwide that are found today in the reasons given for placing children in the care of child welfare or juvenile justice systems. For example, many studies of children in the custody of child welfare systems include children who are placed in custody because of the children’s illegal behavior as well as those who are placed in custody because of maltreatment (see review in Glisson, Bailey, & Post, 2000). These “mixed” samples occur because many child welfare systems assume custody of children labeled “behavior problems,” “unruly,” or “status offenders,” along with those children labeled “neglected” or “abused.” As a result, even in those states with separate child welfare and juvenile justice systems, many child welfare systems care for children with behavior problems that are similar to those of the children in the care of juvenile justice systems (Glisson, 1996). Moreover, in many states, the same outpatient and residential treatment facilities provide services to children through contracts with both child welfare and juvenile justice systems.

Access to mental health care by children in the child welfare and juvenile justice sectors of care is undoubtedly a function of several factors at the individual, family, and organization levels. The organizational factors that affect the access to mental health care can be categorized as technological, strategic, and social. Technological factors include the knowledge, training, and assessment tools used by case managers to identify mental health needs. Strategic factors include the availability of mental health service providers and mental health care funding, and the policies that guide their use. Social factors include the actual norms and expectations in the case management unit that drive work behavior, and the perceptions and attitudes of the case managers who work there. This study of case management units within one state’s child welfare and juvenile justice system focuses on two social factors, organizational culture and climate. The norms, expected behaviors, and perceptions that comprise the culture and climate of case management units are hypothesized to affect the extent to which case managers make use of available resources to ensure that the children in their care receive the needed mental health care.

The technological and strategic factors vary little across the case management units in the study. This is because the educational requirements and training for Department of Children’s Services case managers, as well as the assessment tools they use, are uniform statewide. Moreover, funding for mental health services is uniform as a result of the state’s Medicaid waiver, behavioral managed care program, named TennCare, which serves 75% of the children referred to the state’s child welfare and justice system. However, because the availability of mental health service providers could vary between rural and urban locations, the case management units in the study are categorized as rural or urban to control for any difference in the availability of service providers.

The Cultures and Climates of Child Welfare and Juvenile Justice Systems

Child welfare and juvenile justice systems are inundated with seriously emotionally disturbed children and families with chronic mental health and behavioral problems (Burns et al., 2004; Glisson, 1996; Glisson, Hemmelgarn, & Post, 2002; Lyons et al., 2001; Martin, Peters, & Glisson, 1998; Webb & Harden, 2003). The case managers in these systems are responsible for developing case management plans, referring the children and families for a variety of needed services, and monitoring the children’s well-being.

These responsibilities, the nature of the problems, and the associated demands of judges, attorneys, child advocates, and others make child welfare and juvenile justice work stressful, indeterminate, and complex. In addition, there are huge demands of case managers’ time and energy with minimal remuneration. These characteristics of the work environment explain why dimensions of organizational climates such as role conflict, emotional exhaustion, and depersonalization were found to affect service quality and outcomes in a child welfare and juvenile justice system (Glisson & Hemmelgarn, 1998). These characteristics also explain why culture and climate played significant roles in predicting job satisfaction, commitment, and turnover rates among child welfare and juvenile justice case management teams (Glisson & James, 2002).

There is preliminary evidence that the quality of care provided by child welfare and juvenile justice systems is tied to the cultures and climates of the bureaucracies that provide the services (Glisson, 2000; Glisson & Hemmelgarn, 1998; Glisson & James, 2002; Schorr, 1997). This is important because many of these bureaucracies develop defensive cultures that erect barriers to service, create apathy among service providers, and resist new service technologies that could improve the outcomes of services (Glisson, 1996). These defensive cultural barriers include requirements for extensive documentation of process, micro-management of all decisions, and conformity to a rigid array of strategies meant to serve as protection against intense public criticism, administrative sanctions, and litigation. In contrast, the types of constructive cultures associated with more successful organizations that emphasize performance, motivation, support, interpersonal relationships, and effectiveness are less common in defensive bureaucracies where case managers are expected to follow well-worn, organizational paths of behavior that are unrelated to service quality or successful outcomes (Martin et al., 1998).

Defensive cultures and negative climates high in emotional exhaustion, role overload, and depersonalization promote reactivity rather than responsiveness to the behavioral and mental health problems of the children and create self-defeating work expectations. For example, case managers in defensive work environments avoid providing service to those children and families who are most in need of mental health care (Glisson & James, 1992; Nugent & Glisson, 1999). Although avoidance and other defensive strategies appear to be self-defeating and irrational approaches to service, they serve an important purpose in many organizations. As observed in a study of children’s emergency health services, expected behaviors and normative beliefs that define defensive cultures in high stress work environments have what Schein (1992) labeled “survival value” regardless of their contribution to care (Hemmelgarn et al., 2001). In these types of work settings, defensive cultural norms support expected behaviors that workers come to depend on in their efforts to survive in a stressful work environment that makes intense demands of their time, energy, and emotional resources. Even when the survival strategies detract from effective service, service providers maintain these strategies in an effort to survive psychologically in the stressful work environment. A better understanding of the roles that culture and climate play in child welfare and juvenile justice organizations that make these types of demands is necessary to improving the quality and outcomes of the services the organizations provide.

Definitions of Culture and Climate

Climate. The definition of climate used here includes a distinction between psychological and organizational climate. Psychological climate is defined as the individual employee’s perception of the psychological impact of the work environment on his or her own well-being (James & James, 1989). When employees in a particular work unit agree on their perceptions of the impact of their work environment, their shared perceptions describe the organizational climate in their work unit (Glisson & James, 2002; James, 1982; Jones & James, 1979; Joyce & Slocum, 1984).

Although the psychological impact of the work environment is measured as multiple dimensions (e.g., emotional exhaustion, depersonalization, role conflict, role overload), a single, second-order, general psychological climate factor (PCg) is believed to underlie climate. This general PCg factor represents the individual’s perception of the overall psychological impact of the work environment on the individual in positive or negative terms (Brown & Leigh, 1996; Glisson & Hemmelgarn, 1998; Glisson & James, 2002; James & James, 1989; James, James, & Ashe, 1990).

Culture. Culture is defined as the normative beliefs and shared behavioral expectations in an organizational unit (Cooke & Szumal, 1993). These beliefs and expectations prescribe the way work is approached in the organizational unit and are the basis for socializing coworkers in the way things are done in that unit. Organizational culture is sometimes described in layers, with shared behavioral expectations and norms representing an outer, conscious layer, and values and assumptions representing an inner layer that is less conscious (Rousseau, 1990). As stated by Hofstede (1998), behavior is the visible part of culture while values represent the invisible part.

Although this inner layer or invisible part is important to understanding culture, there is evidence that culture is expressed and transmitted among employees through “visible” shared behavioral expectations and norms rather than through “invisible” values or assumptions (Hofstede, 1998; Hofstede, Neuijen, Ohayv, & Sanders, 1990; Ashkanasy, Broadfoot, & Falkus, 2000). That is, individuals in an organization are compliant with behavioral norms and expectations without necessarily being conscious of the values and assumptions that lie at the core of the norms and expectations they guide their work.

The Cross-level Effects of Culture and Climate

Theoretical models linking organizational culture and climate at the work unit level to individual level outcomes are rare and generally untested (Glisson & James, 2002; Glisson & Schoenwald, in press). In the model described by Kopelman, Brief and Guzzo (1990), culture is described as the foundation on which managerial policies and practices are built, which has a psychological impact on workers (climate) which in turn affects their work attitudes and behavior. Only parts of this model have been tested, and there are questions about which of the work environment characteristics have the greatest direct effects on individual level attitudes and behavior. For example, Glisson and James (2002) found that constructive cultures in case management teams had a greater impact than climate on lowering staff turnover, increasing job satisfaction, and improving service quality.

The model depicted here examines a child’s access to specialty mental health care as a function of the culture and climate of the child welfare and juvenile justice case management unit that is responsible for the child. Although a series of sequential relationships link culture, climate, and work behavior, almost no studies to date have examined empirically the simultaneous, cross-level effects of culture and climate on these types of services.

Distinct case management unit cultures and climates are believed to emerge within larger statewide systems when case management units function independently under separate supervisors in different geographical locations (Glisson & James, 2002; Trice & Beyer, 1993). Interactions within these case management units form the basis for the socialization of new members and for the individuals’ interpretations of the meaning and impact of their work environment (Rentsch, 1990). We hypothesize that case managers in units with more constructive cultures and more positive climates will be more successful than other units in the system in gaining access to specialty mental health care for the children in their care.

Methodology

This study relies on data describing two levels of sampling units. The first level represents children who were referred for child welfare and juvenile justice services in each of 21 counties. The second level represents 15 case management units that provided child welfare and juvenile justice services for the 21 counties from which the children were selected. Each of the child welfare and juvenile justice case management units serve from one to four counties. Each of the two urban counties are served by one unit composed of four and six case management teams, respectively. In the rural areas, several counties are served by a single unit composed of one to three teams each, depending on the size of the population in the counties.

The children selected for the study were referred for child welfare and juvenile justice services through the juvenile and family court in their county of residence. The children were recruited for the study when they and their caregivers (usually a mother) appeared in juvenile and family court in each of 21 Tennessee counties. Research assistants trained by the Children’s Mental Health Services Research Center (CMHSRC) were placed on-site in each of the 21 juvenile and family courts (one court per county) to recruit subjects. The 21 counties in the study include two of the state’s major urban centers (Hamilton and Knox) and 19 additional rural counties. The 19 additional rural counties are Anderson, Blount, Bradley, Claiborne, Cocke, Franklin, Grainger, Grundy, Jefferson, Loudon, Marion, McMinn, Meigs, Monroe, Morgan, Rhea, Scott, Sequatchie, and Sevier.

The research assistants obtained written informed consent from the children’s caregivers (and from the children over 10 years of age) for participation in the study following protocols approved by The University of Tennessee and the National Institute of Mental Health. After obtaining informed consent, the research assistants conducted a baseline interview with each child’s caregiver within 72 hours of the child’s referral to the child welfare and juvenile justice system. A follow-up interview with the caregiver was conducted 6-months after the baseline interview to identify the mental health services received by the child in the 6-month period following the child’s referral to the child welfare and juvenile justice system.

Subjects

Research assistants selected 733 school-aged children (4–18 years) from the children who were referred to juvenile and family court in the 21 counties over a 1-year period. Children younger than 4 years were excluded because of the difficulty in assessing the mental health needs of preschool children in field studies of this type. In addition, children who appeared in court for reasons that did not require child welfare and juvenile justice services (e.g., visitation rights, child support) were also excluded. Finally, children who were placed in state custody at the time of their referral were also excluded.

Longitudinal studies of children referred to child welfare and juvenile justice systems are challenging because it is difficult to locate families for follow-up interviews or because some families decide they want to withdraw from the study prior to the follow-up interviews. Of the children assessed at baseline, 6-month follow-up interviews were conducted with 588 caregivers (80% of the baseline sample). Comparisons of baseline data between the children who were retained for the follow-up interview and for those who were not, confirmed there were no differences between the two groups with two exceptions. The subjects who provided follow-up interviews were more likely to be minorities (17% of dropouts were minorities and 26% of retained sample were minorities). In addition, the subjects in the retained sample were somewhat more likely to have a history of substance abuse (42%) than the dropouts (39%).

As shown in Table 1, a majority of the children selected for the sample were male (62%), about one-quarter were minorities (26%), and they had a mean age of 14.4 years. Children referred to juvenile and family courts statewide for all reasons in 2003 were 60% male, 75% Caucasian, and had an average age of 15 years (Tennessee Commission on Children and Youth, 2004; Tennessee Council of Juvenile and Family Court Judges, 2003). Although the sample was not randomly selected, it shares these characteristics with the children referred to juvenile and family courts statewide.
Table 1.

Descriptive Statistics (Level 1 Subjects=588; Level 2 Units=15)

Variable

%

Minimum

Maximum

Mean

SD

Level 1—child

 Child age

 

4.00

18.00

14.40

2.44

 Child gender (female=1)

38.5

    

 Child ethnicity (minority=1)

26.1

    

 Family functioning (FAD)

 

14.00

44.00

25.75

4.94

 Caregiver mental health (BSI)

 

.00

211.00

48.31

42.46

 Child’s substance abuse

42.4

    

 Parental substance abuse

59.6

    

 Monthly income

 

0

8000.00

1603.93

1237.34

 Child mental health (CIS)

 

0.00

52.00

19.47

11.14

 Child mental health (SAC)

 

0.00

91.00

34.26

17.87

 Child in urban county

43.9

    

 Received specialty MH care

26.0

    

 Received any MH service

33.0

    

Level 2—case management unit

 Unit climate

 

68.87

120.00

85.20

11.73

 Unit culture

 

78.60

122.98

105.94

11.32

 Unit in urban county

13.0

    

About 60% of the children in the sample were referred to juvenile court for delinquency, 30% were referred for unruly behavior, and 10% were referred for dependency and neglect. Statewide, of the children referred to juvenile and family court for reasons that place them at risk of state custody, 59% were referred for delinquency, 33% for unruly behavior, and 8% for dependency and neglect (Tennessee Council of Juvenile and Family Court Judges, 2003). As with gender and age, the reasons for referral to juvenile court in the sample matched closely the reasons for referral statewide.

The families in the sample reflect a range of incomes, but the distribution of incomes is positively skewed with lower income families being much more heavily represented. The families’ median income was $1300 per month and their mean income was $1600 per month. Approximately 60% of the families’ incomes were below the poverty line as defined by the U.S. Department of Health and Human Services at the time of the study (e.g., $1508 per month for a family of four).

Measures of Child and Family Functioning

The interviews conducted by research assistants included questions about the demographic characteristics of the child and family, questions about family structure and living arrangements, and a number of instruments designed to obtain information about services received by the child and family, the mental health of the child and caregiver, and family functioning.

Shortform Assessment for Children (SAC). The SAC is a validated, standardized measure developed by the Children’s Mental Health Services Research Center with the support of the National Institute of Mental Health to assess the overall mental health and psychosocial functioning of children referred to juvenile justice and child welfare systems (Glisson et al., 2002; Hemmelgarn, Glisson, & Sharp, 2003; Tyson & Glisson, 2005). The SAC has separate norms for girls and boys, for preadolescents and adolescents, and for information collected from parents (or guardians) and teachers. These norms are used to identify children with clinical levels of both internalizing and externalizing behavioral problems that require mental health services and to monitor changes in children’s psychosocial functioning over time (McMahon, 1994; Ollendick & King, 1994).

The validity of the SAC was confirmed in three previous studies, each using a distinct and separate sample of children (Glisson et al., 2002; Hemmelgarn et al., 2003; Tyson & Glisson, 2005). Each of these studies used large samples to establish the construct validity, criterion validity, and cross-ethnic validity of the SAC when used to assess children referred to child welfare and juvenile justice systems. In the present study, the child’s primary caregiver, who was usually the child’s mother, completed the SAC at the baseline interview (time of referral to the child welfare and juvenile justice system). The alpha reliability coefficients for the SAC total scale, externalizing behavior scale, and internalizing behavior scale for the present sample are .94, .94, and .90, respectively.

Columbia Impairment Scale (CIS). The CIS is a validated, standardized measure of children’s overall mental health used to identify children with clinical levels of mental health and behavioral problems that require mental health services (Bird, 1999). Similar to the SAC, the CIS provides an overall measure of a child’s mental health and an indication of whether a child’s behavioral and mental health problems require clinical mental health services. Unlike the SAC, the CIS does not have individual internalizing and externalizing problem scales. The child’s primary caregiver completed the CIS at the baseline interview (time of referral) and the alpha reliability coefficient of the CIS for this sample is .85.

Brief Symptom Inventory (BSI). The BSI is a validated, standardized measure of mental health used to identify adults with mental health problems that may require clinical intervention (Derogatis, 1993). The BSI provides an overall measure of mental health as well as measures of nine specific dimensions of mental health such as depression and psychoticism. The child’s primary caregiver completed the BSI at the baseline interview (time of referral to court). For this sample, the alpha reliability coefficient for the total BSI scale is .97 and the reliabilities for the nine individual dimensions range between .74 and .89.

Family Assessment Device (FAD). The FAD is a validated, standardized measure of family functioning used to assess problems in family functioning and to identify those families who are likely to benefit from clinical intervention (Miller, Epstein, Bishop, & Keitner, 1985). The measure provides an overall measure of a family’s well being and the family’s success in cooperating to solve problems and address issues that confront the family’s psychosocial functioning. The child’s primary caretaker completed the FAD at the baseline interview (time of referral). The alpha reliability coefficient of the FAD for this sample is .83.

Services Assessment for Children and Adolescents (SACA). The SACA is a validated measure developed with the support of the National Institute of Mental Health used to describe a variety of social, behavioral, and mental health services received by a child or adolescent (Hoagwood et al., 2000; Stiffman et al., 2000). The measure has been used successfully to obtain information from primary caregivers about the type of services received by the children or adolescents in their care. The measure has provided accurate and valid descriptions of mental health services using information from caregivers with backgrounds and service needs that are similar to the sample selected for this study (Hoagwood et al., 2000; Stiffman et al., 2000). The child’s primary caregiver completed the SACA during the baseline interview (time of referral to court) and at the second interview (6 months after the baseline).

Measures of Climate and Culture

The measures of climate and culture relied on responses from 216 case managers in the 15 case management units serving the 21 counties from which the children were served. About half of the case managers served the two urban counties and about half served the 19 rural counties. Data were collected from case managers with the culture and climate instruments described below at scheduled case management team meetings. The instruments presented Likert scale response categories on scannable questionnaires that required approximately 45 minutes to complete.

Climate. Climate was measured with the Organizational Climate Survey (OCL), which was designed for mental health and social service organizations and linked in earlier research to staff turnover, work attitudes, service quality and outcomes (Glisson & Hemmelgarn, 1998; Glisson & James, 2002). Positive climates are perceived by case managers as low in depersonalization (e.g., “I worry that this job is hardening me”), emotional exhaustion (e.g., “I feel used up”), role conflict (e.g., “I do things that are against my better judgment”) and role overload (e.g., “The amount of work I have to do keeps me from doing a good job”). The alpha reliabilities for these dimensions were .69 (depersonalization), .92 (emotional exhaustion), .87 (role conflict), and .88 (role overload) with this sample. The reliability of the overall climate measure was .95.

Culture. Organizational culture was measured with the Organizational Culture Survey (OCU) that was developed to assess culture in mental health and social service organizations (Glisson & James, 2002). The scales measuring culture were selected from the constructive culture dimension. Constructive cultures promote positive, proactive behavior and encourage interactions that meet higher satisfaction needs (Cooke & Szumal, 2000). Constructive culture was measured by scales that assess motivation norms (e.g., “take on challenging cases”), individualistic norms (e.g., “develop full potential”), supportive norms (e.g., “encourage others”), and interpersonal norms (e.g., “treat people as important”). The alpha reliabilities for these scales were .86 (motivation), .86 (individualistic), .89 (supportive), and .91 (interpersonal) for this sample. The reliability of the overall measure of constructive culture was .96.

Within Group Analyses and Composition Models for Culture and Climate

A within-group consistency analysis using rwg tests whether members in each case management unit agreed in their responses to the culture and climate scales. A minimum rwg value of .70 is considered a necessary prerequisite for composing the individual-level responses to higher-level (i.e., unit) constructs (Glisson & James, 2002). In the present sample, rwg values for the measure of climate ranged from .86 to .98, with an average of .94. Values of rwg for the measure of culture ranged between .82 and .98, with an average of .95. The values indicate that there was adequate agreement in the responses of case managers within each region to compose their responses into aggregate measures of culture and climate for the groups of case managers serving the identified counties.

Composition models specify the relationships between constructs that reference the same content but describe qualitatively different phenomena at the individual and work-unit levels (Chan, 1998; Rousseau, 1985). Composition models play an important role in cross-level inferences that link organizational climate and culture to individual-level outcome variables, but organizational culture and organizational climate require different models of elemental composition. This distinction was first noted in a multilevel study of culture and climate in child welfare and juvenile justice case management teams (Glisson & James, 2002).

Climate. The appropriate composition model for climate is the direct consensus model (Chan, 1998). This model uses within-group consensus at the lower-level (e.g., among individuals in a work unit) as a precondition for measuring the higher-level construct (e.g., organizational climate of the work unit) as an aggregate of the individual-level responses. In this example, a shared psychological climate at the individual level—represented by a value of .70 or above on the rwg index of within-group consensus—is a prerequisite for calculating organizational climate as a case management unit mean of the individual responses (James, Demaree, & Wolf, 1984). When there is consensus and the psychological climate is shared by members of a work unit, the aggregate composes a construct at the work unit level (e.g., organizational climate) although the perceptions remain a property of the individuals in the work unit.

Culture. The appropriate composition model for organizational culture is the referent-shift consensus model (Glisson & James, 2002). Culture is a property of the work unit, not of the individual, and this is reflected in the shift in the referent from the individual to the collective. As applied to culture, the referent-shift consensus model uses individual responses to questions about expectations for all members of a work unit. Referent-shift consensus composition is similar to direct consensus composition, but there is a shift in the referent prior to consensus assessment (Chan, 1998). In assessing organizational culture using the referent-shift consensus model, the respondent is asked to describe the expectations and norms for people in the respondent’s organizational work unit. The focus is on what the individual believes are the expectations and norms for the people in the respondent’s work unit rather than on what the individual respondent thinks is expected of him or her personally. Within-group consensus (e.g., rwg values above .70) is required to justify the aggregation of the individuals’ descriptions of the expectations and norms for members of the work unit as a representation of the unit-level construct, i.e., culture.

Assessing Multilevel Relationships

Links between individual-level variables such as a child’s receipt of mental health care and work unit-level variables such as organizational culture and climate require statistical models that provide estimates of relationships between variables measured at different levels (Glisson, Dukes, & Green, in press; Glisson & James, 2002; James & Williams, 2000; Rousseau, 1985). Although cross-level inferences can be made using a variety of approaches, hierarchical linear models analysis (HLM) was designed specifically for cross-level inferences that link the characteristics of individuals to the characteristics of the groups in which they are nested (Raudenbush & Bryk, 2002). All of the children served by a given case management unit are “nested” within that unit.

When HLM is applied to organizational research, questions about cross-level relationships in multilevel studies can be formulated as two-level random effects models. Two HLM analyses were conducted using a random slopes model to estimate cross-level relationships between work unit variables (i.e., culture, climate, urban/rural location) and individual-level access to mental health care (Hedeker, Gibbons, & Flay, 1994; Raudenbush & Bryk, 2002). Specifically, the first HLM analysis estimates the effects of case management unit organizational climate, culture, and location (urban versus rural) on the access to specialty mental health services provided by mental health professionals to the children served by those work units, controlling for individual-level covariates (i.e., child’s age, gender, minority status, mental health, family income, caregiver and family functioning). The second HLM analysis estimates the effects of case management unit organizational climate, culture, and location (urban versus rural) on the access to any mental health services provided by mental health professionals, general health care providers, or schools to the children served by those case management units, controlling for the same individual-level covariates.

Results

HLM analyses using a two-level, random slopes model were conducted to estimate the cross-level effects of the case management unit’s organizational culture and climate on the children’s access to mental health care (Hedeker, Gibbons, & Flay, 1994; Raudenbush & Bryk, 2002). Specifically, HLM analyses examined the slope of the relationship between the children’s need for mental health care (measured by the SAC) and their receipt of mental health care as a function of the organizational culture and climate of the case management units serving the children. The model controls for the random effects associated with the case management units and the effects of first-level covariates describing the child and family (child’s age, gender, minority status, substance abuse, family functioning, caregiver’s mental health, etc.).

HLM analyses were conducted using maximum marginal likelihood estimation for mixed effects regression models using HLM 6 software (Raudenbush, Bryk, & Congdon, 2004). The HLM 6 analysis of the receipt of mental health services as a binary outcome (i.e., criterion equaled 1 if the child received mental health services and 0 if he or she did not) used a binomial sampling model with a Bernoulli distribution and logit link (Raudenbush & Bryk, 2002, pp. 294–295). Therefore, the HLM analyses provide estimates of the effects of case management unit culture and climate on the access to mental health care defined as the relationship between the children’s need for mental health care and their receipt of mental health services, after controlling for the covariates in the model and case management unit random effects (Hofmann, Griffin, & Gavin, 2000; Raudenbush & Bryk, 2002).

The slope of the relationship that represents access to mental health care is represented by the relative log odds coefficient associated with the SAC in predicting the child’s receipt of mental health care as reported on the SACA. That is, access to mental health care is defined as the increase in the odds of a child receiving mental health care for every unit increase in the estimate of the need for mental health services as represented by the child’s SAC score.

The HLM approach used to estimate the receipt of services addressed the binary nature of the mental health care criterion (each child either did or did not receive mental health care in the 6 months following their referral to the child welfare and juvenile justice system) by using a binomial sampling model based on the Bernoulli distribution and a logit link function with the log of the odds of the receipt of mental health care as the outcome variable (Raudenbush & Bryk, 2002, pp. 294–309). The log of the odds of receiving mental health care was predicted at the individual level (level one) with the child’s mental health, age, gender, minority status, substance abuse, family functioning, income, caregiver’s mental health, and parental substance abuse. At the case management unit level (level two), the analysis examined the effect of the unit’s culture, climate, and location (urban/rural) on the slope of the relationship between the child’s mental health care needs at baseline and the log of the odds of receiving mental health care within 6 months following their referral to the child welfare and juvenile justice system.

Among the children in the present study, 26% received outpatient specialty mental health care from a mental health service professional in the 6 months following their referral to juvenile and family court. The percentage of children receiving mental health care increased to 33% when non-specialty mental health services from health care providers (e.g., family physicians, pediatricians) and schools (e.g., counseling, special classrooms) were included.

Predicting Access to Specialty Mental Health Care

As shown in Table 2, the estimated intercept (bo) in the unconditional HLM analysis that included only the random effects in the model is −1.119. Because this HLM model includes only random effects, this is the log of the odds of a child receiving specialty mental health care in a typical or average case management unit in the sample. The exponent of this log (exp b) equals .327. This exponent is the odds ratio (i.e., the probability of receiving specialty mental health care divided by the probability of not receiving the care) in a typical unit. The odds ratio (η) translates into an estimated probability (ϕ) of receiving specialty mental health care of .25 in a typical unit, which is slightly lower than the overall proportion of children in the sample who received specialty mental health care (.26). This is a function of the difference between individual level and unit level rates of access to care. It is also a function of the positively skewed probability of receiving care that produced a mean (.26) that is slightly higher than the median turnover rate of the typical unit (.25). A benefit of estimating the log odds ratio (η) instead of the probability (ϕ) is that η is normally distributed while ϕ is positively skewed (Raudenbush & Bryk, 2002, pp. 297–298).
Table 2.

HLM Analysis of Access to Specialty Mental Health Services Using Bernoulli Distribution with Logit Link Function

Model

Variable

Coefficient

Odds ratio

SE

T-ratio

p

Random effects only

 

Constant

−1.119

.327

.147

−7.602

.000

Unit variance

.120

    

χ2

25.069

   

.034

df

14

    

Individual level covariates

 

Constant

−1.140

.320

.149

−7.646

.000

Child age

.015

1.015

.044

.326

.744

Childgender (female=1)

−.119

.888

.204

−.584

.559

Child ethnicity (minority=1)

−.142

.868

.257

−.551

.581

Family functioning (FAD)

.011

1.011

.022

.485

.627

Caregiver mental health (BSI)

.002

1.002

.003

.643

.520

Child’s substance abuse

.206

1.228

.219

.937

.350

Parental substance abuse

−.214

.807

.206

−1.037

.300

Family monthly income

.000

1.000

.000

.526

.598

Child’s mental health (SAC)

.011

1.011

.007

1.566

.118

Individual-level covariates and unit-level predictors

 

Constant

−1.162

.313

.154

−7.537

.000

Child age

.021

1.021

.045

.473

.636

Child gender (female=1)

−.123

.885

.206

−.595

.552

Child ethnicity (minority=1)

−.177

.838

.258

−.687

.492

Family functioning (FAD)

.014

1.014

.023

.601

.548

Caregiver mental health (BSI)

.001

1.001

.003

.476

.634

Child’s substance abuse

.213

1.237

.222

.958

.339

Parental substance abuse

−.225

.799

.208

−1.084

.279

Family monthly income

.000

1.000

.000

.485

.627

Child’s mental health (SAC)

−.108

.898

.087

−1.232

.219

    Unit location (urban=1)

−.010

.990

.014

−.690

.490

    Unit climate

−.000

.999

.001

−.516

.605

    Unit culture

.001

1.001

.001

2.008

.045

None of the coefficients shown in Table 2 representing the relationships between the level-one (child level) predictors and the receipt of specialty mental health care were significant, indicating that the child-level covariates did not explain a significant amount of unique variation in the odds of a child receiving specialty mental health care, after controlling for the other child-level covariates in the model and the random effects associated with the case management unit. It is important to note that the SAC is significantly associated with the receipt of services at the zero-order level, but the association is low (r=.12, p<.004). This is because only 39% of the children with a clinical SAC score indicating a need for mental health services received mental health care in the 6 months following the baseline assessment of needs.

At level-two (case management unit level), the culture of the case management organizational unit predicted the slope of the relationship between the child’s need for mental health care and the odds of the child receiving specialty mental health care. The odds of a child receiving the needed mental health care were significantly higher in case management units with more constructive cultures. The relative log-odds coefficient for constructive culture (.001) at level two indicates that the relative log-odds coefficient for the child’s SAC score at level one predicting the child’s receipt of specialty mental health services increases by .001 for every one point increase in the measure of constructive culture at level two. In other words, the association between the child’s SAC score and the receipt of mental health care is higher in case management units with more constructive cultures than in case management units with less constructive cultures.

The significance of this finding is more evident when considering that the average SAC score is 34 and the constructive culture scores for the case management units in the sample ranged from 78.6 to 122.98. To illustrate the effect of case management unit culture on the children’s access to mental health care, Table 4 compares the expected odds and probabilities of children with the same characteristics receiving mental health care between units with the least constructive cultures and units with the most constructive cultures. For example, the probability of children with the “average” characteristics in the sample as reported in Table 1 (noting that the average SAC score of 34 in the sample is above the clinical cut point for all gender and age groups) receiving specialty mental health services is .67 in the unit with the most constructive culture and .20 in the unit with the least constructive culture.

The probabilities of receiving mental health care were estimated for other children in the sample with various profiles based on age, gender, minority status, location and other covariates. Two of these additional profiles are included in Table 4. The lowest estimated probability of a child with a SAC score of 34 receiving mental health care is represented by a 6 year old girl who is a minority residing in an urban county. The highest estimated probability of a child with a SAC score of 34 receiving mental health care is represented by an 18 year old boy who is a non-minority residing in a rural county. Although as shown in Tables 2 and 3, age, gender, minority status and location did not individually explain a unique amount of variation in the access to care, the combination of characteristics defined the range in access and was used to illustrate the effect of constructive culture on access using specific children’s profiles. Of course, the estimated probabilities would be higher for higher SAC scores and lower for lower SAC scores.
Table 3.

HLM Analysis of Access to Any Mental Health Services Using Bernoulli Distribution with Logit Link Function

Model

Variable

Coefficient

Odds ratio

SE

T-ratio

p

Random effects only

 

Constant

−.821

.440

.146

−5.615

.000

Unit variance

.131

    

χ2

27.148

   

.018

df

14

    

Individual level covariates

 

Constant

−.840

.432

.149

−5.637

.000

Child age

.020

1.020

.042

.480

.631

Child gender (female=1)

−.241

.786

.192

−1.256

.210

Child ethnicity (minority=1)

−.167

.846

.240

−.695

.487

Family functioning (FAD)

.010

1.010

.021

.485

.627

Caregiver mental health (BSI)

.002

1.002

.003

.621

.534

Child’s substance abuse

.048

1.049

.206

.234

.816

Parental substance abuse

−.206

.814

.193

−1.068

.286

Family monthly income

.000

1.000

.000

.092

.927

Child’s mental health (SAC)

.012

1.012

.007

1.894

.058

Individual-level covariates and unit-level predictors

 

Constant

−.861

.423

.154

−5.584

.000

Child age

.028

1.028

.042

.658

.511

Child gender (female=1)

−.255

.775

.195

−1.311

.190

Child ethnicity (minority=1)

−.204

.816

.241

−.844

.399

Family functioning (FAD)

.014

1.014

.021

.644

.520

Caregiver mental health (BSI)

.001

1.001

.003

.454

.650

Child’s substance abuse

.047

1.048

.209

.223

.824

Parental substance abuse

−.222

.801

.194

−1.140

.255

Family monthly income

.000

1.000

.000

.090

.929

Child’s mental health (SAC)

−.106

.899

.081

−1.318

.188

     Unit location (urban=1)

−.012

.989

.014

−.841

.401

     Unit climate

−.001

.999

.001

−1.008

.314

     Unit culture

.002

1.002

.001

2.450

.015

Table 4.

Estimated Probabilities and Odds of Children with Different Profiles Receiving Mental Health Care

Examples of children with SAC scores of 34

Specialty mental health care

Any mental health services

Least constructive culture

Most constructive culture

Least constructive culture

Most constructive culture

Child with average values on all covariates

 ϕ=probability of receiving care

.20

.67

.20

.74

 Odds ratio=ϕ/(1−ϕ)

.24

2.01

.25

2.86

Minority 6 year old girl in urban county

 ϕ=probability of receiving care

.11

.51

.09

.53

 Odds ratio=ϕ/(1−ϕ)

.13

1.06

.10

1.12

Non-minority 18 year old boy in rural county

  ϕ=probability of receiving care

.24

.72

.28

.81

  Odds ratio=ϕ/(1−ϕ)

.32

2.62

.39

4.36

Using these profiles, Table 4 provides concrete examples of the significant cross-level effect of constructive culture that is reported in Tables 2 and 3. As shown in Table 4, the probability of receiving specialty mental health care in units with the most constructive culture is three to almost five times as high (e.g., .11 versus .51) as in units with the least constructive culture in the sample. The comparison of the odds associated with these probabilities is even more striking, with the odds of receiving specialty mental health services in the units with the most constructive cultures ranging as high as eight times the odds (e.g., .32 versus 2.62) in the units with the least constructive cultures.

Predicting Access to Any Mental Health Care

As shown in Table 3, the estimated intercept (bo) in the unconditional HLM analysis of access to any mental health care that included only the random effects in the model is −.821. Because this HLM model includes only random effects, this is the log of the odds of a child receiving any mental health care in a typical or average case management unit in the sample. The exponent of this log (exp b) equals .44. This exponent is the odds ratio (i.e., the probability of receiving specialty mental health services divided by the probability of not receiving the services) in a typical unit. The odds ratio (η) translates into an estimated probability (ϕ) of receiving specialty mental health services of .31 in a typical team, which is slightly lower than the overall proportion of children in the sample who received mental health services (.33). As explained for the analysis of specialty mental health care, this is a function of the difference between individual level and case management unit level rates of access and of the positively skewed probability of mental health care.

The findings for the predictors of the children’s access to any mental care, whether provided by general health care providers (e.g., family physician, pediatrician), schools (e.g., school counselors, special programs), or specialty mental health care providers were similar to those for the analysis of specialty mental health care. As with the analysis of access to specialty mental health care, none of the coefficients shown in Table 3 representing the slopes of the relationships between the level-one (child level) predictors and the receipt of specialty mental health services were significant, indicating that the child-level covariates did not explain a significant amount of unique variation in the odds of a child receiving mental health care, after controlling for the variation explained by other child-level covariates in the model and the random effects associated with the case management unit.

Also similar to the previous analysis, in level-two the culture of the case management unit explained the relationship between the child’s need for mental health care and the odds of the child receiving mental health care. That is, the association between the child’s SAC score and the odds of a child receiving mental health care (provided by specialists, health care providers, or schools) significantly increased in case management units with more constructive cultures.

The relative log-odds ratio coefficient for constructive culture (.002) translates into a relative odds ratio of 1.002 (exp [.002]=1.002). As in the previous analysis, this effect must be interpreted to fully understand the impact of the case management unit’s organizational culture on the access to any mental health care. The log-odds coefficient indicates that the relative log-odds ratio for the child’s SAC score at level one predicting the child’s receipt of specialty mental health services increases by .002 for every one point increase in the measure of constructive culture at level two.

Table 4 provides examples of the effect of constructive unit culture on the probabilities of children receiving any mental health service (e.g., from a specialty mental health provider, general health care provider, or school). For the hypothetical child with average values on each covariate, including a clinical SAC score of 34, the probability of receiving mental health care was .20 and .74 for children served by units with the least constructive and most constructive cultures, respectively. Using the example of the children’s profile representing the lowest probabilities of receiving mental health care, the probabilities for the most constructive cultures were almost six times the probabilities for the least constructive cultures (.09 and .59). For the children’s profile with the highest probabilities, the probabilities associated with the most constructive cultures were almost three times the probabilities associated with the least constructive cultures, or .28 and .81, respectively. The odds of a child receiving any mental health care when served by case management units with the most constructive cultures are consistently eleven times the odds of a child with the same level of need receiving mental health services in units with the least constructive cultures.

Using the range of values in case management unit constructive culture actually found in the sample, these estimates illustrate the effect of culture on the access to mental health care. By estimating the probabilities and odds of children with the same characteristics receiving mental health care when served by case management units with different cultures, the practical as well as the statistical significance of the effect of culture on the access to mental health care is evident.

Discussion

We know that a high proportion of the children served by child welfare and juvenile justice systems need mental health care. We also know that there is poor access to mental health care in many of these systems and that many of the children who need mental health care do not receive it. However, there is very little information about the factors in child welfare and juvenile justice systems that improve children’s access to the mental health care they need.

Organizational factors such as culture and climate are believed to be central to the effectiveness and success of many different types of organizations. Moreover, preliminary studies confirmed that culture and climate affect service quality, service outcomes, staff attitudes, and staff turnover in child welfare and juvenile justice systems. As in other types of organizations, there is evidence that culture and climate determine how work is approached, the priorities of work efforts, the tenor of work relationships, and the effort and commitment made by workers to achieve work goals. Although previous studies of child welfare and juvenile justice systems identified the effects of organizational culture and climate, only one study to date examined these factors simultaneously and no studies examined the specific contributions that culture and climate make to the access to mental health care. This study builds on these preliminary studies and addresses this knowledge gap by focusing on the contribution that the culture and climate of case management units in a child welfare and juvenile justice system make to the children’s access to mental health care.

Preliminary studies of child welfare and juvenile justice systems concluded that positive service outcomes depended heavily on the case manager’s consideration of each child’s unique needs, the caseworkers’ responses to unexpected problems, and their tenacity in navigating bureaucratic hurdles to achieve the needed services for each child. Although these case management behaviors are in part a function of training and resources, the studies also provided evidence that individuals within the same system, with similar training and resources, functioned differently as a result of the culture and climate of their immediate work environments. The norms and expectations that defined their work environment and the impact of those norms and expectations on the individual worker either encouraged or inhibited the case management activities that led to better service quality and outcomes.

The present findings add to these studies by showing that constructive organizational cultures encourage the type of case management efforts that result in children’s access to needed mental health care. That is, case managers in work units that expected case managers to be mutually supportive, develop their individual abilities, maintain positive interpersonal relationships, and be motivated to succeed were more likely to ensure that children received the mental health care they needed. As a result, children who needed mental health care were much more likely to receive that care when served by case management units with more constructive cultures than when served by case management units with less constructive cultures. These findings contribute to a better understanding of the role played by social context in the effectiveness of child welfare and juvenile justice systems and of the impact of organizational culture on service quality and outcomes in those systems.

The finding that organizational climate did not have a unique effect on the children’s access to mental health care over and above the effect of culture suggests that culture could be more important to mental health care access than climate. It also underscores the need to include both culture and climate in subsequent studies of organizational factors in child welfare, juvenile justice, and mental health systems. If these findings are sustained in further research, efforts to improve service quality in these systems through organizational development should focus on changing the norms and behavioral expectations (i.e., culture) of work environments. This is an important issue because preliminary studies show that culture is more deeply embedded and resistant to change than climate (Glisson et al., in press). This suggests that while efforts to improve climate might be successful, the maximum effect on service outcomes will require changes in the work environment norms and expectations that define organizational culture.

Limitations to these findings include concerns about their generalizability to other states. Because the study was conducted in 21 urban and rural counties in Tennessee, replications in other states or with nationwide samples are needed. The effects of organizational cultures and climates in child welfare and juvenile justice systems might be similar across states or there may be differences associated with any number of system characteristics. For example, the effects of county-based versus state-based systems, separate versus unified child welfare and juvenile justice systems, and differences in community social characteristics by region of the country are likely to be important sources of variation. These and other issues that characterize differences among child welfare and juvenile justice systems must be addressed to fully understand the roles that culture and climate play in the children’s access to mental health care.

These findings and other research to date suggest that future efforts by policy makers and administrators to improve child welfare and juvenile justice services should focus on the work environment cultures and climates in which the services are provided, but much more research is needed to understand how changes in culture and climate can be accomplished. Although there is a large organizational literature that describes intervention strategies for improving organizational performance by addressing social factors such as culture and climate, almost none of that literature addresses social service or mental health service organizations. Moreover, the literature relies heavily on case studies and almost no specific organizational interventions have been tested with rigorous randomized controlled trials. These issues must be addressed in future research if organizational change strategies for improving child welfare and juvenile justice systems are to be useful to policy makers and administrators.

Copyright information

© Springer Science+Business Media, Inc. 2005