Advertisement

Concordance Between Administrator and Clinician Ratings of Organizational Culture and Climate

  • Rinad S. Beidas
  • Nathaniel J. Williams
  • Philip D. Green
  • Gregory A. Aarons
  • Emily M. Becker-Haimes
  • Arthur C. Evans
  • Ronnie Rubin
  • Danielle R. Adams
  • Steven C. Marcus
Original Article

Abstract

Organizational culture and climate are important determinants of behavioral health service delivery for youth. The Organizational Social Context measure is a well validated assessment of organizational culture and climate that has been developed and extensively used in public sector behavioral health service settings. The degree of concordance between administrators and clinicians in their reports of organizational culture and climate may have implications for research design, inferences, and organizational intervention. However, the extent to which administrators’ and clinicians’ reports demonstrate concordance is just beginning to garner attention in public behavioral health settings in the United States. We investigated the concordance between 73 administrators (i.e., supervisors, clinical directors, and executive directors) and 247 clinicians in 28 child-serving programs in a public behavioral health system. Findings suggest that administrators, compared to clinicians, reported more positive cultures and climates. Organizational size moderated this relationship such that administrators in small programs (<466 youth clients served annually) provided more congruent reports of culture and climate in contrast to administrators in large programs (≥466 youth clients served annually) who reported more positive cultures and climates than clinicians. We propose a research agenda that examines the effect of concordance between administrators and clinicians on organizational outcomes in public behavioral health service settings.

Keywords

Organizational social context Organizational culture and climate Concordance Leadership Organizational size 

Notes

Acknowledgements

We are especially grateful for the support that the Department of Behavioral Health and Intellectual disAbility Services has provided for this project, and for the Evidence Based Practice and Innovation (EPIC) group. We thank Charles Glisson, PhD, and Sonja Schoenwald, PhD, for their comments on earlier versions of this manuscript.

Funding

Funding for this research project was supported by NIMH K23 MH099179 (Beidas).

Compliance with Ethical Standards

Conflict of interest

Dr. Beidas receives royalties from Oxford University Press. Dr. Marcus has received grant support from Ortho-McNeil Janssen and Forest Research Institute and has served as a consultant to AstraZeneca and Alkermes. All other authors have no conflicts of interest to report.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

References

  1. Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255–274. doi: 10.1146/annurev-publhealth-032013-182447.CrossRefPubMedPubMedCentralGoogle Scholar
  2. Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., Sklar, M., & Horowitz, J. (2015). Discrepancies in leader and follower ratings of transformational leadership: Relationship with organizational culture in mental health. Administration and Policy in Mental Health. doi: 10.1007/s10488-015-0672-7.Google Scholar
  3. Aarons, G. A., Ehrhart, M. G., Torres, E. L., Finn, N. K., & Beidas, R. S. (2016). The humble leader: Association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health organizations. Psychiatric Services. doi: 10.1176/appi.ps.201600062.
  4. Aarons, G. A., & Sawitzky, A. C. (2006a). Organizational climate partially mediates the effect of culture on work attitudes and staff turnover in mental health services. Administration and Policy in Mental Health, 33(3), 289–301. doi: 10.1007/s10488-006-0039-1.CrossRefPubMedPubMedCentralGoogle Scholar
  5. Aarons, G. A., & Sawitzky, A. C. (2006b). Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychological Services, 3(1), 61–72. doi: 10.1037/1541-1559.3.1.61.CrossRefPubMedPubMedCentralGoogle Scholar
  6. Aarons, G. A., Sommerfeld, D. H., Hecht, D. B., Silovsky, J. F., & Chaffin, M. J. (2009). The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: Evidence for a protective effect. Journal of Consulting and Clinical Psychology, 77(2), 270. doi: 10.1037/a0013223.CrossRefPubMedPubMedCentralGoogle Scholar
  7. Aarons, G. A., Sommerfeld, D. H., & Willging, C. E. (2011). The soft underbelly of system change: The role of leadership and organizational climate in turnover during statewide behavioral health reform. Psychological Services, 8(4), 269–281.CrossRefPubMedPubMedCentralGoogle Scholar
  8. Antonakis, J., & Atwater, L. (2002). Leader distance: A review and a proposed theory. Leadership Quarterly, 13(6), 673–704. doi: 10.1016/S1048-9843(02)00155-8.CrossRefGoogle Scholar
  9. Avolio, B. J., Zhu, W. C., Koh, W., & Bhatia, P. (2004). Transformational leadership and organizational commitment: Mediating role of psychological empowerment and moderating role of structural distance. Journal of Organizational Behavior, 25(8), 951–968. doi: 10.1002/job.283.CrossRefGoogle Scholar
  10. Bashshur, M. R., Hernández, A., & González-Romá, V. (2011). When managers and their teams disagree: A longitudinal look at the consequences of differences in perceptions of organizational support. Journal of Applied Psychology, 96(3), 558. doi: 10.1037/a0022675.CrossRefPubMedGoogle Scholar
  11. Beidas, R. S., Aarons, G. A., Barg, F. K., Evans, A., Hadley, T., Hoagwood, K. E., … Mandell, D. S. (2013). Policy to implementation: Evidence-based practice in community mental health–study protocol. Implementation Science, 8(1), 1. doi: 10.1186/1748-5908-8-38.CrossRefGoogle Scholar
  12. Beidas, R. S., Marcus, S., Aarons, G. A., Hoagwood, K. E., Schoenwald, S. K., Evans, A. C., … Mandell, D. S. (2015). Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatrics, 169(4), 374–382. doi: 10.1001/jamapediatrics.2014.3736.CrossRefPubMedPubMedCentralGoogle Scholar
  13. Beidas, R. S., Marcus, S., Wolk, C. B., Powell, B., Aarons, G. A., Evans, A. C., … Mandell, D. S. (2016). A prospective examination of clinician and supervisor turnover within the context of implementation of evidence-based practices in a publicly-funded mental health system. Administration and Policy in Mental Health, 43(5), 640–649. doi: 10.1007/s10488-015-0673-6.CrossRefPubMedPubMedCentralGoogle Scholar
  14. Beidas, R. S., Wolk, C. L., Walsh, L. M., Evans, A. C. Jr., Hurford, M. O., & Barg, F. K. (2014). A complementary marriage of perspectives: Understanding organizational social context using mixed methods. Implementation Science, 9, 175. doi: 10.1186/s13012-014-0175-z.CrossRefPubMedPubMedCentralGoogle Scholar
  15. Cameron, K. S., & Quinn, R. E. (2011). Diagnosing and changing organizational culture: Based on the competing values framework (3 edn.). San Francisco, CA: Jossey-Bass.Google Scholar
  16. Carlfjord, S., Andersson, A., Nilsen, P., Bendtsen, P., & Lindberg, M. (2010). The importance of organizational climate and implementation strategy at the introduction of a new working tool in primary health care. Journal of Evaluation in Clinical Practice, 16(6), 1326–1332. doi: 10.1111/j.1365-2753.2009.01336.x.CrossRefPubMedGoogle Scholar
  17. Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83(2), 234–246. doi: 10.1037/0021-9010.83.2.234.CrossRefGoogle Scholar
  18. Cole, M. S., Carter, M. Z., & Zhang, Z. (2013). Leader-team congruence in power distance values and team effectiveness: The mediating role of procedural justice climate. Journal of Applied Psychology, 98(6), 962. doi: 10.1037/a0034269.CrossRefPubMedGoogle Scholar
  19. Cooke, R. A., & Rousseau, D. M. (1988). Behavioral norms and expectations: A quantitative approach to the assessment of organizational culture. Group & Organization Studies, 13(3), 245–273. doi: 10.1177/105960118801300302.CrossRefGoogle Scholar
  20. Cunningham, C. T., Quan, H., Hemmelgarn, B., Noseworthy, T., Beck, C. A., Dixon, E., … Jetté, N. (2015). Exploring physician specialist response rates to web-based surveys. BMC Medical Research Methodology, 15(1), 1. doi: 10.1186/s12874-015-0016-z.CrossRefGoogle Scholar
  21. Ehrhart, M., Schneider, B., & Macey, W. H. (2014). Organizational climate and culture: An introduction to theory, research, and practice. New York, NY: Routledge.Google Scholar
  22. Fleenor, J. W., McCauley, C. D., & Brutus, S. (1996). Self-other rating agreement and leader effectiveness. Leadership Quarterly, 7(4), 487–506. doi: 10.1016/S1048-9843(96)90003-X.CrossRefGoogle Scholar
  23. Gibson, C. B., Cooper, C. D., & Conger, J. A. (2009). Do you see what we see? The complex effects of perceptual distance between leaders and teams. Journal of Applied Psychology, 94(1), 62. doi: 10.1037/a0013073.CrossRefPubMedGoogle Scholar
  24. Glisson, C. (2002). The organizational context of children’s mental health services. Clinical Child and Family Psychology Review, 5(4), 233–253. doi: 10.1023/A:1020972906177.CrossRefPubMedGoogle Scholar
  25. Glisson, C., & Green, P. (2011). Organizational climate, services, and outcomes in child welfare systems. Child Abuse & Neglect, 35(8), 582–591. doi: 10.1016/j.chiabu.2011.04.009.CrossRefGoogle Scholar
  26. Glisson, C., Green, P., & Williams, N. J. (2012). Assessing the Organizational Social Context (OSC) of child welfare systems: Implications for research and practice. Child Abuse & Neglect, 36(9), 621–632. doi: 10.1016/j.chiabu.2012.06.002.CrossRefGoogle Scholar
  27. Glisson, C., & Hemmelgarn, A. (1998). The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse & Neglect, 22(5), 401–421. doi: 10.1016/S0145-2134(98)00005-2.CrossRefGoogle Scholar
  28. Glisson, C., Landsverk, J., Schoenwald, S. K., Kelleher, K., Hoagwood, K. E., Mayberg, S., … Research Network on Youth Mental Health. (2008a). Assessing the Organizational Social Context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health, 35(1–2), 98–113. doi: 10.1007/s10488-007-0148-5.CrossRefPubMedGoogle Scholar
  29. Glisson, C., Schoenwald, S. K., Hemmelgarn, A., Green, P., Dukes, D., Armstrong, K. S., & Chapman, J. E. (2010). Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal of Consulting and Clinical Psychology, 78(4), 537–550. doi: 10.1037/a0019160.CrossRefPubMedPubMedCentralGoogle Scholar
  30. Glisson, C., Schoenwald, S. K., Kelleher, K., Landsverk, J., Hoagwood, K. E., Mayberg, S., … Research Network on Youth Mental Health. (2008b). Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Administration and Policy in Mental Health, 35(1–2), 124–133. doi: 10.1007/s10488-007-0152-9.CrossRefPubMedGoogle Scholar
  31. Glisson, C., Williams, N. J., Hemmelgarn, A., Proctor, E., & Green, P. (2016). Aligning organizational priorities with ARC to improve youth mental health service outcomes. Journal of Consulting and Clinical Psychology, 84(8), 713–725. doi: 10.1037/ccp0000107.CrossRefPubMedPubMedCentralGoogle Scholar
  32. Hansen, L. O., Williams, M. V., & Singer, S. J. (2011). Perceptions of hospital safety climate and incidence of readmission. Health Services Research, 46(2), 596–616. doi: 10.1111/j.1475-6773.2010.01204.x.CrossRefPubMedPubMedCentralGoogle Scholar
  33. Hartmann, C. W., Rosen, A. K., Meterko, M., Shokeen, P., Zhao, S., Singer, S., … Gaba, D. M. (2008). An overview of patient safety climate in the VA. Health Services Research, 43(4), 1263–1284. doi: 10.1111/j.1475-6773.2008.00839.x.CrossRefPubMedPubMedCentralGoogle Scholar
  34. Hartnell, C. A., Ou, A. Y., & Kinicki, A. (2011). Organizational culture and organizational effectiveness: A meta-analytic investigation of the competing values framework’s theoretical suppositions. Journal of Applied Psychology, 96(4), 677–694. doi: 10.1037/a0021987.CrossRefPubMedGoogle Scholar
  35. Hasson, H., Gilbert-Ouimet, M., Baril-Gingras, G., Brisson, C., Vezina, M., Bourbonnais, R., & Montreuil, S. (2012). Implementation of an organizational-level intervention on the psychosocial environment of work comparison of managers’ and employees’ views. Journal of Occupational and Environmental Medicine, 54(1), 85–91. doi: 10.1097/JOM.0b013e31823ccb2f.CrossRefPubMedGoogle Scholar
  36. Hasson, H., von Thiele Schwarz, U., Nielsen, K., & Tafvelin, S. (2016). Are we all in the same boat? The role of perceptual distance in organizational health interventions. Stress and Health. doi: 10.1002/smi.2703.Google Scholar
  37. Hedeker, D., & Gibbons, R. D. (2006). Longitudinal data analysis (Vol. 451). Hoboken, NJ: Wiley.Google Scholar
  38. Hoagwood, K. E., Burns, B. J., Kiser, L., Ringeisen, H., & Schoenwald, S. K. (2001). Evidence-based practice in child and adolescent mental health services. Psychiatric Services, 52(9), 1179–1189. doi: 10.1176/appi.ps.52.9.1179.CrossRefPubMedGoogle Scholar
  39. James, L. R., Hater, J. J., Gent, M. J., & Bruni, J. R. (1978). Psychological climate: Implications from cognitive social-learning theory and interactional psychology. Personnel Psychology, 31(4), 783–813. doi: 10.1111/j.1744-6570.1978.tb02124.x.CrossRefGoogle Scholar
  40. Kimberly, J. R. (1976). Organizational size and the structuralist perspective: A review, critique, and proposal. Administrative Science Quarterly, 21(4), 571–597. doi: 10.2307/2391717.CrossRefGoogle Scholar
  41. Klein, K. J., & Kozlowski, S. W. (2000). Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions. San Francisco, CA: Jossey-Bass.Google Scholar
  42. Martin, A. J., Jones, E. S., & Callan, V. J. (2006). Status differences in employee adjustment during organizational change. Journal of Managerial Psychology, 21(2), 145–162. doi: 10.1108/02683940610650758.CrossRefGoogle Scholar
  43. Olin, S. S., Williams, N., Pollock, M., Armusewicz, K., Kutash, K., Glisson, C., & Hoagwood, K. E. (2014). Quality indicators for family support services and their relationship to organizational social context. Administration and Policy in Mental Health, 41(1), 43–54. doi: 10.1007/s10488-013-0499-z.CrossRefPubMedGoogle Scholar
  44. Ostroff, C., Kinicki, A., & Tamkins, M. (2003). Organizational culture and climate (Vol. 12). Hoboken, NJ: Wiley.Google Scholar
  45. Raudenbush, S. W. (2004). HLM 6: Hierarchical linear and nonlinear modeling. Lincolnwood, IL: Scientific Software International.Google Scholar
  46. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). Thousand Oaks, CA: Sage.Google Scholar
  47. Rindfuss, R. R., Choe, M. K., Tsuya, N. O., Bumpass, L. L., & Tamaki, E. (2015). Do low survey response rates bias results? Evidence from Japan. Demographic Research, 32, 797. doi: 10.4054/DemRes.2015.32.26.CrossRefGoogle Scholar
  48. Rogers, E. M. (2003). Elements of diffusion. In Diffusion of innovations (5th edn., pp. 1–38). New York, NY: Free Press.Google Scholar
  49. Rosen, A. K., Singer, S., Shibei, Z., Shokeen, P., Meterko, M., & Gaba, D. (2010). Hospital safety climate and safety outcomes: Is there a relationship in the VA? Medical Care Research and Review, 67(5), 590–608. doi: 10.1177/1077558709356703.CrossRefPubMedGoogle Scholar
  50. Rothman, K. J. (1990). No adjustments are needed for multiple comparisons. Epidemiology (Cambridge, Mass), 1(1), 43–46.CrossRefGoogle Scholar
  51. Schneider, B., Ehrhart, M. G., & Macey, W. H. (2013). Organizational climate and culture. Annual Review of Psychology, 64, 361–388. doi: 10.1146/annurev-psych-113011-143809.CrossRefPubMedGoogle Scholar
  52. Schoenwald, S. K., Chapman, J. E., Kelleher, K., Hoagwood, K. E., Landsverk, J., Stevens, J., … Research Network on Youth Mental Health. (2008). A survey of the infrastructure for children’s mental health services: Implications for the implementation of empirically supported treatments (ESTs). Administration and Policy in Mental Health, 35(1–2), 84–97. doi: 10.1007/s10488-007-0147-6.CrossRefPubMedGoogle Scholar
  53. Singer, S., Lin, S., Falwell, A., Gaba, D., & Baker, L. (2009). Relationship of safety climate and safety performance in hospitals. Health Services Research, 44(2 Pt 1), 399–421. doi: 10.1111/j.1475-6773.2008.00918.x.CrossRefPubMedPubMedCentralGoogle Scholar
  54. Van Velsor, E., Taylor, S., & Leslie, J. (1993). An examination of the relationships among self-perception accuracy, self-awareness, gender, and leader effectiveness. Human Resource Management, 32(2–3), 249–264. doi: 10.1002/hrm.3930320205.CrossRefGoogle Scholar
  55. Verbeke, W., Volgering, M., & Hessels, M. (1998). Exploring the conceptual expansion within the field of organizational behaviour: Organizational climate and organizational culture. Journal of Management Studies, 35(3), 303–329. doi: 10.1111/1467-6486.00095.CrossRefGoogle Scholar
  56. Weisz, J. R. (1997). Therapist background questionnaire. Los Angeles: University of California.Google Scholar
  57. Williams, N. J., & Glisson, C. (2014). Testing a theory of organizational culture, climate and youth outcomes in child welfare systems: A United States national study. Child Abuse & Neglect, 38(4), 757–767. doi: 10.1016/j.chiabu.2013.09.003.CrossRefGoogle Scholar
  58. Wolf, D. A. P. S., Dulmus, C., Maguin, E., Keesler, J., & Powell, B. (2014). Organizational leaders’ and staff members’ appraisals of their work environment within a children’s social service system. Human Service Organizations Management Leadership & Governance, 38(3), 215–227. doi: 10.1080/23303131.2014.884032.CrossRefGoogle Scholar
  59. Yammarino, F. J., & Atwater, L. E. (1997). Do managers see themselves as others see them? Implications of self-other rating agreement for human resources management. Organizational Dynamics, 25(4), 35–44. doi: 10.1016/S0090-2616(97)90035-8.CrossRefGoogle Scholar
  60. Zyphur, M. J., Zammuto, R. F., & Zhang, Z. (2016). Multilevel latent polynomial regression for modeling (in)congruence across organizational groups: The case of organizational culture research. Organizational Research Methods, 19(1), 53–79. doi: 10.1177/1094428115588570.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Rinad S. Beidas
    • 1
  • Nathaniel J. Williams
    • 2
  • Philip D. Green
    • 3
  • Gregory A. Aarons
    • 4
  • Emily M. Becker-Haimes
    • 1
  • Arthur C. Evans
    • 1
    • 5
  • Ronnie Rubin
    • 1
    • 5
  • Danielle R. Adams
    • 1
  • Steven C. Marcus
    • 6
  1. 1.Department of PsychiatryUniversity of Pennsylvania Perelman School of MedicinePhiladelphiaUSA
  2. 2.School of Social WorkBoise State UniversityBoiseUSA
  3. 3.Center for Behavioral Health Research, College of Social WorkUniversity of TennesseeKnoxvilleUSA
  4. 4.Department of PsychiatryUniversity of California San DiegoSan DiegoUSA
  5. 5.Department of Behavioral Health and Intellectual Disability ServicesPhiladelphiaUSA
  6. 6.School of Social Policy and PracticeUniversity of PennsylvaniaPhiladelphiaUSA

Personalised recommendations