Skip to main content
SpringerLink
Log in

Measures of Workplace Inclusion: A Systematic Review Using the COSMIN Methodology

  • Review
  • Published:
Journal of Occupational Rehabilitation Aims and scope Submit manuscript

Abstract

Purpose To systematically assess the measurement properties and the quality of the evidence for measures of inclusion or exclusion at work. Methods Comprehensive searches of five electronic databases were conducted up to February 2019. Eligible studies aimed to develop a measure of workplace inclusion or exclusion or assessed at least one measurement property. Pairs of reviewers independently screened articles and assessed risk of bias. Methodological quality was appraised with the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. A best-evidence synthesis approach guided the analysis. For each measurement property, evidence quality was rated as high, moderate, low, or very low and results were classified as sufficient, insufficient, or inconsistent. Results The titles and abstracts of 14,380 articles were screened, with 151 full-text articles reviewed for eligibility. Of these, 27 studies were identified, 10 of which were measure development studies. Included measures were the Workplace Ostracism Scale, Ostracism Interventionary Behaviour Scale, Workplace Culture Survey, Workplace Exclusion Scale, Perceived Group Inclusion Scale, Organizational Cultural Intelligence Scale, Inclusion–Exclusion Scale, Climate for Inclusion Scale, Workplace Social Inclusion Scale and the Inclusion-Diversity Scale. Most workplace inclusion instruments were not examined for some form of validity or reliability and evidence for responsiveness was absent. The quality of the evidence for content validity was low for 30% of studies and very low for 70% of studies. Conclusion Future research should focus on comprehensive evaluations of the psychometric properties of existing measures, with an emphasis on content validity, measurement error, reliability and responsiveness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Data Availability

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

References

  1. Bethge M, Radoschewski FM. Physical and psychosocial work stressors, health-related control beliefs and work ability: cross-sectional findings from the German Sociomedical Panel of Employees. Int Arch Occup Environ Health. 2010;83(3):241–250.

    PubMed  Google Scholar 

  2. Nishii LH. The benefits of climate for inclusion for gender-diverse groups. Acad Manage J. 2013;56(6):1754–1774.

    Google Scholar 

  3. Mor Barak ME, Cherin DA. A tool to expand organizational understanding of workforce diversity: exploring a measure of inclusion-exclusion. Adm Soc Work. 1998;22(1):47–644.

    Google Scholar 

  4. Benach J, Vives A, Amable M, Vanroelen C, Tarafa G, Muntaner C. Precarious employment: understanding an emerging social determinant of health. Annu Rev Public Health. 2014;35:229–253.

    CAS  PubMed  Google Scholar 

  5. Benach J, Muntaner C, Santana V. EMCONET (employment conditions knowledge network). Employment conditions and health inequalities: final report to the WHO commission on social determinants of health (CSDH); 2007. p. 172.

  6. Greenglass E, Fiksenbaum L, Burke RJ. Components of social support, buffering effects and burnout: implications for psychological functioning. Anxiety Stress Coping. 1996;9(3):185–197.

    Google Scholar 

  7. Mor Barak ME, Cherin DA, Berkman S. Organizational and personal dimensions in diversity climate: ethnic and gender differences in employee perceptions. J Appl Behav Sci. 1998;34(1):82–104.

    Google Scholar 

  8. Schaufeli WB, Dierendonck DV, Gorp KV. Burnout and reciprocity: towards a dual-level social exchange model. Work Stress. 1996;10(3):225–237.

    Google Scholar 

  9. Mor Barak ME, Levin A. Outside of the corporate mainstream and excluded from the work community: a study of diversity, job satisfaction and well-being. Community, Work Fam. 2002;5(2):133–157.

    Google Scholar 

  10. Mor Barak ME. Managing diversity: toward a globally inclusive workplace. Thousand Oaks: Sage Publications; 2011.

    Google Scholar 

  11. Davidson MN, Ferdman BM. Inclusion: what can I and my organization do about it. Ind-Organ Psychol. 2002;39(4):80–85.

    Google Scholar 

  12. Tajfel H. Social psychology of intergroup relations. Annu Rev Psychol. 1982;33(1):1–39.

    Google Scholar 

  13. Findler L, Wind LH, Mor Barak ME. The challenge of workforce management in a global society: modeling the relationship between diversity, inclusion, organizational culture, and employee well-being, job satisfaction and organizational commitment. Adm Soc Work. 2007;31(3):63–94.

    Google Scholar 

  14. Le H, Jiang Z, Fujimoto Y, Nielsen I. Inclusion and affective well-being: roles of justice perceptions. Pers Rev. 2018;47(4):805–820.

    Google Scholar 

  15. De Jonge J, Schaufeli WB. Job characteristics and employee well-being: a test of Warr's Vitamin Model in health care workers using structural equation modelling. J Organ Behav Int J Ind Occup Organ Psychol Behav. 1998;19(4):387–407.

    Google Scholar 

  16. Snow DL, Kline ML. Preventive interventions in the workplace to reduce negative psychiatric consequences of work and family stress. In: Mazure CM, editor. Does stress cause psychiatric illness?. Washington, DC: American Psychiatric Press; 1995. p. 221–270.

    Google Scholar 

  17. Harrison D, Sellers A. Occupation for mental health and social inclusion. Br J Occup Ther. 2008;71(5):216–219.

    Google Scholar 

  18. Bertram G, Stickley T. Mental health nurses, promoters of inclusion or perpetuators of exclusion? J Psychiatr Ment Health Nurs. 2005;12(4):387–395.

    CAS  PubMed  Google Scholar 

  19. Leary MR, Downs DL. Interpersonal functions of the self-esteem motive. In: Kernis MH, editor. Efficacy, agency, and self-esteem. New York: Springer; 1995. p. 123–144.

    Google Scholar 

  20. Roberson QM. Disentangling the meanings of diversity and inclusion in organizations. Group Organ Manage. 2006;31(2):212–236.

    Google Scholar 

  21. Shore LM, Randel AE, Chung BG, Dean MA, Holcombe Erhart K, Singh G. Inclusion and diversity in work groups: a review and model for future research. J Manage. 2011;37(4):1262–1289.

    Google Scholar 

  22. Brewer MB. The social self: on being the same and different at the same time. Pers Soc Psychol Bull. 1991;17(5):475–482.

    Google Scholar 

  23. Festinger L. A theory of social comparison processes. Human Relat. 1954;7(2):117–140.

    Google Scholar 

  24. Blau P. Power and exchange in social life. New York: Wiley; 1964.

    Google Scholar 

  25. Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191.

    CAS  PubMed  Google Scholar 

  26. Mor Barak ME, Findler L, Wind LH. Diversity, inclusion, and commitment in organizations: international empirical explorations. Institute of applied and behavioral management. J Behav Appl Manage. 2001;2(2):70–91.

    Google Scholar 

  27. United Nations Convention on the Rights of Persons with Disabilities. https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-with-disabilities.html Accessed 7 May 2019.

  28. Zhu X, Law KS, Sun C, Yang D. Thriving of employees with disabilities: the roles of job self-efficacy, inclusion, and team-learning climate. Hum Resour Manage. 2019;58(1):21–34.

    Google Scholar 

  29. Lindsay S, Cagliostro E, Albarico M, Mortaji N, Karon L. A systematic review of the benefits of hiring people with disabilities. J Occup Rehabil. 2018;28(4):634–655.

    PubMed  Google Scholar 

  30. Burke J, Bezyak J, Fraser RT, Pete J, Ditchman N, Chan F. Employers' attitudes towards hiring and retaining people with disabilities: a review of the literature. Aust J Rehabil Couns. 2013;19(1):21–38.

    Google Scholar 

  31. Green JH, Brooke V. Recruiting and retaining the best from America's largest untapped talent pool. J Vocat Rehabil. 2001;16(2):83–88.

    Google Scholar 

  32. Baker PM, Linden MA, LaForce SS, Rutledge J, Goughnour KP. Barriers to employment participation of individuals with disabilities: addressing the impact of employer (mis) perception and policy. Am Behav Sci. 2018;62(5):657–675.

    Google Scholar 

  33. Coombs T, Nicholas A, Pirkis J. A review of social inclusion measures. Aust N Z J Psychiatry. 2013;47(10):906–919.

    PubMed  Google Scholar 

  34. Cordier R, Milbourn B, Martin R, Buchanan A, Chung D, Speyer R. A systematic review evaluating the psychometric properties of measures of social inclusion. PLoS ONE. 2017;12(6):e0179109.

    PubMed  PubMed Central  Google Scholar 

  35. Lysaght R, Cobigo V, Hamilton K. Inclusion as a focus of employment-related research in intellectual disability from 2000 to 2010: a scoping review. Disabil Rehabil. 2012;34(16):1339–1500.

    PubMed  Google Scholar 

  36. Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. The Lancet. 2014;383(9912):166–175.

    Google Scholar 

  37. Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, De Vet HCW, Terwee CB. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1147–1157.

    CAS  PubMed  PubMed Central  Google Scholar 

  38. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–269.

    PubMed  Google Scholar 

  39. World Health Organization. WHO policy on the employment of persons with disabilities. https://www.who.int/disabilities/policies/documents/who_policy_disability.pdf. Accessed 7 May 2019.

  40. Terwee CB, Prinsen CAC, Chiarotto A, de Vet HC, Bouter LM, Alonso J, et al. COSMIN methodology for assessing the content validity of PROMs. User manual, 2018. Version 1.0. p. 72.

  41. Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37(5):360–363.

    PubMed  Google Scholar 

  42. GRADE. GRADE handbook—handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach; 2013.

  43. Mokkink LB, De Vet HCW, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, Terwee CB. COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1171–1179.

    CAS  PubMed  Google Scholar 

  44. Mokkink LB, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, de Vet HCW, Terwee CB. COSMIN methodology for systematic reviews of patient-reported outcome measures (PROMs). User manual, 2018. Version 1.0. p. 78.

  45. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. 2010;10(1):22.

    PubMed  PubMed Central  Google Scholar 

  46. Terwee CB, Mokkink LB, Knol DL, Ostelo R, Bouter LM, de Vet HCW. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21(4):651–657.

    PubMed  Google Scholar 

  47. Mokkink LB, Terwee CB, Gibbons E, Stratford PW, Alonso J, Patrick DL, et al. Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) checklist. BMC Med Res Methodol. 2010;10(1):82.

    PubMed  PubMed Central  Google Scholar 

  48. Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018;27(5):1159–1170.

    CAS  PubMed  PubMed Central  Google Scholar 

  49. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174.

    CAS  PubMed  Google Scholar 

  50. Altman DG. Practical statistics for medical research. Boca Raton: CRC Press; 1990.

    Google Scholar 

  51. Hagner DC. Breaks and birthday cakes: evaluating workplace cultures to develop natural supports for employees with disabilities. St. Augustine: Training Resource Network Inc.; 2000.

    Google Scholar 

  52. Mor Barak ME. Managing diversity: toward a globally inclusive workplace. Thousand Oaks: Sage Publishers; 2005.

    Google Scholar 

  53. Mor Barak ME, Findler L, Wind LH. Cross-cultural aspects of diversity and well-being in the workplace: an international perspective. J Soc Work Res Eval. 2003;4(2):145–169.

    Google Scholar 

  54. Cho S, Mor Barak ME. Understanding of diversity and inclusion in a perceived homogeneous culture: a study of organizational commitment and job performance among Korean employees. Adm Soc Work. 2008;32(4):100–126.

    Google Scholar 

  55. Hwang J, Hopkins K. Organizational inclusion, commitment, and turnover among child welfare workers: a multilevel mediation analysis. Adm Soc Work. 2012;36(1):23–39.

    Google Scholar 

  56. Cottrill K, Denise Lopez P, Hoffman CC. How authentic leadership and inclusion benefit organizations. Equal Divers Incl Int J. 2014;33(3):275–292.

    Google Scholar 

  57. Goswami S, Goswami BK. Exploring the relationship between workforce diversity, inclusion and employee engagement. Drishtikon Manage J. 2018;9(1):65–89.

    Google Scholar 

  58. Chen C, Tang N. Does perceived inclusion matter in the workplace? J Manage Psychol. 2018;33(1):43–57.

    Google Scholar 

  59. Ma E, Qu H, Wilson M. The affective and dispositional consequences of organizational citizenship behavior: a cross-cultural study. J Hosp Tour Res. 2016;40(4):399–43131.

    Google Scholar 

  60. Hitlan RT, Cliffton RJ, DeSoto MC. Perceived exclusion in the workplace: the moderating effects of gender on work-related attitudes and psychological health. N Am J Psychol. 2006;8(2):217–236.

    Google Scholar 

  61. Hitlan RT, Noel J. The influence of workplace exclusion and personality on counterproductive work behaviours: an interactionist perspective. Eur J Work Organ Psychol. 2009;18(4):477–502.

    Google Scholar 

  62. Hagner D, Dague B, Phillips K. Including employees with disabilities in workplace cultures: strategies and barriers. Rehabil Couns Bull. 2015;58(4):195–202.

    Google Scholar 

  63. Jansen WS, Otten S, van der Zee KI. Being different at work: how gender dissimilarity relates to social inclusion and absenteeism. Group Process Intergroup Relat. 2017;20(6):879–893.

    Google Scholar 

  64. Chow IH-S. Cognitive diversity and creativity in teams: the mediating roles of team learning and inclusion. Chin Manage Stud. 2018;12(2):369–383.

    Google Scholar 

  65. Downey SN, van der Werff L, Thomas KM, Plaut VC. The role of diversity practices and inclusion in promoting trust and employee engagement. J Appl Soc Psychol. 2015;45(1):35–44.

    Google Scholar 

  66. Fiset J, Boies K. Seeing the unseen: ostracism interventionary behaviour and its impact on employees. Eur J Work Organ Psychol. 2018;27(4):403–417.

    Google Scholar 

  67. Jansen WS, Otten S, van der Zee KI, Jans L. Inclusion: conceptualization and measurement. Eur J Soc Psychol. 2014;44(4):370–385.

    Google Scholar 

  68. Pearce JL, Randel AE. Expectations of organizational mobility, workplace social inclusion, and employee job performance. J Organ Behav Int J Ind Occup Organ Psychol Behav. 2004;25(1):81–988.

    Google Scholar 

  69. Lima JE, West GR, Winston BE, Wood JA. Measuring organizational cultural intelligence: the development and validation of a scale. Int J Cross Cult Manage. 2016;16(1):9–31.

    Google Scholar 

  70. Ferris DL, Brown DJ, Berry JW, Lian H. The development and validation of the Workplace Ostracism Scale. J Appl Psychol. 2008;93(6):1348–1366.

    PubMed  Google Scholar 

  71. Fisher RJ. Social desirability bias and the validity of indirect questioning. J Consum Res. 1993;20(2):303–315.

    Google Scholar 

  72. Nelissen PT, Hulsheger UR, van Ruitenbeek GMC, Zijlstra FRH. Lending a helping hand at work: a multilevel investigation of prosocial motivation, inclusive climate and inclusive behavior. J Occup Rehabil. 2017;27(3):467–476.

    PubMed  Google Scholar 

Download references

Acknowledgements

This study was funded in part by the Canadian Institutes of Health Research-Social Sciences and Humanities Research Council (CIHR-SSHRC) Partnership Grant (01561-000 and 895-2018-4002) awarded to Sally Lindsay and the Kimel Family Fund through the Holland Bloorview Kids Rehabilitation Hospital.

Author information

Authors and Affiliations

  1. Bloorview Research Institute, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON, Canada

    Mana Rezai, Kendall Kolne, Sunny Bui & Sally Lindsay

  2. Rehabilitation Sciences Institute, University of Toronto, Toronto, ON, Canada

    Sunny Bui

  3. Department of Occupational Science and Occupational Therapy, University of Toronto, Toronto, ON, Canada

    Sally Lindsay

Authors
  1. Mana Rezai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kendall Kolne
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sunny Bui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sally Lindsay
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Mana Rezai and Sally Lindsay contributed to the study conception and design. All authors contributed to the literature search and data extraction. Analyses were primarily performed by Mana Rezai, and Kendall Kolne. The first draft of the manuscript was written by Mana Rezai and all authors commented on subsequent versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mana Rezai.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical Approval

No animal or human studies were carried out by the authors for this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rezai, M., Kolne, K., Bui, S. et al. Measures of Workplace Inclusion: A Systematic Review Using the COSMIN Methodology. J Occup Rehabil 30, 420–454 (2020). https://doi.org/10.1007/s10926-020-09872-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10926-020-09872-4

Keywords

Search

Navigation