Abstract
A variety of collective phenomena are understood to exist to the extent that workers agree on their perceptions of the phenomena, such as perceptions of their organization’s climate or perceptions of their team’s mental model. Researchers conducting group-level studies of such phenomena measure individuals’ perceptions via surveys and then aggregate data to the group level if the mean within-group agreement for a sample of groups is sufficiently high. Despite this widespread practice, we know little about the factors potentially affecting mean within-group agreement. Here, focusing on work climate, we report an investigation of a number of expected contextual (social interaction) and methodological predictors of mean rWG, a common statistic for judging within-group agreement in applied psychology and management research. We used the novel approach of meta-CART, which allowed us to assess the relative importance and possible interactions of the predictor variables. Notably, mean rWG values are driven by both contextual (average number of individuals per group and cultural individualism-collectivism) and methodological factors (the number of items in a scale and scale reliability). Our findings are largely consistent with expectations concerning how social interaction affects within-group agreement and psychometric arguments regarding why adding more items to a scale will not necessarily increase the magnitude of an index based on a Spearman-Brown “stepped-up correction.” We discuss the key insights from our results, which are relevant to the study of multilevel phenomena relying on the aggregation of individual-level data and informative for how meta-analytic researchers can simultaneously examine multiple moderator variables.
References
*An asterisk indicates that a study was included in the meta-CART analyses. aInformation from Van de Voorde et al. (2010) and Van de Voorde et al. (2014) was combined due to use of the same sample..
Abdelhadi, N., & Drach-Zahavy, A. (2012). Promoting patientcare: Work engagement as a mediator between ward service climate and patient-centered care. Journal of Advanced Nursing, 68, 1276–1287.
*Afsharian, A., Zadow, A., Dollard, M. F., Dormann, C., & Ziaian, T. (2018). Should psychosocial safety climate theory be extended to include climate strength? Journal of Occupational Health Psychology, 23, 496–507.
*Alfes, K., Shantz, A. D., & Ritz, A. (2018). A multilevel examination of the relationship between role overload and employee subjective health: The buffering effect of support climates. Human Resource Management, 57, 659–673.
Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Brooks/Cole Publishing Company.
*Ambrose, M. L., Schminke, M., & Mayer, D. M. (2013). Trickle-down effects of supervisor perceptions of interactional justice: A moderated mediation approach. Journal of Applied Psychology, 98, 678–689.
*Anderson, N. R., & West, M. A. (1998). Measuring climate for work group innovation: Development and validation of the team climate inventory. Journal of Organizational Behavior, 19, 235–258.
*Auh, S., Menguc, B., & Jung, Y. S. (2014). Unpacking the relationship between empowering leadership and service-oriented citizenship behaviors: A multilevel approach. Journal of the Academy of Marketing Science, 42, 558–579.
*Bai, Y., Lin, L., & Liu, J. T. (2017). Leveraging the employee voice: A multi-level social learning perspective of ethical leadership. The International Journal of Human Resource Management.
*Bain, P. G., Mann, L., & Pirola-Merlo, A. (2001). The innovation imperative: The relationships between team climate, innovation, and performance in research and development teams. Small Group Research, 32, 55–73.
Bandalos, D., & Enders, C. (1996). The effects of nonnormality and number of response categories on reliability. Applied Measurement in Education, 9, 151–160.
*Berson, Y., Da’as, R., & Waldman, D. (2015). How do leaders and their teams bring about organizational learning and outcomes? Personnel Psychology, 68, 79–108.
Biemann, T., Cole, M. S., & Voelpel, S. (2012). Within-group agreement: On the use (and misues) of rWG and rWG(J) in leadership research and some best practice guidelines. The Leadership Quarterly, 23, 66–80.
Biemann, T., Ellwart, T., & Rack, O. (2014). Quatifying similarity of team mental models: An introduction of the rRG index. Group Processes & Intergroup Relations, 17, 125–140.
*Bilinska, P., Wegge, J., & Kliegel, M. (2016). Caring for the elderly but not for one’s own old employees? Journal of Personnel Psychology, 15(3), 95–105.
*Bin, D.A. (2018). Organizational climate and climate strength: Three essays. Doctoral Dissertation, University of Lausanne, Switzerland.
*Binci, D. (2011). Climate for innovation and ICT implementation effectiveness: A missing link in Italian e-government projects. International Journal of Public Administration, 34, 49–53.
Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Klein & S. W. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations (pp. 349–381). Jossey-Bass.
*Bodla, A.A., Tang, N., Jiang, W., & Tian, L. (2016). Diversity and creativity in cross-national teams: The role of team knowledge sharing and inclusive climate. Journal of Management and Organization, 1–19.
*Boehm, S. A., Dwertmann, D. J. G., Bruch, H., & Shamir, B. (2015). The missing link? Investigating organizational identity strength and transformational leadership climate as mechanisms that connect CEO charisma with firm performance. The Leadership Quarterly, 26, 156–171.
Bogaert, S., Boone, C., & van Witteloostuijn, A. (2012). Social value orientation and climate strength as moderators of the impact of work group cooperative climate on affective commitment. Journal of Management Studies, 49, 918–944.
Borman, W. C., Kubisiak, C., & Schneider, R. J. (1999). Work styles. In N. G. Peterson, M. D. Mumford, W. C. Borman, P. R. Jeanneret, & E. A. Fleishman (Eds.), An occupational information system for the 21st century: The development of O*NET (pp. 213–226). American Psychological Association.
*Bornay-Barrachina, M., & Herrero, I. (2018). Team creative environment as a mediator between CWX and R&D team performance and moderating boundary conditions. Journal of Business and Psychology, 33, 311–323.
*Borucki, C. C., & Burke, M. J. (1999). An examination of service-related antecedents to retail store performance. Journal of Organizational Behavior, 20, 943–962.
*Bosak, J., Dawson, J., Flood, P., & Peccei, R. (2017). Employee involvement climate and climate strength: A study of employee attitudes and organizational effectiveness in UK hospitals. Journal of Organizational Effectiveness: People and Performance, 4, 18–38.
*Bosak, J., Fu, N., Ma, Q., & Flood, P. (2016). Intellectual capital and organizational ambidexterity in Chinese and Irish professional service firms. Journal of Organizational Effectiveness: People and Performance, 3, 94–114.
*Boyce, A.S. (2010). Organizational climate and performance: An examination of causal priority. Unpublished doctoral dissertation, Michigan State University, East Lansing, MI.
*Brahm, T., & Kunze, F. (2012). The role of trust climate in virtual teams. Journal of Managerial Psychology, 27, 595–614.
Brieman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1998). Classification and regression trees. Chapman and Hall/CRC.
*Bronkhorst, B., & Vermeeren, B. (2016). Safety climate, worker health and organizational health performance. International Journal of Workplace Health Management, 9(270), 289.
Brown, R. D., & Hauenstein, N. M. A. (2005). Interrater agreement reconsidered: An alternative to the rWG indices. Organizational Research Methods, 8(2), 165–184.
*Buck, M.A. (2011). Proactive personality and big five traits in supervisors and workgroup members: Effects on safety climate and safety motivation. Dissertations and Theses. Paper 268. https://doi.org/10.15760/etd.268
Burke, M. J., Chan-Serafin, S., Salvador, R., Smith, A., & Sarpy, S. (2008). The role of national culture and organizational climate in safety training effectiveness. European Journal of Work and Organizational Psychology, 17, 133–154.
Burke, M. J., Cohen, A., Doveh, E., & Smith-Crowe, K. (2018). Central tendency and matched difference approaches for assessing interrater agreement. Journal of Applied Psychology, 103, 1198–1229.
Burke, M. J., Finkelstein, L. M., & Dusig, M. S. (1999). On average deviation indices for estimating interrater agreement. Organizational Research Methods, 2, 49–68.
Burke, M. I., Landis, R. L., & Burke, M. J. (2017). Estimating group-level relationships: General recommendations and considerations for the use of intraclass correlation coefficients. Journal of Business and Psychology, 32, 611–626.
*Buvik, M. P., & Tvedt, S. D. (2016). The impact of commitment and climate strength on the relationship between trust and performance in cross-functional project teams: A moderated mediation analysis. Team Performance Management, 22, 114–138.
*Carrasco, H., Martínez-Tur, V., Peiró, J. M., & Moliner, C. (2012). Validation of a measure of service climate in organizations. Journal of Work and Organizational Psychology, 28, 69–80.
Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234–246.
*Chen, S., Zhang, G., Zhang, A., & Xu, J. (2016). Collectivism-oriented human resource management and innovation performance: An examination of team reflexivity and team psychological safety. Journal of Management & Organization, 22, 535–554.
*Chen, X. P., Liu, D., & Portnoy, R. (2012). A multilevel investigation of motivational cultural intelligence, organizational diversity climate, and cultural sales: Evidence from U.S. real estate firms. Journal of Applied Psychology, 97, 93–106.
*Chuang, C., & Liao, H. (2010). Strategic human resource management in service context: Taking care of business by taking care of employees and customers. Personnel Psychology, 63, 153–196.
*Cole, M. S., Carter, M., & Zhang, Z. (2013). Leader-team congruence in power distance values and team effectiveness: The mediating role of procedural justice climate. Journal of Applied Psychology, 98, 962–973.
Colquitt, J. A., Noe, R. A., & Jackson, C. L. (2002). Justice in teams: Antecedents and consequences of procedural justice climate. Personnel Psychology, 55, 83–109.
*Costa, A. C. (2003). Work team trust and effectiveness. Personnel Review, 32, 605–622.
*Darr, W., & Johns, G. (2004). Political decision-making climates: Theoretical processes and multi-level antecedents. Human Relations, 57, 169–200.
*de Jong, A., de Ruyter, K., & Lemmink, J. (2005). Service climate in self-managing teams: Mapping the linkage of team member perceptions and service performance outcomes in a business-to-business setting. Journal of Management Studies, 42, 1593–1620.
*Dietz, J., Pugh, S. D., & Wiley, J. (2004). Service climate effects on customer attitudes: An examination of boundary conditions. Academy of Management Journal, 47, 81–92.
*Ehrhart, M. G. (2004). Leadership and procedural justice climate as antecedents of unit-level organizational citizenship behavior. Personnel Psychology, 57, 61–94.
*Eisenbeiss, S., Knippenberg, D. V., & Boerner, S. (2008). Transformational leadership and team innovation: Integrating team climate principles. Journal of Applied Psychology, 93, 1438–1446.
*Elche, D., Ruiz-Palomino, P., & Linuesa-Langreo, J. (2020). Servant leadership and organizational citizenship behavior: The mediating effect of empathy and service climate. International Journal of Contemporary Hospitality Management, 32, 2035–2053.
Enders, C., & Bandalos, D. (1999). The effects of heterogeneous item distributions on reliability. Applied Measurement in Education, 12, 133–150.
*Frazier, M. L., & Bowler, W. M. (2015). Voice climate, supervisor undermining, and work outcomes: A group-level examination. Journal of Management, 41, 841–863.
*Frazier, M. L., & Fainshmidt, S. (2012). Voice climate, work outcomes, and the mediating role of psychological empowerment: A multilevel examination. Group & Organization Management, 37, 691–715.
*Gelade, G. A., & Young, S. (2005). Test of a service profit chain model in the retail-banking sector. Journal of Occupational & Organizational Psychology, 78, 1–22.
*Getachew, D. S., & Zhou, E. (2018). The influences of transformational leadership on collective efficacy: The moderating role of perceived organizational support. The International Journal of Organizational Innovation, 10, 7–15.
Ginsburg, L., & Oore, D. G. (2016). Patient safety climate strength: A concept that requires more attention. BMJ Quality & Safety, 25, 680–687.
*Gonzalez, J. A., & Denisi, A. S. (2009). Cross-level effects of demography and diversity climate on organizational attachment and firm effectiveness. Journal of Organizational Behavior, 30, 21–40.
*Gonzalez-Mulé, E., Courtright, S. H., DeGeest, D., Seong, J.-Y., & Hong, D.-S. (2016). Channeled autonomy: The joint effects of autonomy and feedback on team performance through organizational goal clarity. Journal of Management, 42, 2018–2033.
*Gonzalez-Roma, V., & Hernandez, A. (2014). Climate uniformity: Its influence on team communication quality, task conflict, and team performance. Journal of Applied Psychology, 99, 1042–1058.
Gonzalez-Roma, V., & Peiro, J. (2014). Climate and culture strength. In B. Schneider & K. M. Barbera (Eds.), The Oxford handbook of organizational climate and culture (pp. 496–531). Oxford University Press.
Gonzalez-Roma, V., Peiro, J. M., & Tordera, N. (2002). An examination of the antecedents and moderator influences of climate strength. Journal of Applied Psychology, 87, 465–473.
*Griffith, J. (2006). A compositional analysis of the organizational climate-performance relation: Public schools as organizations. Journal of Applied Social Psychology, 36, 1848–1880.
Guolo, A., & Varin, C. (2017). Random-effects meta-analysis: The number of studies matters. Statistical Methods in Medical Research, 26, 1500–1518.
*Guerrero, E. G., Fenwick, K., & Kong, Y. (2017). Advancing theory development: Exploring the leadership–climate relationship as a mechanism of the implementation of cultural competence. Implementation Science, 12, 1–12.
Gupta, V., Hanges, P. J., & Dorfman, P. (2002). Cultural clusters: Methodology and findings. Journal of World Business, 37, 11–15.
Hare, A. P. (1981). Group size. American Behavioral Scientist, 24, 695–708.
Hayes, T., Usami, S., Jacobucci, R., & McArdle, J. J. (2015). Using classification and regression trees (CART) and random forests to analyze attrition: Results from two simulations. Psychology and Aging, 30, 911–929.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Academic Press.
*Heled, E., Somech, A., & Waters, L. (2016). Psychological capital as a team phenomenon: Mediating the relationship between learning climate and outcomes at the individual and team levels. The Journal of Positive Psychology, 11, 303–314.
Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539–1558.
*Hirst, G., Budhwar, P., Cooper, B. K., West, M., Long, C., Chongyuan, X., & Shipton, H. (2008). Cross-cultural variations in climate for autonomy, stress and organizational productivity relationships: A comparison of Chinese and UK manufacturing organizations. Journal of International Business Studies, 39, 1343–1358.
*Hofmann, D. A., & Mark, B. (2006). An investigation of the relationship between safety climate and medication errors as well as other nurse and patient outcomes. Personnel Psychology, 59, 847–869.
*Hofmann, D. A., & Stetzer, A. (1996). A cross-level investigation of factors influencing unsafe behaviors and accidents. Personnel Psychology, 49, 307–339.
Hofstede, G. (2001). Culture’s consequences: Comparing values, behaviors, institutions and organizations across nations (2nd ed.). Sage.
Hofstede, G., Hofstede, G. J., & Minkov, M. (2010). Cultures and organizations (3rd ed.). McGraw-Hill.
Hothorn, T., Bretz, F., & Westfall, P. (2008). Simultaneous inference in general parametric models. Biometrical Journal, 50(3), 346–363.
*Huang, C.-C., & Hsieh, P.-N. (2017). Inspiring creativity in teams: Perspectives of transactive memory systems. Journal of Pacific Rim Psychology, 11, 1–14.
*Huang, Y.-H., Verma, S. K., Chang, W.-R., Courtney, T. K., Lombardi, D. A., Brennan, M. J., & Perry, M. J. (2012). Supervisor vs. employee safety perceptions and association with future injury in US limited-service restaurant workers. Accident Analysis & Prevention, 47, 45–51.
Huang, X., & van de Vliert, E. (2004). Job level and national culture as joint roots of job satisfaction. Applied Psychology: An International Review, 53, 329–348.
Huedo-Medina, T. B., Sanchez-Meca, J., Marın-Martınez, F., & Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q Statistic or I2 Index? Psychological Methods, 11(2), 193–206.
*Hui, C. H., Chiu, W. C. K., Yu, P. L. H., Cheng, K., & Tse, H. (2007). The effects of service climate and the effective leadership behaviour of supervisors on frontline employee service quality: A multi-level analysis. Journal of Occupational & Organizational Psychology, 80, 151–172.
*Huo, W., Cai, Z., Luo, J., Men, C., & Jia, R. (2016). Antecedents and intervention mechanisms: A multi-level study of R&D team’s knowledge hiding behavior. Journal of Knowledge Management, 20, 880–897.
*Idris, M. A., Dollard, M. F., Coward, J., & Dormann, C. (2012). Psychosocial safety climate: Conceptual distinctiveness and effect on job demands and worker psychological health. Safety Science, 50, 19–28.
James, L. R., Choi, C. C., Ko, C.-H.E., McNeil, P. K., Minton, M. K., Wright, M. A., & Kim, K.-i. (2008). Organizational and psychological climate: A review of theory and research. European Journal of Work and Organizational Psychology, 17, 5–32.
James, D. L., Demaree, R. G., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85–98.
James, L. A., & James, L. R. (1989). Integrating work environment perceptions: Exploration into the measurement of meaning. Journal of Applied Psychology, 74, 739–751.
Jeanneret, P.R., Borman, W.C., Kubisiak, U.C., & Hanson, M.A. (1999). Generalized work activities. In N.G. Peterson, M.D. Mumford, W.C. Borman, P.R. Jeanneret, and E.A. Fleishman (Eds.), An occupational information system for the 21st century: The development of O*NET, 105–125. Washington, D.C.: American Psychological Association.
*Jiang, K., Chuang, C., & Chiao, Y. (2015). Developing collective customer knowledge and service climate: The interaction between service-oriented high-performance work systems and service leadership. Journal of Applied Psychology, 100, 1089–1106.
*Jiang, W., & Gu, Q. (2016). How abusive supervision and abusive supervisory climate influence salesperson creativity and sales team effectiveness in China. Management Decision, 54, 455–475.
*Johnson, A., Nguyen, H., Groth, M., & White, L. (2018). Reaping the rewards of functional diversity in healthcare teams: Why team processes improve performance. Group & Organization Management, 43, 440–474.
*Johnson, J. W. (1996). Linking employee perceptions of service climate to customer satisfaction. Personnel Psychology, 49, 831–851.
*Jones, G.D. (2017). Modeling effective work groups and teams: An assessment of the inclusion of social value orientation. Unpublished doctoral dissertation, University of Louisville: Louisville, KY. Retrieved from https://ir.library.louisville.edu/etd/2877
*Kao, R.-H. (2017). Task-oriented work characteristics, self-efficacy, and service-oriented organizational citizenship behavior: A cross-level analysis of the moderating effect of social work characteristics and collective efficacy. Personnel Review, 46, 718–739.
*Kao, F.-H., & Cheng, B.-S. (2017). Proservice or antiservice employee behaviors: A multilevel ethics perspective. Human Performance, 30, 272–290.
*Katz-Navon, T., Naveh, E., & Stern, Z. (2005). Safety climate in healthcare organizations: A multidimensional approach. Academy of Management Journal, 48, 1075–1089.
*King, E. B., de Chermont, K., West, M., Dawson, J. F., & Hebl, M. (2007). How innovation can alleviate negative consequences of demanding work contexts: The influence of climate for innovation on organizational outcomes. Journal of Occupational and Organizational Psychology, 80, 631–645.
Klein, K. J., Conn, A. B., Smith, D. B., & Sorra, J. S. (2001). Is everyone in agreement? An exploration of within-group agreement in employee perceptions of the work environment. Journal of Applied Psychology, 86, 3–16.
*Koene, B., Vogelaar, A., & Soeters, J. L. (2002). Leadership effects on organizational climate and financial performance: Local leadership effect in chain organizations. The Leadership Quarterly, 13, 193–215.
*Koopmann, J., Lanaj, K., Wang, M., Zhou, L., & Shi, J. (2016). Nonlinear effects of team tenure on team psychological safety climate and climate strength: Implications for average team member performance. Journal of Applied Psychology, 101, 940–957.
Kopelman, R. E., Brief, A. P., & Guzzo, R. A. (1990). The role of climate and culture in productivity. In B. Schneider (Ed.), Organizational climate and culture (pp. 282–318). Jossey-Bass.
*Kozlowski, S. W. J., & Hults, B. M. (1987). An exploration of climates for technical updating and performance. Personnel Psychology, 40, 539–563.
Krackhardt, D., & Kilduff, M. (1990). Friendship patterns and culture: The control of organizational diversity. American Anthropologist, 92, 142–154.
LeBreton, J. M., James, L. R., & Lindell, M. K. (2005). Recent issues regarding rWG, r*WG rWG(J), and r*WG(J). Organizational Research Methods, 8, 128–137.
LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815–852.
*Lee, J. (2014). Moderation effect of safety climate variability on the relationship between safety climate level and safety behavior and its boundary conditions. Doctoral Dissertations. 504. https://opencommons.uconn.edu/dissertations/504
*Lee, L.-Y., & Vaesna, S. (2013). The effects of social exchange perspective on employee creativity: A multilevel investigation. Psychology Research, 3, 660–678.
Li, X., Dusseldorp, E., Liu, K., Claramunt, J., & Meulman, J. (2019). Meta-CART: A flexible approach to identify moderators in meta-analysis. R package version 2.0–1. https://CRAN.R-project.org/package=metacart
Li, X., Dusseldorp, E., & Meulman, J. J. (2017). Meta-CART: A tool to identify interactions between moderators in meta-analysis. British Journal of Mathematical and Statistical Psychology, 70(1), 118–136.
*Liao, H., & Chuang, A. (2004). A multilevel investigation of factors influencing employee service performance and customer outcomes. Academy of Management Journal, 47, 41–58.
*Lin, C-P., Liu, C-M., Liu, N-T., & Huang, H-T. (2018). Being excellent teams: Managing innovative climate, politics, and team performance. Total Quality Management & Business Excellence.
Lindell, M. K., & Brandt, C. J. (2000). Climate quality and climate consensus as mediators of the relationship between organizational antecedents and outcomes. Journal of Applied Psychology, 85, 331–348.
*Ling, Q., Liu, F., & Wu, X. (2017). Servant versus authentic leadership: Assessing effectiveness in China’s hospitality industry. Cornell Hospitality Quarterly, 58, 53–68.
*Linuesa-Langreo, J., Ruiz-Palomino, P., & Elche-Hortelano, D. (2017). New strategies in the new millennium: Servant leadership as enhancer of service climate and customer service performance. Frontiers in Psychology, 8,. https://doi.org/10.3389/fpsyg.2017.00786
*Loh, M.Y., Idris, M.A., Dollard, M.F., & Isahak, M. (2018). Psychosocial safety climate as a moderator of the moderators: Contextualizing JDR models and emotional demands effects. Journal of Occupational and Organizational Psychology.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Addison-Wesley Publishing Company Inc.
*Luria, G. (2008). Climate strength: How leaders from consensus. The Leadership Quarterly, 19, 42–53.
*Luria, G., & Yagil, D. (2008). Procedural justice, ethical climate and service outcomes in restaurants. International Journal of Hospitality Management, 27, 276–283.
*Mascherek, A. C., & Schwappach, D. L. B. (2017). Patient safety climate profiles across time: Strength and level of safety climate associated with a quality improvement program in Switzerland – A cross-sectional survey study. PLoS ONE, 12(7), e0181410. https://doi.org/10.1371/journal.pone.0181410
*Maynard, M. T., Mathieu, J. E., Marsh, W. M., & Ruddy, T. R. (2007). A multilevel investigation of the influences of employees’ resistance to empowerment. Human Performance, 20, 147–171.
*McKay, P. F., Avery, D. R., & Morris, M. A. (2009). A tale of two climates: Diversity climate from subordinates’ and managers’ perspectives and their role in store unit sales performance. Personnel Psychology, 62, 767–791.
*Menges, J. I., Walter, F., Vogel, B., & Bruch, H. (2011). Transformational leadership climate: Performance linkages, mechanisms, and boundary conditions at the organizational level. The Leadership Quarterly, 22, 893–909.
*Mohammadi, B. (2016). Empowerment climate, supervisory support for creativity and work role performance in new product development settings: A team level investigation. Unpublished masters thesis, Queensland University of Technology: Brisbane, Australia.
*Molina, A., Moliner, C., Martínez-Tur, V., Cropanzano, R., & Peiró, J. M. (2016). Validating justice climate and peer justice in a real work setting. Journal of Work and Organizational Psychology, 32, 191–205.
*Moon, K. (2016). Determinants and consequences of organizational justice climate in the U.S. federal government: An organizational level, longitudinal study. Unpublished doctoral dissertation, University of Georgia: Athens, GA.
*Morrison, E. W., Wheeler-Smith, S. L., & Kamdar, D. (2011). Speaking up in groups: A cross-level study of group voice climate and voice. Journal of Applied Psychology, 96, 183–191.
Mumford, M. D., & Peterson, N. G. (1999). The O*NET content model: Structural considerations in describing jobs. In N. G. Peterson, M. D. Mumford, W. C. Borman, P. R. Jeanneret, & E. A. Fleishman (Eds.), An occupational information system for the 21st century: The development of O*NET (pp. 21–30). American Psychological Association.
*Myer, A. T., Thoroughgood, C. N., & Mohammed, S. (2016). Complementary or competing climates? Examining the interactive effect of service and ethical climates on company-level financial Performance. Journal of Applied Psychology, 101, 1178–1190.
*Naumann, S., & Bennett, N. (2002). The effects of procedural justice climate on work group performance. Small Group Research, 33, 361–377.
*Naveh, E., & Katz-Navon, T. (2015). A longitudinal study of an intervention to improve road safety climate: Climate as an organizational boundary spanner. Journal of Applied Psychology, 100, 216–226.
*Naveh, E., Katz-Navon, T., & Stern, Z. (2005). Treatment errors in healthcare: A safety climate approach. Management Science, 51, 948–960.
*Neal, A., & Griffin, M. A. (2006). A study of the lagged relationships among safety climate motivation, safety behavior, and accidents at the individual and group levels. Journal of Applied Psychology, 91, 946–953.
*Neal, A., West, M. A., & Patterson, M. G. (2005). Do organizational climate and competitive strategy moderate the relationship between human resource management and productivity? Journal of Management, 31, 492–512.
Newman, D. A., & Sin, H. (2020). Within-group agreement (rWG): Two theoretical parameters and their estimators. Organizational Research Methods, 23, 30–64.
*Ning, N., Wang, C.J., Zhaohong, C., & Zheng, C.Z. (2017). The direct and moderating effect of learning orientation on individual performance in the banking industry in China: Contextualization of high-performance work systems. Asia Pacific Journal of Human Resources.
*Ostroff, C., Kinicki, A., & Clark, M. (2002). Substantive and operational issues of response bias across levels of analysis: An example of climate-satisfaction relationships. Journal of Applied Psychology, 87, 355–368.
*Parker, A. W., Tones, M. J., & Ritchie, G. E. (2017). Development of a multilevel health and safety climate survey tool within a mining setting. Journal of Safety Research, 62, 173–180.
Parr, N.J., Loan, C.M., & Tanner-Smith, E.E. (2021). Using machine learning to identify and investigate moderators of alcohol use intervention effects in meta-analyses. Alcohol and Alcoholism. agab036, https://doi.org/10.1093/alcalc/agab036
*Patras, L., Martínez-Tura, V., Estredera, Y., Gracia, E., Molinera, C., & Peiró, J. M. (2018). Organizational performance focused on users’ quality of life: The role of service climate and “contribution-to-others” wellbeing beliefs. Research in Developmental Disabilities, 77, 114–123.
Patterson, M. G., West, M. A., Shackleton, V. J., Dawson, J. F., Lawthom, R., Maitlis, S., Robinson, D. L., & Wallace, A. M. (2005). Validating the organizational climate measure: Links to managerial practices, productivity and innovation. Journal of Organizational Behavior, 26, 379–408.
*Pecino-Medina, V., Díaz-Fúnez, P. A., & Mañas-Rodríguez, M. A. (2017). Climate, stress and satisfaction: A multilevel study in the public sector. International Journal of Social Psychology, 32, 52–79.
*Peng, J. C., Jien, J.-J., & Lin, J. (2016). Antecedents and consequences of psychological contract breach. Journal of Managerial Psychology, 31, 1312–1326.
*Prentice, C., Ma, E., & Wong, I.A. (2018). Performance driven outcomes—the case of frontline employees in the hospitality sector. Journal of Hospitality Marketing & Management.
*Raes, E., Decuyper, S., Lismont, B., Van den Bossche, P., Kyndt, E., Demeyere, S., & Dochy, F. (2013). Facilitating team learning through transformational leadership. Instructional Science, 41, 287–305.
Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. Routledge.
Rentsch, J. R. (1990). Climate and culture: Interaction and qualitative differences in organizational meanings. Journal of Applied Psychology, 75, 668–681.
*Riordan, C. M., Vandenberg, R. J., & Richardson, H. A. (2005). Employee involvement climate and organizational effectiveness. Human Resource Management, 44, 471–488.
Roberson, Q. M. (2006). Justice in teams: The effects of interdependence and identification on referent choice and justice climate strength. Social Justice Research, 19, 323–344.
Ryan, A. M., McFarland, L., Baron, H., & Page, R. (1999). An international look at selection practices: Nation and culture as explanations for variability in practice. Personnel Psychology, 52, 359–391.
*Salanova, M., Agut, S., & Peiro, J. M. (2005). Linking organizational resources and work engagement to employee performance and customer loyalty: The mediation of service climate. Journal of Applied Psychology, 90, 1217–1227.
*Sandell, K.J. (2015). Climate for workplace fun in a retail setting. Unpublished doctoral dissertation, Colorado State University, Fort Collins, CO.
*Schechter, C., & Qadach, M. (2016). Principals’ learning mechanisms: Exploring an emerging construct, leadership and policy in schools. Leadership and Policy in Schools, 15, 141–167.
*Schneider, B., & Barbera, K. M. (2014). Introduction: The Oxford handbook of organizational climate and culture. In B. Schneider & K. M. Barbera (Eds.), The Oxford handbook of organizational climate and culture (pp. 3–20). Oxford University Press.
Schneider, B., Ehrhart, M. G., Mayer, D. M., Saltz, J. L., & Niles-Jolly, K. (2005). Understanding organization-customer links in service settings. Academy of Management Journal, 48, 1017–1032.
Schneider, B., Gonzalez-Roma, V., Ostroff, C., & West, M. A. (2017). Organizational climate and culture: Reflections on the history of the constructs in the Journal of Applied Psychology. Journal of Applied Psychology, 102, 468–482.
Schneider, B., Gunnarson, S. K., & Niles-Jolly, K. (1994). Creating the climate and culture of success. Organizational Dynamics, 23, 17–29.
Schneider, B., & Reichers, A. E. (1983). On the etiology of climates. Personnel Psychology, 36, 19–39.
*Schneider, B., White, S. S., & Paul, M. C. (1998). Linking service climate and customer perceptions of service quality: Test of a causal model. Journal of Applied Psychology, 83, 150–163.
*Schulte, M., Ostroff, C., Shmulyian, S., & Kinicki, A. (2009). Organizational climate configurations: Relationships to collective attitudes, customer satisfaction, and financial performance. Journal of Applied Psychology, 3, 618–634.
*Shen, J., & Benson, J. (2016). When CSR as a social norm: How socially responsible human resource management affects employee work behavior. Journal of Management, 42, 1723–1746.
*Shen,Y., Zhang, R.P., Koh, Y., Rowlinson, S., & Leicht, R.M. (2017). The effects of group safety climate on construction personnel’s safety behavior: A cross-level investigation. In A. Mahalingam, T. Shealy, & N. Gil (Eds.), Working Paper Proceedings of the 15th Engineering Project Organization Conference, 1–11. California: Stanford Sierra Camp.
*Shepherd, W. J., Ployhart, R. E., & Kautz, J. (2020). The neglected role of collective customer perceptions in shaping collective employee satisfaction, service climate, voluntary turnover, and involuntary turnover: A cautionary note. Journal of Applied Psychology, 105, 1327–1337.
*Shih, H.-A., Chiang, Y.-H., & Chen, T.-J. (2012). Transformational leadership, trusting climate, and knowledge-exchange behaviors in Taiwan. The International Journal of Human Resource Management, 23, 1057–1073.
Simon, H. A. (1973). The organization of complex systems. In H. H. Pattee (Ed.), Hierarchy theory (pp. 1–27). Braziller.
*Simons, T., & Roberson, Q. (2003). Why managers should care about fairness: The effects of aggregate justice perceptions on organizational outcomes. Journal of Applied Psychology, 88, 432–443.
Smith-Crowe, K., Burke, M. J., Kouchaki, M., Signal, S. M. (2013) Assessing interrater agreement via the average deviation index given a variety of theoretical and methodological problems. Organizational Research Methods, 16(1), 127–151. https://doi.org/10.1177/1094428112465898
*Sowinski, D. R., Fortmann, K. A., & Lezotte, D. V. (2008). Climate for service and the moderating effects of climate strength on customer satisfaction, voluntary turnover, and profitability. European Journal of Work and Organizational Psychology, 17, 73–88.
*Spell, H. B., Eby, L. T., & Vandenberg, R. J. (2014). Developmental climate: A cross-level analysis of voluntary turnover and job performance. Journal of Vocational Behavior, 84, 283–292.
*Spielfogel, J. E., Leathers, S. J., & Christian, E. (2016). Agency culture and climate in child welfare: Do perceptions vary by exposure to the child welfare system? Human Service Organizations: Management, Leadership & Governance, 40, 382–396.
Strobl, C., Malley, J., & Tutz, G. (2009). An introduction to recursive partitioning: Rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychological Methods, 14, 323–348.
*Tang, P. M. (2016). Ethical leadership in social enterprises: Multilevel investigation of its influence on team and individual prosocial voice. Unpublished masters thesis, Lingnan University: Hong Kong. Retrieved from http://commons.ln.edu.hk/mgt_etd/27
*Tang, C., & Naumann, S. (2016). Team diversity, mood, and team creativity: The role of team knowledge sharing in Chinese R&D teams. Journal of Management & Organization, 22, 420–434.
Taras, V., Kirkman, B. L., & Steel, P. (2010). Examining the impact of culture‘s consequences: A three-decade, multi-level, meta-analytic review of Hofstede‘s cultural value dimensions. Journal of Applied Psychology, 95(3), 405–439.
*Tesler, R. (2013). Alignment is not just for tires: The impact of leader-follower climate perceptions on learning and safety behaviors. Unpublished doctoral dissertation, The Pennsylvania State University, State College, PA.
*Tews, M. J., Hoefnagels, A., Jolly, P. M., & Stafford, K. (2021). Turnover among young adults In the hospitality industry: Examining the impact of fun in the workplace and training climate. Employee Relations: The International Journal, 43, 245–261.
Therneau, T. M., & Atkinson, B. (2017). An introduction to recursive partitioning using the RPART routines. Mayo Foundation.
*Tremblay, M. (2017). Humor in teams: Multilevel relationships between humor climate, inclusion, trust, and citizenship behaviors. Journal of Business and Psychology, 32, 363–378.
*Tremblay, M., Gaudet, M. C., & Parent-Rocheleau, X. (2018). Good things are not eternal: How consideration leadership and initiating structure influence the dynamic nature of organizational justice and extra-role behaviors at the collective level. Journal of Leadership & Organizational Studies, 25, 211–232.
*Tse, H. H. M., Dasborough, M. T., & Ashkanasy, N. M. (2008). A multi-level analysis of team climate and interpersonal exchange relationships at work. The Leadership Quarterly, 19, 195–211.
Tung, R. L., & Baumann, C. (2009). Comparing the attitudes toward money, material possessions and savings of overseas Chinese vis-a`-vis Chinese in China: Convergence, divergence or cross-vergence, vis-a`-vis “one size fits all” human resource management policies and practices. International Journal of Human Resource Management, 20, 2382–2401.
Tung, R. L., & Verbeke, A. (2010). Beyond Hofstede and GLOBE: Improving the quality of cross-cultural research. Journal of International Business Studies, 41, 1259–1274.
*a Van De Voorde, K., Van Veldhoven, M., & Paauwe, J. (2010). Time precedence in the relationship between organizational climate and organizational performance: A cross-lagged study at the business unit level. The International Journal of Human Resource Management, 21, 1712–1732.
*a Van De Voorde, K., Van Veldhoven, M., & Paauwe, J. (2014). Relationships between work unit climate and labour productivity in the financial sector: A longitudinal test of the mediating role of work satisfaction. European Journal of Work and Organizational Psychology, 23(2), 295–309.
van Lissa, C. J. (2020). Small sample meta-analyses: Exploring heterogeneity using MetaForest. In R. van de Schoot & M. Miocevic (Eds.), Small sample size solutions: A guide for applied researchers and practitioners (pp. 186–202). Routledge.
*Van Woerkom, M., & Meyers, M. (2015). My strengths count! Effects of a strengths-based psychological climate on positive affect and job performance. Human Resource Management, 54, 81–103.
*Vera, M., Martınez, I. M., Lorente, L., & Chambel, M. A. (2016). The role of co-worker and supervisor support in the relationship between job autonomy and work engagement among Portuguese nurses: A multilevel study. Social Indicators Research, 12, 1143–1156.
Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48.
*Vogus, T. J., Cull, M. J., Hengelbrok, N. E., Modell, S. J., & Epstein, R. A. (2016). Assessing safety culture in child welfare: Evidence from Tennessee. Children and Youth Services Review, 65, 94–103.
*Wallace, C., & Chen, G. (2006). A multilevel integration of personality, climate, self-regulation, and performance. Personnel Psychology, 59, 529–557.
Wallace, J. C., Edwards, B. D., Paul, J., Burke, M. J., Christian, M., & Eissa, G. (2016). Change the referent? A meta-analytic investigation of direct and referent-shift consensus models for organizational climate. Journal of Management, 42, 838–861.
*Wallace, J. C., Johnson, P. D., Mathe, K., & Paul, J. (2011). Structural and psychological empowerment climates, performance, and the moderating role of shared felt accountability: A managerial perspective. Journal of Applied Psychology, 96, 840–850.
*Walumba, F. O., Peterson, S. J., Avolio, B. J., & Hartnell, C. A. (2010). An investigation of the relationships among leader and follower psychological capital, service climate, and job performance. Personnel Psychology, 63, 937–963.
*Wang, P., & Rode, J. C. (2010). Transformational leadership and follower creativity: The moderating effects of identification with leader and organizational climate. Human Relations, 63, 1105–1128.
*Wang, Z., Chen, Y.-f.N., Tjosvold, D., & Shi, K. (2010). Cooperative goals and team agreeableness composition for constructive controversy in China. Asia Pacific Journal of Management, 27, 139–153.
*Williams, N. J., Glisson, C., Hemmelgarn, A., & Green, P. (2017). Mechanisms of change in the ARC organizational strategy: Increasing mental health clinicians’ EBP adoption through improved organizational culture and capacity. Administration Policy in Mental Health, 44, 269–283.
*Winkler, R.B. (2004). Organizational climate in a health care setting and its relationship to patient safety. Unpublished dissertation, DePaul University, Chicago, IL.
*Wood, S., Stride, C., & Johnson, S. (2012). Personal and team morale in health services: Distinct and measurable concepts. Journal of Health Management, 14, 535–557.
*Woznyj, H.M., Heggestad, E.D., Kennerly, S., & Yap, T.L. (2018). Climate and organizational performance in long-term care facilities: The role of affective commitment. Journal of Occupational and Organizational Psychology.
Yohannes, Y., & Webb, P. (1999). Classification and regression trees, CART: A user manual for identifying indicators of vulnerability to famine and chronic food insecurity (No. 593–2016–39922). Washington, D.C.: International Food Policy Research Institute (IFPRI).
*Yu, T., de Ruyter, K., Patterson, P., & Chen, C.-F. (2018). The formation of a cross-selling initiative climate and its interplay with service climate. European Journal of Marketing, 52, 1457–1484.
*Yusof, N.A.M., Jusoh, W.J.W., & Hashim, K.S.H. (2020). Service climate and service behaviour to customer satisfaction: A multilevel study of Muslim-friendly hotel. Journal of Halal Industry & Services, 3 (Special Issue), 1–6.
*Zappala, S., Martinez-Tur, V., & Mariani, M. G. (2018). Service climate in organizations: Validating the Italian version of the service climate scale (ISCS). TPM, 25, 5–20.
*Zhang, Y., & Begley, T. M. (2011). Perceived organizational climate, knowledge transfer and innovation in China-based research and development companies. The International Journal of Human Resource Management, 22, 34–56.
*Zhang, X., & Kwan, H. K. (2018). Team learning goal orientation and innovation: Roles of transactive memory system and task interdependence. GSL Journal of Business Management and Administration Affairs, 1(109), 1–10.
*Zhang, N., & Zhang, J. (2016). Chinese insurance agents in “bad barrels”: A multilevel analysis of the relationship between ethical leadership, ethical climate and business ethical sensitivity. Springerplus, 5, 2078.
Zohar, D., & Tenne-Gazit, O. (2008). Transformational leadership and group interaction as climate antecedents: A social network analysis. Journal of Applied Psychology, 93, 744–757.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Table 4
Rights and permissions
About this article
Cite this article
Burke, M.J., Smith-Crowe, K., Burke, M.I. et al. The Relative Importance and Interaction of Contextual and Methodological Predictors of Mean rWG for Work Climate. J Bus Psychol 37, 923–951 (2022). https://doi.org/10.1007/s10869-021-09789-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10869-021-09789-6