Skip to main content

The Relative Importance and Interaction of Contextual and Methodological Predictors of Mean rWG for Work Climate

Abstract

A variety of collective phenomena are understood to exist to the extent that workers agree on their perceptions of the phenomena, such as perceptions of their organization’s climate or perceptions of their team’s mental model. Researchers conducting group-level studies of such phenomena measure individuals’ perceptions via surveys and then aggregate data to the group level if the mean within-group agreement for a sample of groups is sufficiently high. Despite this widespread practice, we know little about the factors potentially affecting mean within-group agreement. Here, focusing on work climate, we report an investigation of a number of expected contextual (social interaction) and methodological predictors of mean rWG, a common statistic for judging within-group agreement in applied psychology and management research. We used the novel approach of meta-CART, which allowed us to assess the relative importance and possible interactions of the predictor variables. Notably, mean rWG values are driven by both contextual (average number of individuals per group and cultural individualism-collectivism) and methodological factors (the number of items in a scale and scale reliability). Our findings are largely consistent with expectations concerning how social interaction affects within-group agreement and psychometric arguments regarding why adding more items to a scale will not necessarily increase the magnitude of an index based on a Spearman-Brown “stepped-up correction.” We discuss the key insights from our results, which are relevant to the study of multilevel phenomena relying on the aggregation of individual-level data and informative for how meta-analytic researchers can simultaneously examine multiple moderator variables.

Many collective phenomena hinge theoretically and, thus, empirically on whether people agree on their views of reality. Such phenomena include things like team mental models and organizational climate (see Biemann et al., 2014; Wallace et al., 2016), which are studied via multiple levels of analysis: individuals are surveyed, and if agreement among them is deemed sufficient, then the collective level phenomenon is confirmed (Chan, 1998). Most often, researchers quantify agreement for each group within a sample of groups via the within-group agreement index, rWG (James et al., 1984). Then, the mean agreement across groups (mean rWG) is compared to some practical cutoff value (LeBreton & Senter, 2008), and if the mean agreement is greater than the cutoff, the phenomenon is arguably understood to exist. Many researchers also employ other average, aggregation statistics such as mean average deviation (AD) indices and ICC statistics as supplemental indicators of within-group agreement and similarly interpret them relative to practical cutoffs for acceptable agreement (see LeBreton & Senter, 2008; Smith-Crowe et al., 2013).

Another common practice, as Burke et al. (2018) observed, is that in almost all studies where individual-level data are aggregated, the author does not eliminate any group based on a low or out-of-bound interrater agreement result, and the author decides to aggregate even when mean interrater agreement values are below rules-of-thumb cutoff values. This observation was notable, as there is a body of literature discussing why and how researchers could systematically delete aggregated data that is below practical cutoff values (see Bliese, 2000; Chan, 1998; LeBreton & Senter, 2008). In addition, as Burke et al. (2018) indicated, sampling of groups occurs in almost all group-level studies at both the individual and group levels. Given that a complex form of sampling occurs in group-level studies, we cannot “establish” or “determine” sharedness in beliefs/perceptions. That is, we can only “estimate” sharedness and that estimation includes measurement and sampling error.

To account for the complex form of sampling error in group-level studies, Burke et al. (2018) stressed the importance of estimating mean within-group agreement (for rWG as well as for other aggregation statistics) and constructing confidence intervals for mean within-group agreement. As those authors discussed, knowledge of mean within-group agreement and the confidence interval for mean within-group agreement can be useful for not only informing data aggregation decisions and understanding agreement in the population of groups, but this knowledge can also assist in efforts to cumulatively integrate findings from similar group-level studies.

Despite the emphasis placed on mean rWG for data aggregation decisions and the potential advantages associated with knowledge of mean rWG and its confidence interval for better understanding group-level phenomena, we know very little about how mean rWG is affected by contextual factors that promote social interaction or methodological factors. For instance, the extant methodological literature would suggest that adding members to groups would positively affect within-group agreement values and hence mean rWG (Brown & Haunstein, 2005). Yet, the social interaction literature would suggest the opposite effect in that increasing group size would be expected to lessen social relations and, thus, limit the development of shared perception of the work environment. In the latter case, group rWG values and mean rWG for a sample of groups would be expected to decrease as more members are added to each group.

Obtaining an understanding of how methodological relative to contextual factors predict mean rWG will provide researchers and practitioners with knowledge of when and where they should expect certain levels of mean rWG. This information is important for several reasons. For both researchers and practitioners, this knowledge could be especially useful in the design of studies involving data aggregation and in the development of measures to better assess individual-level phenomena, where such measures will serve as indicators of emergent constructs. In addition, for primary and meta-analytic researchers, this knowledge could provide insights as to how different levels of mean within-group agreement are determined and why they might expect lower versus higher relationships between aggregated variables. Finally, for practitioners, such knowledge could be beneficial in directing their efforts to promote workers’ shared perceptions of the work environment. The later point would be particularly relevant in contexts where they desire to develop strong organizational climates.

The aim of this study is to examine how contextual characteristics of group-level studies, such as national culture and the nature of work, relative to methodological considerations, such as measurement characteristics, predict average levels of within-group agreement across samples of groups, as indexed by mean rWG. In doing so, we use a new approach that capitalizes on advances in machine learning, meta-analysis combined with classification and regression trees (meta-CART; Li et al., 2017; Li, Dusseldorp, Liu, Claramunt, & Meulman, 2019), to identify the most important contextual and methodological study-level predictors of average levels of within-group agreement. In addition, we examine whether any of these two types of predictors interact in the prediction of average levels of within-group agreement. In studying predictors of mean within-group agreement, we note that our research is not necessarily intended to inform when within-group agreement will be weaker or stronger for a specific group.

We chose work climate as the domain of study as it arguably has the largest number of investigations in applied psychology and management where researchers have aggregated individual-level data to the group level of analysis (see Burke, Landis, & Burke, 2017; Schneider & Barbera, 2014; Wallace et al., 2016). As such, our investigation focuses on clarifying the meaning of average levels of within-group agreement for samples of groups in relation to only work climate. That said, we hope our application of meta-CART serves as an example for how applied psychology and management researchers might use this powerful methodological and statistical procedure to explore relationships between variables at the study level in other domains of inquiry.

Prior to discussing unanswered questions in the literature concerning within-group agreement and different types of contextual and methodological predictors of mean within-group agreement, we refer the reader to the Appendix, Table 4. In the Appendix, Table 4, we include information on several potential contextual and methodological predictors of mean within-group agreement as well as information on mean rWG values across studies. Notably, information in the Appendix, Table 4, illustrates that researchers have employed rWG(J) (producing notable across-study variability in mean rWG values) in a wide range of contexts and with respect to considerable variations in methodology/measurement in the domain of work climate. As shown in the Appendix, Table 4, in few studies were either the variance or range of rWG(J) values reported and considered in data aggregation decisions. These variations in mean within-group agreement with respect to the different types of contextual and methodological considerations coupled with the lack of attention to within-study variation in rWG(J) values point further to our arguments concerning the need to better understand when and where we should expect different levels of mean within-group agreement. While our study concerns predictors of mean rWG, this focus on rWG should not be interpreted as a pretense for suggesting that rWG should be used as the sole statistic for justifying data aggregation nor does our focus speak to data aggregation per se.

Unanswered Questions in the Extant Literature Concerning Within-Group Agreement

In the methodological literature, authors have argued that the magnitude of rWG and, consequently, mean rWG is a function of number of items for a scale, the number of group members, the null response distribution that a researcher chooses, and the mean of item variances for a scale (see Brown & Hauenstein, 2005; Burke et al., 2018; LeBreton & Senter, 2008; Newman & Sin, 2020). Notably, several authors have presented strong methodological arguments in relation to what can produce lower versus higher values of rWG. For instance, Newman and Sin (2020, p. 37) in discussing rWG for a scale concluded by emphasizing (and italicizing) the following statement “As such, large rWG(J) values can always be obtained by simply adding more items to the scale, regardless of the amount of true within-group agreement between individuals (ψWG).” In addition, as noted above, other authors have presented strong, albeit qualified, arguments and conclusions in relation to how adding group members affects rWG. For instance, Brown and Hauenstein (2005, p. 167) presented a sound psychometric and statistical argument for how observed item (or scale) variances, and consequently, rWG is influenced by sample size. Given the assumption that ratings were comparable across small in comparison to large samples, those authors concluded that agreement will likely be judged as greater for the larger samples, independent of the true level of consensus. As we will discuss below, these arguments and positions are based on particular psychometric assumptions, which may not hold with actual data.

On the other hand and from a social interaction perspective, authors have argued that shared perceptions of the work environment or within-group agreement emerge from social exchanges (Schneider & Reichers, 1983). Authors within this body of literature have examined how different forms of social interaction and factors that promote social interaction (e.g., group size, proximity of workers to each other, the degree of work interdependence) affect the strength of workers’ within-group perceptions. Here, authors sometimes employ rWG as indicator of the strength of workers’ within-group perceptions (e.g., Lindell & Brandt, 2000; Luria, 2008). Notably, the literature on social interaction will sometimes lead to opposing predictions, in comparison to methodological arguments, as to what leads to lower versus higher levels of within-group agreement. For instance, Colquitt et al. (2002) argued that increases in group size will lead to decreases in social interaction and, consequently, lower degrees of workers’ shared perceptions of the work environment.

Furthermore, research from a social interaction or social constructionist perspective has not examined how certain contextual factors that may operate at the study level such as occupational factors that promote social interaction or cultural values toward collectivistic action affect mean within-group agreement. Moreover, we have no knowledge of the relative importance of contextual variables (factors expected to promote social interaction) relative to methodological factors in terms of their influence on mean levels of within-group agreement. Related, we have no understanding of how methodological and contextual factors might interact in determining lower versus higher levels of mean within-group agreement.

These shortcomings lead to a number of questions that no prior theories can readily answer but are important for understanding within-group agreement and especially mean within-group agreement as indexed by rWG. First, are reductions in average group size associated with lower or higher levels of mean within-group agreement? As noted, the methodological perspective would suggest that values would be higher when the number of judges or workgroup members is greater, whereas the social interaction perspective would suggest that such values would be lower, given fewer opportunities for social interaction in larger groups. Second, do researchers observe higher levels of mean within-group agreement in studies conducted within nations/cultures that prefer tightly knit associations (i.e., collectivism) relative to loosely knit social frameworks (i.e., individualism)? Third, are average levels of within-group agreement higher in studies where the sampled job/occupation requires greater work interdependence and, consequently, higher levels of social interaction? Alternatively, per methodological arguments, does the number of items on a scale simply override factors that promote social interaction in the determination of lower versus higher levels of mean within-group agreement? Answers to these types of questions will assist us in further understanding where and why researchers can expect higher levels of mean within-group agreement and stronger justifications for data aggregation with respect to contextual (social interaction) versus methodological considerations. In addition, answers to these types of questions may also point to why we observe between-study differences in relationships between group-level phenomena.

Study-Level Predictors of Mean Within-Group Agreement

Here, we discuss both expected and potential predictors of mean within-group agreement that we investigated. In every case, we provide a rationale for including the predictor in our study and, in some cases, evidence that the predictor works to align individuals’ perceptions at the group level. However, in practice, the basis of data aggregation is almost always at the study level or mean agreement across groups. Importantly, this investigation is the first investigation concerning whether these predictors operate at the study level. Further, by combining multiple predictors in the same investigation, we can assess the relative importance of contextual and methodological predictors and possible interactions among them. To add to this point, while we present conceptual arguments for expected directional relationships between many of the potential predictors of mean within-group agreement, we do not present formal hypotheses for the multitude of expected main effects. Rather, our aim is to examine the relative importance and possible interactions among the predictors to understand how our set of moderators interact and to illustrate how meta-analytic researchers can use meta-CART procedures to simultaneously examine large sets of potential moderator variables. That said, we do present bivariate relationships between all study variables in the “Results” section.

Expected Contextual Predictors

As noted above, shared perceptions of the work environment are expected to emerge from social interaction. Empirical support for the relationship between social interaction and the degree to which workers share perceptions of work environment phenomena has come from a number of studies where social interaction was measured via ratings of the frequency of social interaction (e.g., Gonzalez-Roma et al., 2002) and social network analysis (e.g., Zohar & Tenne-Gazit, 2008). Notably, researchers have conducted many of these studies with respect to examining antecedents of climate strength. Here, we point out that this literature is relevant to rWG, as rWG itself is often employed as a measure or indicator of climate strength (e.g., see Bogaert et al., 2012; Ginsburg and Oore, 2016; Lindell & Brandt, 2000; Luria, 2008; Mascherek & Schwappach, 2017).

Based on the rationale that work interdependence fosters social interaction (Krackhardt & Kilduff, 1990; Rentsch, 1990), a few studies have examined the relationship between work interdependence and climate strength (Klein et al., 2001; Roberson, 2006). In these field and laboratory studies, researchers have found work interdependence to correlate positively with climate strength. While several studies examined aggregated perceptions as indicators of work interdependence, or required teamwork, across large numbers of work contexts and samples, they did not examine the degree to which teamwork was an occupational requirement. An assessment of the extent to which teamwork is an occupational requirement may add to our understanding of study-level factors that underlie the degree to which workers, on average, share perceptions of the work environment (i.e., with respect to mean within-group agreement).

In considering work interdependence as a study-level predictor of within-group agreement, we note that many investigations where data are aggregated include workers from one or just a few occupations (e.g., nurse, commercial driver, waiter, or waitress). As a result, within any one group-level investigation, it is likely that work interdependence, as an occupational requirement, would be similar across groups. Given this assumption, we examined the extent to which work interdependence as a general, occupational requirement positively predicts within-group agreement at the study level. In this regard, we note that occupational requirements concerning work interdependence and teamwork are defined within the extensive Occupational Information Network (O*NET) system with respect to work activities, work styles, and work values (Mumford & Peterson, 1999). As described in more detail below, the O*NET provides a standardized means for scoring and examining these aspects of work interdependence as potential study-level predictors of within-group agreement.

Related, as group size increases, the proximity of group members is expected to decrease with fewer opportunities to interact (Colquitt et al., 2002). Notably, these points concerning group sizes have been discussed for some time in relation to the unit of analysis or level within an organization (e.g., team, department, business unit) and how physically dispersed individuals are from one another (e.g., see Hare, 1981; Simon, 1973). That is, as the level of analysis moves from the team to business unit, group size and physical dispersion increase with fewer opportunities for social interaction. However, we are unaware of any extension of these arguments to how average group size or level of analysis affects mean within-group agreement. Importantly, our investigation permits a cross-study examination of the extent to which average group size and level of analysis independently affect mean within-group agreement.

In addition, arguments have been presented in the literature that national/cultural values influence work practices and work attitudes (e.g., see Burke et al., 2008; Huang & van de Vliert, 2004; Kopelman et al., 1990; Ryan et al., 1999). Often, these investigations examine expectations concerning how national/cultural values moderate relationships between work attitudes. Our interest, however, is in whether a cultural value expected to underlie close social interactions within groups, individualism-collectivism, has any effect on within-group agreement at the study level.

The basis for considering individualism-collectivism is that the low end of this dimension, collectivism, represents a preference for a tightly knit framework in society in which individuals can expect members of a particular in-group to look after them in exchange for unquestioning loyalty. Hence, higher levels of collectivism would be expected to promote greater uniformity in work attitudes. Importantly, such a positive effect may be evident at the study level (i.e., with respect to mean within-group agreement), where countries and, thus, individualism-collectivism scores are known to vary.

Finally, the nature of the climate assessment might also produce lower versus higher levels of mean within-group agreement. General assessments of climate or concern for employees are largely premised on conceptual arguments that personal values (e.g., clarity, support, fairness) undergird perceptions of climate relative to particular climate dimensions (e.g., goal emphasis, management support, reward orientation) (see James & James, 1989). As such, individual differences in and sensitivity to personal values could produce considerable within-group variability in climate perceptions and lower mean values of rWG at the study level. James and James (1989) and James et al. (2008) have presented detailed discussions on this matter. On the other hand, assessments of a climate-for-something such as a climate for safety, service, or innovation are premised on the conceptual argument that organizationally espoused values (with respect to strategic foci such as safety, service, or innovation) primarily underlie workers’ climate perceptions (see James et al., 2008; Schneider et al., 1994). Assuming that organizationally espoused values are enacted, the possibility exists that there would be, on average, greater variation in within-group climate assessments for general climate (and lower rWG values) relative to assessments of climate-for-something.

Expected Methodological Predictors

Regarding how the measurement of workers’ perceptions might relate to mean within-group agreement, we turn to a discussion of several factors known to affect the reliability of attitudinal measures. Assuming they are parallel in a classical test theory sense (see Raykov & Marcoulides, 2011), as the number of items for a survey measure increases, the reliability of a measure will increase, resulting in a more consistent assessment of the phenomenon. Likewise, because rWG has a computational relationship or similarity to the Spearman-Brown “stepped-up correction” in reliability estimation (see LeBreton et al., 2005), one can see just as more items will increase the reliability of a measure, adding more items would be expected to increase rWG values for groups. For both the estimation of reliability and within-group agreement, this expectation is premised on the assumption that items have equal means, variances, and covariances. In practice, this broad assumption likely does not hold, as nonequivalence factors increase as the number of items becomes large or when researchers carelessly add items to a measure (Allen & Yen, 1979; Lord & Novick, 1968). Again, this result would be due to the generation of items that are nonequivalent (i.e., have unequal variances).

Another measurement consideration is the number of response options for survey items. In the climate literature, for instance, the number of response options typically ranges from 4 to 9, where climate items with a small number of response options such as 4 may fail to discriminate between respondents with different climate judgments as effectively as those with more options. In turn, the reliability of a climate measure and the validity of the climate assessment could be adversely affected. As Bandalos and Enders (1996) and Enders and Bandalos (1999) demonstrated, the reliability of a measure of attitudes generally increases as the number of response options goes from 3 to 9. This conclusion from Bandalos and Enders’ work is important as 4- and 5-point response scales are common for climate items (e.g., see Ambrose et al., 2013; Binci, 2011; Hirst et al., 2008), a methodological choice that may be constraining reliabilities in this literature.

While the above measurement considerations are suggestive of how the number of response options, number of items, or scale reliability generally might relate to average levels of within-group agreement, no clear predictions can be made. Thus, we included the number of response options, number of items, and scale reliability among the set of potential predictors of mean within-group agreement.

Another methodological consideration is the data aggregation model, direct consensus vs. referent shift (Chan, 1998). Wallace et al. (2016) argued that a direct consensus climate assessment (focusing on “I”) relative to a referent shift climate assessment (focusing on “We”) is more affectively based and, thus, more idiosyncratic. This argument implies that greater variability in individuals’ attitudes introduced by more affectively based measures could adversely affect mean within-group agreement. As such, we included the model for data aggregation among the set of potential study-level predictors of within-group agreement.

In addition, given that the magnitude of an rWG(J) statistic and its mean are known to be affected by the chosen null or random response distribution, we also considered the type of null response distribution as a potential predictor of mean within-group agreement. Also, since authors rely on different measures of central tendency (mean, median, or midpoint of range) when justifying data aggregation relative to a practical cutoff value, we examined the measure of central tendency as a potential predictor variable. Further, while not technically a methodological factor, we examined publication status (published versus unpublished study) as a predictor of mean within-group agreement given the possibility that lower levels of rWG may be more prevalent in unpublished works due to the “file-drawer problem.”

Finally, we also viewed average group size as a methodological factor, given psychometric and statistical arguments that adding group members will increase rWG(J) (see Brown & Haunstein, 2005; LeBreton & Senter, 2008). This expectation in regard to mean rWG is also based on the assumption that items are essentially parallel (i.e., have equal item variances).

Although there is a reasonable basis for examining the above, broad set of potential study-level predictors of mean within-group agreement, we do not have a basis for suggesting which of the study-level predictors are the most important ones or how the study-level predictor variables will interact. A key advantage of our methodological approach, as described in detail below, is that meta-CART will identify the most important predictors of mean within-group agreement as well as how the best set of predictors interact. As such, our approach has the means to overcome one of the major disadvantages of traditional approaches to integrating results from similar studies (i.e., meta-analyses and meta-regressions), the difficulty of identifying interactions among a somewhat large set of predictor (moderator) variables. The reader is referred to Li et al., (2017, 2019) for detailed discussions of the advantages of a CART (tree-based) approach relative to traditional meta-regression and subgroup analyses for identifying moderators in meta-analyses.

Method

Search and General Inclusion Criteria

Our search attempted to identify all published manuscripts, as well as publicly available dissertations and masters theses about work climate, where employees’ perceptions of climate were aggregated to the group or business unit level and where the study reported a central tendency measure of rWG(J), which is the rWG formula for a scale versus a single item. Keywords for the literature searches included combinations of the following terms: work-group climate, group-level climate, organizational climate, rWG, and James et al. (1984). Searches were conducted on databases including PsycINFO, PsycARTICLES, and Google Scholar. In addition, given that this study is part of a larger investigation, we examined other searches that included additional terms such as workgroup performance and organizational performance. Notably, these other searches identified additional studies for review and inclusion. We also conducted manual searches of journals and review articles and chapters focused on work climate and climate strength (e.g., Academy of Management Journal, Journal of Applied Psychology). To be retained, a study needed to have minimally reported a central tendency measure of rWG(J) for one or more climate measures. Additionally, a study needed to report information that would allow for the identification of a general occupation for the sample of groups in a study.

Our searches identified over 2,500 potentially relevant articles, dissertations, and masters theses. The abstracts and titles of these studies were reviewed for possible inclusion, with over 800 studies being identified for further review. Subsequently, for 187 studies that appeared to meet most of our key inclusion criteria, one author coded the study and a second author checked that coding for accuracy. Any discrepancies in coding between the coders were discussed to reach a consensus. Of the 187 studies, 141 samples from 137 studies were included in the final analyses. Specific coding considerations and reasons for including or excluding studies from the analyses are discussed below.

Coding of Studies for Contextual (Social Interaction) Factors

Occupational Work Interdependence Variables

In many cases, the sample for a study could be associated with a single occupation. When more than one job or occupation was listed, two coders discussed whether the jobs or occupations were similar enough to include. If so, then the occupational information associated with each occupational code was recorded. In a number of studies, an author described a broad occupational field (e.g., production workers in a food manufacturing plant, see Luria, 2008). In such cases, we located occupational codes for up to five related occupations. Related occupations are listed in the O*NET database. For instance, in the case of Luria (2008), we recorded occupational codes and information for two related occupations: 51–3093.00—Food Cooking Machine Operators and Tenders and 51–9198.00—Helpers-Production Workers. In some studies, occupational information was too vague and the study could not be included.

Within the O*NET content model, there are two sections, worker requirements and occupational requirements, that include variables expected to promote social interaction and work interdependence (see Borman et al., 1999; Jeanneret, Borman, Kubisiak, & Hanson, 1999; Mumford & Peterson, 1999). As such, two composite variables were initially computed, a “worker” composite and an “occupational” composite, comprised of relevant variables that were computed. For the worker composite, we included skills, work style requirements, and work values expected to promote social interaction. More specifically, the variables pertaining to two social skills (i.e., social perceptiveness and service orientation), three work style requirements (i.e., cooperation, concern for others, and social orientation), and one work value (i.e., relationships) made up the worker composite. For the occupational composite, one generalized work activity (i.e., establishing and maintaining personal relationships) and one work context characteristic (i.e., engaging in teamwork) were included. We computed an overall work interdependence score by combining the worker and occupational composites. We note that the culling of O*NET variables for the worker and occupational composites was based on independent reviews of the O*NET database by three coders and subsequent consensus discussion on the relevance of these variables. Finally, we note that we conducted analyses separately with the worker and occupational composites. Given a high degree of consistency between results from the latter analyses and results based on the overall work interdependence score, we only report findings based on overall work interdependence.

All O*NET variables were initially measured within the O*NET database via a 5-point or 7-point scale and then converted to a standardized 100-point scale, with only the standardized scores being reported in the database. The exception being that teamwork was only reported on the original 5-point scale, which we transformed to a 100-point scale. Basically, the O*NET system averages the ratings by job analysts for each variable and standardizes them to a scale ranging from 0 to 100. From the O*NET database, we recorded these standardized scores, and when more than one relevant occupation was identified for a study sample, we averaged the respective scores.

Group Size and Organizational Level

In cases where the author did not report average group size, we computed an average group size from the available statistical information. For organizational level, we coded for three levels: workgroup/team, department within a business unit, and organization/business unit (e.g., store, restaurant).

Individualism-Collectivism

Cultural individualism-collectivism (IDV) was scored according to Hofstede’s system (https://www.hofstede-insights.com/product/compare-countries/; see Hofstede, 2001; Hofstede et al., 2010; Taras et al., 2010), where IDV is on a 0 to 100 scale. A low IDV score indicates that a country is more collectivistic, where people are expected to be loyal to the group. As such, maintaining harmony among group members overrides other issues. Examples of countries in our sample with very low IDV scores (high collectivism) are China, Ethiopia, South Korea, and Taiwan. Examples of countries in our sample that have mid-range IDV scores are India, Israel, and Spain. A high IDV score indicates a weaker interpersonal connection among group members. Examples of countries in our sample with very high IDV scores (high individualism) are Australia, the UK, and the USA.

General Climate Versus Climate-for-Something

Assessments of climate were categorized as general (concern for employees) if the researcher’s focus was on one or more general climate dimensions such as means emphasis, goal emphasis, and management support (see Kopelman et al., 1990; James et al., 2008). On the other hand, climate assessment was categorized as a climate-for-something if the researchers focused on a strategic emphasis such as service, safety, citizenship, or innovation (see Schneider et al., 1994).

Average Within-Group Tenure

Finally, we attempted to code for tenure within the group, as increases in within-group tenure could also provide increased opportunities for social interaction. While average organizational tenure was available for some studies, authors rarely reported within-group tenure. In addition, in the latter cases, average within-group tenure was often considerably less than average organizational tenure. This situation precluded any meaningful analyses with organizational or within-group tenure as contextual variables.

Coding Studies for Methodological Factors

Measurement Considerations

The study-level measurement characteristics that we coded for were the number of items for a climate scale, the number of response options on a climate measure, and the internal consistency reliability of the climate measure. In addition, almost all studies reported measurement characteristics for their climate scale, which we directly recorded.

Data Aggregation Model and Measure of Central Tendency

For data aggregation model, we coded the model as direct consensus if all climate items referenced the individual (e.g., “I believe that …”). We coded the data aggregation model as referent shift when all climate items referenced the workgroup (e.g., “My team believes that …”). In addition, we included a mixed category. For the latter category, an author would have measured some climate items with respect to direct consensus and other items with respect to a referent shift. We also coded for whether the author used the mean, median, or midpoint of a range for within-group agreement values when justifying data aggregation relative to a practical cutoff value.

Null Response Distribution

For the chosen null or random response distribution, we coded for two types of null response distribution: slightly skewed or uniform. Unfortunately, the vast majority of authors in our sample of work climate studies (approximately 83%) did not provide information on the employed null response distribution. Therefore, we were unable to include this variable in our analyses.

Publication Status

We coded publication status as published article versus unpublished paper (dissertation, masters thesis, or conference proceeding). Approximately, 10% of our studies were unpublished.

Sampling Procedure

Finally, we attempted to code for the nature of the sampling procedure(s) employed by authors for both individual-level and group-level sampling. While many studies reported the overall percentage of individuals responding to a survey, authors’ reporting on sampling procedures was very limited and inconsistent. This situation led us to be unable to code for sampling techniques as methodological factors.

meta-CART Procedures

To examine the relationship between mean within-group agreement, indexed by mean rWG(J), and the set of potential study-level predictor variables as well as study-level predictor variable interactions, the meta-analysis combined with classification and regression trees method was employed (meta-CART; Li et al., 2017). Initially, we used the metacart R package (Li et al., 2019) to identify the most important study-level predictors and subgroups via a CART analysis, where the dependent variable was the continuous mean rWG(J). Then, we conducted subgroup meta-analyses using metafor (Viechtbauer, 2010) to integrate the data for each subgroup. Below, we initially discuss CART procedures followed by a description of subgroup meta-analytic procedures. The reader is also referred to Brieman et al. (1998); Hayes, Usami, Jacobucci, and McCardle (2015); Strobl et al. (2009); and Therneau and Atkinson (2017) for discussions on and applications of CART.

CART

CART is a flexible nonparametric data analysis tool. The CART method finds a regression tree that indicates the extent to which the average value of the dependent variable (mean within-group agreement in our case) lies in a region which is specified as below and above particular values of a set of independent, predictor variables. The tree is constructed by recursively partitioning the data using a simple rule. The rule in each partition/split is based on a logical “if, then” statement. At each split, the data are partitioned into two mutually exclusive groups, where the variance of the dependent variable is as small as possible within each subgroup. The resulting tree indicates which variables are most important in explaining the dependent variable and what their relationships are with the dependent variable. That is, the variables at the top of the tree are the most discriminating (best predictor) variables, with cut points or values on the predictor variables indicating where subgroups are much lower versus much higher on the dependent variable.

The set of retained variables typically includes a small fraction of the candidate predictors and a small fraction of the predictor variable interactions. The initial decision tree is often too large and risks overfitting the data and thus the possibility of poorly generalizing to new samples. Therefore, in order to improve the predictive accuracy, a pruning process is typically applied to the initial decision tree. It is done by omitting parts that provide little additional power and can be considered as “noise.” This procedure is analogous to the omission of non-significant variables in stepwise regression.

As described below, we conducted multiple CART analyses under different assumptions. Since the CART algorithm is non-deterministic (i.e., for a given input, the output is not always the same), a random seed is used to start the program to yield reproducible results. Further, random seeds themselves can produce different models or partitioning of the data. Therefore, we ran analyses with multiple random seeds. For the models that were consistently produced, a cross-validation procedure is recommended to further evaluate the model(s). For the algorithm that produced a final, pruned tree for each of our analyses, we applied the tenfold cross-validation procedure as follows. Initially, the data were split into approximately 10 equal-size sample sizes. Trees were then produced 10 times. Each time the tree was constructed, one of the samples was left out. The omitted sample was subsequently used to evaluate the “prediction error.” A final, pruned tree was the one with the smallest cross-validation estimate of error. For a pruned tree, the subgroups at the bottom of the tree were, therefore, characterized by a succession of variables each with a particular cutoff value.

meta-CART

meta-CART extended the CART analysis, with meta-analyses being conducted to statistically summarize the results for the subgroups in a pruned tree. Given that meta-CART is designed for use with an effect size as the dependent variable, we needed to define the sampling variance of rWG(J) prior to applying this method. Since the variance of rWG(J) values within a study is not reported, we assumed it to be approximately equal across samples. Therefore, the sampling variance of within-study mean rWG(J) (the effect size for purposes of a meta-analysis) was defined as Constant*(1/N), since the sampling variance of rWG(J) (i.e., for mean and median rWG(J) values) is inversely proportional to sample size N. Here, N is the number of groups in a primary study. Following Eq. 3 of Li et al. (2017), it can be seen that the Constant is reduced when it is used for weighting effects; hence, only 1/N was employed for weighting of effects in the subgroup meta-analyses.

For our meta-CART analyses, we conducted analyses with respect to both fixed-effects and random-effects models. While a random-effects model is generally preferred in traditional meta-analyses, we desired to compare results for analyses conducted with respect to the two models. For our investigation, a fixed-effects model in comparison to a random-effects model assumes subgroups, the respective studies in the terminal nodes, to be homologous (i.e., to reflect common situations) without substantive variability in mean rWG(J). This assumption is a reasonable one for our investigation and, when employing a fixed-effect model, would allow us to describe mean within-group agreement for the final subgroups (terminal nodes) in the pruned tree, where only basic sampling error due to N (N of groups per study) was taken into account. In addition, unlike a random-effects meta-CART, the fixed-effects meta-CART produces variable importance assessments. Therefore, we employed a fixed-effects model, as well as a random-effects model, to assist in a better overall interpretation of findings. Given that meta-CART can produce different results depending on the initial seed for starting an analysis, we conducted supplemental analyses for both fixed-effects and random-effects models with different seeds.

Within the fixed-effects meta-CART, variable importance is technically defined as the sum of the goodness of split measures for each split for which the variable was the primary variable, plus goodness times “adjusted agreement” for all splits in which it was a surrogate. A surrogate variable in CART is that variable that mimics or predicts the split of the primary variable (Yohannes & Webb, 1999). The latter assessments were helpful in determining the relative value of the final set of predictors. In addition, as a check on the assumption concerning heterogeneity in our final subgroups for the fixed-effects analyses, we examined Qw and I2 statistics for the relevant subgroups. Qw, the within-subgroup Q statistic, is used to test for heterogeneity, whereas the I2 index measures the extent of true heterogeneity and is similar to the intraclass correlation in cluster sampling (Higgins & Thompson, 2002; Huedo-Medina, Sanchez-Meca, Marin-Martinez, & Botella, 2006). An I2 = 0 indicates that all subgroup variability is due to basic sampling error, with values of 0.25 and 0.50 reflecting low and moderate variability, respectively (Huedo-Medina et al., 2006). To test for pairwise differences between subgroups in mean within-group agreement, while adjusting for multiple comparisons, we used Holm’s method within the multcomp R library (see Hothorn et al., 2008).

Finally, Li et al. (2017) reported that the statistical power of a CART analysis for fixed-effects as well as random-effects increases to very acceptable levels (i.e., greater than 0.80 with 120 studies) with type I error rates decreasing below 0.05 (always when the number of studies is 120). Importantly, Li et al. (2017) demonstrated that the performance of a meta-CART is not (largely) influenced by the number of moderators (predictors) and the correlation between the moderators. According to Li et al.’s standard of a minimum of 120 studies for adequate power in a meta-CART with a complex design, our fixed-effects and random-effects meta-CARTs with 137 studies (and 141 samples) have very good power.

Results

We organize the presentation of results according to findings pertaining to random-effects and fixed-effects models. Prior to presenting the meta-CART findings, we present the descriptive statistics and correlations between variables in Table 1.

Table 1 Descriptive statistics and correlations between the study variables

Random-Effects Analyses

An initial analysis produced a pruned tree with four subgroups, where average group size, number of climate items, and climate scale reliability were the discriminating variables. A second analysis with a different initial seed did not detect predictor variables. Given that the model from the first analysis produced an interpretable meta-tree (that was also consistent with fixed-effects models discussed below), we present complete results for this analysis where average group size, number of climate items, and climate scale reliability interacted.

The pruned tree for the random-effects meta-CART analysis with four subgroups is presented in Fig. 1. As shown in Fig. 1, average group size was the primary discriminating variable between lower and higher values of mean within-group agreement. When average group size was less than 14, mean within-group agreement was relatively high (i.e., 0.86). When average group size was equal to or greater than 14, the number of climate items further distinguished between mean within-group values. More specifically, when the number of climate items was on average less than 4.5, mean within-group agreement was the lowest (0.66). The reliability of a climate measure also interacted with number of climate items and average group size, which resulted in higher mean within-group agreement values for climate measures with 4.5 (on average) or more items. That is, when reliability was less than 0.88, mean within-group agreement was 0.77 in comparison to 0.88 when reliability was equal to or greater than 0.88.

Fig. 1
figure 1

Random-effects meta-free for the prediction of mean within-group agreement. K = number of samples. Within-group agreement is a weighted estimate of mean rWG values for the respective sets of studies

The subgroup meta-analytic results for the final groupings in the pruned tree are presented in Table 2. The magnitude of mean within-group agreement for studies with average group size of less than 14 (i.e., 0.86) was approximately equal to mean within-group agreement for studies with larger average group size, five or more climate items, and high reliability (i.e., 0.88). Notably, the confidence intervals for these two mean within-group agreement values overlapped, but these confidence intervals did not overlap with conditions leading to lower mean within-group agreement values. As such, mean within-group agreement was significantly lower in studies where (a) average group size was below 14 and the number of climate items was less than five and (b) average group size was below 14, the number of climate items was on average greater than 4.5, and scale reliability was less than 0.88. While we conducted multiple comparisons of means with Tukey contrasts and Holm’s method for adjusted p values, we note that the conclusions from these contrasts did not differ from pairwise comparisons of confidence intervals for the respective groupings and mean within-group agreement values in Table 2. Therefore, we do not report the specific findings from these contrasts.

Table 2 Random-effects meta-CART results for mean within-group agreement

Notably, a test for between-group heterogeneity under the random-effects assumption, Qb, was statistically significant (73.41, df = 3, p < 0.01), with the estimate of residual heterogeneity being zero. Furthermore, the test of moderators, QM, was statistically significant (9024.27, df = 4, p < 0.01). These findings provide good support for the four study-level variables as the best predictors of mean within-group agreement under a random-effects model.

Fixed-Effects Analyses

An initial analysis produced a pruned tree with five subgroups, where average group size, individualism-collectivism, number of climate items, and climate scale reliability were the discriminating variables between lower and higher values of mean within-group agreement. A second analysis with a different seed produced a pruned tree (with five subgroups) that was identical to the pruned tree from the first analysis. Therefore, our presentation of complete results applies to both analyses.

The pruned tree for the fixed-effect CART analysis with five subgroups is presented in Fig. 2. As shown in Fig. 2, average group size was the primary discriminating variable between lower and higher values of mean within-group agreement. Further, average group size interacted with individualism-collectivism to produce the highest level of mean within-group agreement. More specifically, when average group size was less than 13.6 and individualism-collectivism was less than 26.5, mean within-group agreement was relatively high (0.91). For studies with smaller groups (i.e., less than 13.6 average) from more individualistic countries, mean within-group agreement was lower (i.e., 0.84). For both of the final nodes involving individualism-collectivism, the I2 values support these sets of studies as being homogenous (i.e., a lack of true-heterogeneity within studies in each node). For the model in Fig. 2, the importance values were 48.63 for average group size, 26.72 for individualism-collectivism, 26.13 for number of items, and 24.64 for climate scale reliability.

Fig. 2
figure 2

Fixed-effects meta-free for the prediction of mean within-group agreement. K = number of samples. Within-group agreement is a weighted estimate of mean rWG values for the respective set of studies. For individualism-collectivism scores greater than or equal to 26.5, the study was conducted in a more individualistic culture. For individualism-collectivism scores equal to or less than 26.5, the study was conducted in a more collectivistic culture

As shown in Fig. 2 and Table 3, the interactions between average group size, number of climate items, and climate scale reliability were identical to those from the random-effects meta-CART analyses (reported in Fig. 1 and Table 2). Notably, the confidence intervals for mean within-group agreement for the final nodes involving individualism-collectivism overlapped with the confidence interval for mean within-group agreement for studies when the average group size was greater than 13.6, the number of climate items was equal to or greater than 4.5, and climate scale reliability was high (0.885 or greater). Yet, mean within-group agreement for each of these three combinations of study characteristics was significantly greater than mean within-group agreement for the set of studies where average group size was greater than 13.6, number of climate items was less than 4.5, and climate scale reliability was below 0.885. The lowest level of mean within-group agreement was observed when average group size was greater than 13.6 and the number of climate items was less than 4.5 on average. Mean within-group agreement for the latter set of studies was significantly lower than mean within-group agreement for any other combination of study characteristics. In addition, we conducted multiple comparisons of means with Tukey contrasts and Holm’s method for adjusted p values. As reported in Table 3, in only one case (i.e., the comparison involving lower and higher individualism-collectivism), did the pairwise comparisons of confidence intervals for the respective groupings differ from the Tukey contrasts.

Table 3 Fixed-effects meta-CART results for mean within-group agreement

Overall, the pruned trees from fixed-effects and random-effects analyses were very similar in terms of identifying the most important predictors of mean within-group agreement. That is, average group size, the number of climate items, and climate scale reliability were the variables that differentiated between higher and lower levels of mean within-group agreement for both types of models. The primary differences in findings between the respective fixed-effects and random-effects analyses were that individualism-collectivism was identified as an important variable that distinguished between levels of mean rWG in both fixed-effects analyses.

Discussion

In this investigation, we examined the relative importance and interaction of contextual and methodological predictors of mean within-group agreement as indexed by rWG. Consistent with the social interaction literature, our findings indicated that average group size is the most important differentiator between higher and lower levels of mean within-group agreement and that other contextual and methodological factors interact with average group size to affect mean within-group agreement. As such, this study advances our understanding of the phenomenon of within-group agreement for samples of groups by identifying the situations and methodological conditions under which researchers and practitioners can expect lower versus higher levels of mean within-group agreement. Further, our findings have the potential to ground and advance future research in not only the domain of work climate, but also in other domains where investigators aggregate data to study relationships among group-level variables. Below, we draw attention to the theoretical and research implications of our findings for advancing group-level research in relation to the designing group-level studies and measuring phenomena where data are to be aggregated, conducting primary and meta-analytic studies involving aggregated data, and studying potential situational moderators of relationships between aggregated variables. In addition, we discuss how our applications of meta-CART illustrate this procedure and provide guidance for how meta-analytic researchers can simultaneously evaluate large sets of potential moderator variables via meta-CART.

Key Insights for Advancing Future Research

The differences in average within-group agreement across the subgroups in our models (meta-trees) paint an informative picture for where and how low, moderate, and high levels of mean within-group agreement might be determined. Notably, the subgroupings that resulted from the interaction of average group size, individualism-collectivism, and measurement factors minimally suggest a framework for understanding relationships involving work climate where researchers have aggregated climate data. That is, the five subgroups from our fixed-effects analyses offer a framework for both primary and meta-analytic researchers as to where they might expect lower versus higher relationships between aggregated climate measures and other variables. Here, we highlight the key insights from our results.

Average Group Size Is a Primary Determinant of Mean Within-Agreement for Work Climate

A key finding was that average group size was the most important discriminator between lower and higher levels of mean rWG in all analyses. That is, all analyses indicated that smaller group sizes (i.e., less than 14 on average) are associated with higher levels of mean within-group agreement. This latter finding offers indirect support for Colquitt and colleagues’ (2002) argument that decreases in group size lead to increases in social interaction and the development of shared expectations. We also note arguments concerning the effects of group size on workers’ perceptions have been discussed for some time in relation to organizational level (e.g., team, department, business unit) or how physically dispersed individuals are from one another (e.g., see Hare, 1981; Simon, 1973). Clearly, our findings point to group size and not organizational level as the primary factor contributing to weaker versus stronger perceptual agreement on work climate assessments in sets of groups within organizations. This conclusion is supported by the fact that organizational level did not emerge within any of our analyses as a primary contributor to higher versus lower levels of within-group agreement for samples of groups.

In addition, our study-level findings concerning average group size suggest that increases in group size will not always produce higher levels of within-group agreement in primary group-level studies involving work climate. This possibility is somewhat consistent with Brown and Hernstein (2005) and LeBreton and Senter’s (2008) psychometric arguments that, within a sample, all else “needs to be equal” including the assumption of equal item variances, for such a relationship to be observed. It is plausible that the equal item variance assumption does not hold in practice and especially insofar as items on work climate scales are concerned. In our case, indirect evidence of unequal item variances comes from the average item variances on climate scales that often are incorporated into composite climate scores, where these variances themselves tend to have a fair amount of variability (e.g., see Anderson & West, 1998; Ostroff et al., 2002; Parker et al., 2017; Patterson et al., 2005).

The managerial implication of our general finding concerning how average group size affects mean within-group agreement is that as group size becomes somewhat large, and especially greater than 13 members per group, more deliberate efforts and opportunities may need to be made for workers to share perceptions of the work environment. This point would particularly apply to organizations that desire to develop and benefit from strong work climates.

Work Climate Measurement Is a Primary Contributor to Mean Within-Group Agreement

Another key finding was that work climate measurement, the number of climate items, and climate scale reliability distinguished between lower and higher levels of mean within-group agreement. In particular, when average group size was greater than 14 and a climate measure had few items (four or less), mean within-group agreement was the lowest. A straightforward measurement recommendation, consistent with some arguments in the methodological literature on rWG, might be to simply add more items to increase rWG values for a sample of groups. Yet, our findings point to the need for a more nuanced measurement development effort. That is, simply adding more items to increase rWG values and mean rWG (given computational similarity of their formulae to the Spearman-Brown “stepped-up correction” in reliability estimation) will not necessarily produce high levels of rWG and mean within-group agreement.

In relation to the related domain of reliability estimation, several psychometricians argued some time ago that adding more and more items to a measure will lead to item nonequivalence and possibly even lower levels of reliability when reliability is estimated via a Spearman-Brown stepped-up correction (see Allen & Yen, 1979; Lord & Novick, 1968). The reason being that there is a tendency for items to become less parallel as more items are developed. Assuming “all else equal,” these points would hold for within-sample rWG estimation and mean rWG estimation as well. Yet, all else was not equal in uncovering our between-study result pertaining to number of climate scale items, as our finding was made in relation to the interaction of number of items and average number of individuals per group. While our between-study finding does not directly relate to within-sample rWG estimation, it nonetheless is consistent with the above psychometric argument and suggests that researchers attend to a careful item development process (i.e., ensure that all scale items have comparable properties) to better estimate within-sample rWG and mean within-group agreement.

While our meta-CART results indicated that the 11 studies with the lowest average level of mean within-group agreement were somewhat heterogeneous, a closer examination of these studies reveals several commonalities. First, although type of climate measure was not a discriminating variable in our meta-CART analyses, 10 of the 11 studies in the set with the lowest level of mean within-group agreement included climate-for-something measures (e.g., safety, justice, diversity). If within-group agreement tends to be at its lowest level in climate-for-something studies in comparison to studies that measure climate more generally (e.g., with respect to general employee well-being), the possibility exists that work climate-workgroup performance relationships will not be greater in climate-for-something studies. The implication of our discovery is that future research could be directed at cumulatively integrating the literature on different types of climate measures to empirically evaluate expectations concerning the superior predictive effectiveness of climate-for-something measures (see Schneider et al., 2017).

A second commonality is that a majority of studies with the lowest levels of mean within-group agreement were published in highly regarded journals where no group was excluded based on a very low rWG value. Importantly, in these studies, researchers relied on mean or median rWG, not the range of rWG values, for data aggregation decisions. This information points to the fact that researchers, reviewers, and editors are not systematically censoring data from samples of groups with low rWG values. Moreover, this practice of including all sampled groups is further reason why it is important that a requirement be in place for the construction of confidence intervals for mean within-group agreement. Confidence intervals for mean within-group agreement can assist in informing data aggregation decisions, enhance understanding of agreement in the population of groups, and contribute information that will be beneficial in efforts to cumulatively integrate work climate studies. The reader is referred to Burke et al. (2018) for a discussion on a “central tendency approach” to the assessment of within-group agreement including a more detailed discussion on procedures (including R code) for the construction of confidence intervals for alternative indices of mean within-group agreement.

Related to better understanding the effects of sampling on mean within-group agreement, we observed that the majority of studies did not report any sampling information, and of the studies that did report such information, there was wide variation in sampling practices at the group and individual levels. This variation in sampling practices was coupled with virtually no information on actual or average group sizes prior to sampling. These shortcomings in our database precluded us from examining the effects of sampling on mean within-group agreement. We strongly encourage researchers in studies where they aggregate data to report information on sampling procedures at both the group and individual levels. Such information will contribute to better informing when and how sampling affects mean within-group agreement.

Cultural Individualism-Collectivism Is a Determinant of Mean Within-Group Agreement

Another discovery was that cultural individualism-collectivism was a discriminator between lower and higher levels of mean rWG, with the highest level of mean within-group agreement observed in studies conducted within smaller (less than 14 members) groups in more collectivistic cultures. Importantly, the prioritization of strong social relations and group harmony within a culture appear to be drivers of social interactions within samples of groups and, thus, higher levels of within-group agreement on work climate scales for these collections of groups.

Notably, the tipping point for higher versus lower levels of mean agreement was somewhat low, approximately 27 on a 100-point scale for our set of studies. This tipping point suggests that collections of groups and researchers within moderate and highly individualistic national cultures, such as Australia and the USA, are at somewhat of a disadvantage regarding high levels of mean within-group agreement. As such, the ability to justify data aggregation and study relationships between aggregated variables may be somewhat more challenging in small groups in highly individualistic cultures. This finding also points to the possibility that individualism-collectivism may moderate work climate-workgroup performance relationships, where researchers may observe stronger relationships in more collectivistic cultures. This possibility is a direction for future research, especially in relation to meta-analytic efforts to identify why and how work climate-workgroup performance relationships vary across different contextual and study factors. To date, those investigations (e.g., see Wallace et al., 2016) have not considered the theoretical role of individualism-collectivism when examining moderators of work climate-workgroup performance relationships.

We note that the vast majority, over 70%, of studies in our database were from Anglo and Confucian Asian cultural clusters (see Gupta et al., 2002). Countries within the Anglo cluster, such as Australia, the USA, the UK, and Canada, are considered to be highly individualistic, whereas countries in the Confucian Asian cluster, such as China, Hong Kong, Taiwan, and South Korea, are considered to be highly collectivistic. It is plausible that this overrepresentation of countries with respect to two cultural clusters and especially in relation to studies conducted within the USA (approximately one-third) and China and Taiwan (approximately 20%) led to a relatively low country score for distinguishing between lower and higher levels of mean within-group agreement. As such, how individualism-collectivism discriminates between different levels of within-group agreement for groups embedded in a broader set of countries and cultural clusters is an open question.

In addition, the finding that national culture coupled with small group size is antecedent to within-group agreement on work climate scales for a sample of groups suggests that mean within-group agreement itself is worthy of investigation as a study-level variable. In this regard, we note several points. First, as discussed above, researchers often employ rWG as an indicator of climate strength, the extent of within-group agreement concerning members’ climate perceptions. Second, as collectivism increases, it would likely restrict climate strength scores for a sample of groups, especially when the set of groups is embedded within a single country, a feature that characterizes the majority of studies in the work climate literature. In fact, when discussing why researchers have found mixed support for climate strength as a moderator of work climate-workgroup performance relationships, González-Roma and Peiro’s (2014) review of the climate strength literature offered an explanation tied to the lack of variability in climate strength scores within sets of groups. Essentially, the argument is that within-study range restriction in climate strength scores may preclude finding support for the moderator hypothesis in a primary group-level study. Together, these arguments provide a basis for examining the extent to which mean rWG and perhaps other mean interrater agreement statistics moderates work climate-workgroup performance relationships.

Furthermore, while our investigation pointed to the role of national individualism-collectivism as a determinant of within-group agreement, the possibility exists that intra-national differences on this dimension may have a greater effect on study-level interrater agreement (Tung & Baumann, 2009; Tung & Verbeke, 2010). That is, the multitude of differences within peoples of a given country (e.g., ethnic, generational, and religious) may affect social interactions and, thus, study-level interrater agreement. Although Tung and Verbeke point out that measuring such intra-national diversity may be more nebulous and challenging than measuring cultural variables at the country level, we encourage future research to examine the role of intra-national variations in cultural values on study-level within-group agreement. At a minimum, such research could clarify the relative importance of national versus intra-national differences in cultural individualism-collectivism as potential determinants of within-group agreement and, at the same time, provide information concerning the tenability of our assumptions about the relevance of national individualism-collectivism at the study level. To the extent that intra-national differences in individualism-collectivism operate to affect social interactions and study-level interrater agreement, our findings would have conservatively estimated the impact individualism-collectivism on study-level interrater agreement.

Occupational Work Interdependence Is Not a Primary Predictor of Mean Within-Group Agreement

A possible reason for why occupational work interdependence did not emerge as a primary discriminating variable is that most of the occupational skills and work styles that we studied reflected ability and personality requirements that would arguably have only indirect effects, through social interaction, on the emergence of shared perceptions of work environment characteristics. A related possibility for the pattern of findings is that work interdependence itself may be best measured and studied as a within-group phenomenon, not as a between-study occupational variable. Finally, how we measured work interdependence for occupations, O*NET, is based on a system for jobs within the US economy. Most studies in our investigation were non-US based, where the O*NET scoring system for occupational variables may not directly apply.

These Findings and meta-CART Procedures Are Potentially Relevant to Other Domains of Inquiry, Within-Group Agreement Assessments, and Meta-analytic Research

More generally, our overall findings and models provide information for researchers in domains other than work climate to consider when studying relationships involving aggregated data. It is plausible that our overall findings and models can assist researchers in other domains to better understand where and why differences in findings occur when these researchers aggregate data to study group-level relationships. For instance, researchers often study team mental models and leadership variables at the group-level based on aggregated data, where they have found inconsistencies in findings between similar studies (see Biemann et al., 2012, 2014). The insights gained from our investigation may shed light on why such inconsistencies have arisen and at the same time offer suggestions for how to improve research in those domains.

Related to the point about the degree to which our findings assist more broadly in understanding within-group agreement for samples of groups, we note that we produced our models relative to only the rWG index. Future, primary and meta-CART research could provide an indication of the extent to which our models assist in understanding the meaning of other indices of within-group agreement. While our research offered insights into the meaning of mean within-group agreement in relation to the rWG index, researchers have employed other indices for assessing within-group agreement. For instance, climate researchers often employ the average deviation (AD) index (Burke et al., 1999) as both a measure of within-group agreement and an indicator of climate strength (see Gonzalez-Roma and Peiro, 2014). Unlike rWG, the AD index is not dependent on a null response distribution or the number of items on a work climate scale. Therefore, the question remains as to the extent to which the present findings generalize or relate to other indices of agreement such as the AD index.

Furthermore, our applications of meta-CART provide guidance for how meta-analytic researchers in any domain can simultaneously evaluate large sets of potential moderator variables. Presently, meta-analytic researchers in applied psychology and business assess the viability of moderator variables via subgroup analyses or meta-regressions. The subgrouping of studies is often a process that leads to the consideration of just a few moderators at a time. Whereas the meta-regression approach typically allows for the consideration of a broader set of moderators, is not very useful for identifying complex forms of interaction among the potential moderator variables. This point is especially the case for many meta-analyses in psychology, where somewhat small numbers of studies and many potential moderators are common characteristics (see van Lissa, 2020). As a number of authors have pointed out (Guolo & Varin, 2017; Parr, Loan, & Tanner-Smith, 2021; van Lissa, 2020), in these situations, as researchers add more moderators to a meta-regression, they risk overfitting the data. It is our hope that our illustrations and applications of both fixed- and random-effects meta-CART models will assist meta-analytic researchers in studying moderator variables in a more advanced manner, which allows for identifying complex forms of interaction among a set of moderator variables.

When applying meta-CART within any domain, we encourage researchers to conduct multiple analyses with different random seeds, as results may change somewhat in terms of the extent to which meta-trees are developed. Along with multiple analyses, we strongly encourage the use of a tenfold cross-validation procedure to ensure the reproducibility of findings. In addition, we suggest that researchers consider the potential relevance of a fixed-effects model, as studies within the nodes of a pruned meta-tree may well be homogeneous. In doing so, we encourage researchers to compute I2 statistics (see Higgins & Thompson, 2002) from the reported Q statistics to quantify the extent of heterogeneity among a set of effects in the final nodes. I2 statistics are not produced as part of a meta-CART output, yet computing them can assist in determining the meaningfulness of a branch (i.e., results in a terminal node of a meta-tree). Finally, we suggest that researchers attend to the consistency of variable importance assessments from alternative analyses to better interpret overall findings.

Conclusion

In summary, the findings from this investigation point to the conclusion that mean within-group agreement with respect to work climate is fundamentally a property of group members’ social interaction and environmental experiences and that researchers can enhance the estimation of mean within-group agreement through sound measurement (item) development. More specifically, the models or meta-trees that we derived indicate how average group size, individualism-collectivism, number of items on a climate scale, and climate scale reliability interact to affect within-group agreement for work climate at the study level. As a result, our meta-trees specify where and when researchers should expect lower versus higher levels of mean within-group agreement. This information can help guide future primary and meta-analytic research aimed at better understanding within-group agreement, relationships between variables where investigators aggregate data, and how study-level factors affect the strength of workers’ perceptions of work environment characteristics.

References

*An asterisk indicates that a study was included in the meta-CART analyses. aInformation from Van de Voorde et al. (2010) and Van de Voorde et al. (2014) was combined due to use of the same sample..

  • Abdelhadi, N., & Drach-Zahavy, A. (2012). Promoting patientcare: Work engagement as a mediator between ward service climate and patient-centered care. Journal of Advanced Nursing, 68, 1276–1287.

    PubMed  Google Scholar 

  • *Afsharian, A., Zadow, A., Dollard, M. F., Dormann, C., & Ziaian, T. (2018). Should psychosocial safety climate theory be extended to include climate strength? Journal of Occupational Health Psychology, 23, 496–507.

    PubMed  Google Scholar 

  • *Alfes, K., Shantz, A. D., & Ritz, A. (2018). A multilevel examination of the relationship between role overload and employee subjective health: The buffering effect of support climates. Human Resource Management, 57, 659–673.

    Google Scholar 

  • Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Brooks/Cole Publishing Company.

    Google Scholar 

  • *Ambrose, M. L., Schminke, M., & Mayer, D. M. (2013). Trickle-down effects of supervisor perceptions of interactional justice: A moderated mediation approach. Journal of Applied Psychology, 98, 678–689.

    Google Scholar 

  • *Anderson, N. R., & West, M. A. (1998). Measuring climate for work group innovation: Development and validation of the team climate inventory. Journal of Organizational Behavior, 19, 235–258.

    Google Scholar 

  • *Auh, S., Menguc, B., & Jung, Y. S. (2014). Unpacking the relationship between empowering leadership and service-oriented citizenship behaviors: A multilevel approach. Journal of the Academy of Marketing Science, 42, 558–579.

    Google Scholar 

  • *Bai, Y., Lin, L., & Liu, J. T. (2017). Leveraging the employee voice: A multi-level social learning perspective of ethical leadership. The International Journal of Human Resource Management.

  • *Bain, P. G., Mann, L., & Pirola-Merlo, A. (2001). The innovation imperative: The relationships between team climate, innovation, and performance in research and development teams. Small Group Research, 32, 55–73.

    Google Scholar 

  • Bandalos, D., & Enders, C. (1996). The effects of nonnormality and number of response categories on reliability. Applied Measurement in Education, 9, 151–160.

    Google Scholar 

  • *Berson, Y., Da’as, R., & Waldman, D. (2015). How do leaders and their teams bring about organizational learning and outcomes? Personnel Psychology, 68, 79–108.

    Google Scholar 

  • Biemann, T., Cole, M. S., & Voelpel, S. (2012). Within-group agreement: On the use (and misues) of rWG and rWG(J) in leadership research and some best practice guidelines. The Leadership Quarterly, 23, 66–80.

    Google Scholar 

  • Biemann, T., Ellwart, T., & Rack, O. (2014). Quatifying similarity of team mental models: An introduction of the rRG index. Group Processes & Intergroup Relations, 17, 125–140.

    Google Scholar 

  • *Bilinska, P., Wegge, J., & Kliegel, M. (2016). Caring for the elderly but not for one’s own old employees? Journal of Personnel Psychology, 15(3), 95–105.

    Google Scholar 

  • *Bin, D.A. (2018). Organizational climate and climate strength: Three essays. Doctoral Dissertation, University of Lausanne, Switzerland.

  • *Binci, D. (2011). Climate for innovation and ICT implementation effectiveness: A missing link in Italian e-government projects. International Journal of Public Administration, 34, 49–53.

    Google Scholar 

  • Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Klein & S. W. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations (pp. 349–381). Jossey-Bass.

    Google Scholar 

  • *Bodla, A.A., Tang, N., Jiang, W., & Tian, L. (2016). Diversity and creativity in cross-national teams: The role of team knowledge sharing and inclusive climate. Journal of Management and Organization, 1–19.

  • *Boehm, S. A., Dwertmann, D. J. G., Bruch, H., & Shamir, B. (2015). The missing link? Investigating organizational identity strength and transformational leadership climate as mechanisms that connect CEO charisma with firm performance. The Leadership Quarterly, 26, 156–171.

    Google Scholar 

  • Bogaert, S., Boone, C., & van Witteloostuijn, A. (2012). Social value orientation and climate strength as moderators of the impact of work group cooperative climate on affective commitment. Journal of Management Studies, 49, 918–944.

    Google Scholar 

  • Borman, W. C., Kubisiak, C., & Schneider, R. J. (1999). Work styles. In N. G. Peterson, M. D. Mumford, W. C. Borman, P. R. Jeanneret, & E. A. Fleishman (Eds.), An occupational information system for the 21st century: The development of O*NET (pp. 213–226). American Psychological Association.

    Google Scholar 

  • *Bornay-Barrachina, M., & Herrero, I. (2018). Team creative environment as a mediator between CWX and R&D team performance and moderating boundary conditions. Journal of Business and Psychology, 33, 311–323.

    PubMed  Google Scholar 

  • *Borucki, C. C., & Burke, M. J. (1999). An examination of service-related antecedents to retail store performance. Journal of Organizational Behavior, 20, 943–962.

    Google Scholar 

  • *Bosak, J., Dawson, J., Flood, P., & Peccei, R. (2017). Employee involvement climate and climate strength: A study of employee attitudes and organizational effectiveness in UK hospitals. Journal of Organizational Effectiveness: People and Performance, 4, 18–38.

    Google Scholar 

  • *Bosak, J., Fu, N., Ma, Q., & Flood, P. (2016). Intellectual capital and organizational ambidexterity in Chinese and Irish professional service firms. Journal of Organizational Effectiveness: People and Performance, 3, 94–114.

    Google Scholar 

  • *Boyce, A.S. (2010). Organizational climate and performance: An examination of causal priority. Unpublished doctoral dissertation, Michigan State University, East Lansing, MI.

  • *Brahm, T., & Kunze, F. (2012). The role of trust climate in virtual teams. Journal of Managerial Psychology, 27, 595–614.

    Google Scholar 

  • Brieman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1998). Classification and regression trees. Chapman and Hall/CRC.

    Google Scholar 

  • *Bronkhorst, B., & Vermeeren, B. (2016). Safety climate, worker health and organizational health performance. International Journal of Workplace Health Management, 9(270), 289.

    Google Scholar 

  • Brown, R. D., & Hauenstein, N. M. A. (2005). Interrater agreement reconsidered: An alternative to the rWG indices. Organizational Research Methods, 8(2), 165–184.

    Google Scholar 

  • *Buck, M.A. (2011). Proactive personality and big five traits in supervisors and workgroup members: Effects on safety climate and safety motivation. Dissertations and Theses. Paper 268. https://doi.org/10.15760/etd.268

  • Burke, M. J., Chan-Serafin, S., Salvador, R., Smith, A., & Sarpy, S. (2008). The role of national culture and organizational climate in safety training effectiveness. European Journal of Work and Organizational Psychology, 17, 133–154.

    Google Scholar 

  • Burke, M. J., Cohen, A., Doveh, E., & Smith-Crowe, K. (2018). Central tendency and matched difference approaches for assessing interrater agreement. Journal of Applied Psychology, 103, 1198–1229.

    Google Scholar 

  • Burke, M. J., Finkelstein, L. M., & Dusig, M. S. (1999). On average deviation indices for estimating interrater agreement. Organizational Research Methods, 2, 49–68.

    Google Scholar 

  • Burke, M. I., Landis, R. L., & Burke, M. J. (2017). Estimating group-level relationships: General recommendations and considerations for the use of intraclass correlation coefficients. Journal of Business and Psychology, 32, 611–626.

    Google Scholar 

  • *Buvik, M. P., & Tvedt, S. D. (2016). The impact of commitment and climate strength on the relationship between trust and performance in cross-functional project teams: A moderated mediation analysis. Team Performance Management, 22, 114–138.

    Google Scholar 

  • *Carrasco, H., Martínez-Tur, V., Peiró, J. M., & Moliner, C. (2012). Validation of a measure of service climate in organizations. Journal of Work and Organizational Psychology, 28, 69–80.

    Google Scholar 

  • Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234–246.

    Google Scholar 

  • *Chen, S., Zhang, G., Zhang, A., & Xu, J. (2016). Collectivism-oriented human resource management and innovation performance: An examination of team reflexivity and team psychological safety. Journal of Management & Organization, 22, 535–554.

    Google Scholar 

  • *Chen, X. P., Liu, D., & Portnoy, R. (2012). A multilevel investigation of motivational cultural intelligence, organizational diversity climate, and cultural sales: Evidence from U.S. real estate firms. Journal of Applied Psychology, 97, 93–106.

    Google Scholar 

  • *Chuang, C., & Liao, H. (2010). Strategic human resource management in service context: Taking care of business by taking care of employees and customers. Personnel Psychology, 63, 153–196.

    Google Scholar 

  • *Cole, M. S., Carter, M., & Zhang, Z. (2013). Leader-team congruence in power distance values and team effectiveness: The mediating role of procedural justice climate. Journal of Applied Psychology, 98, 962–973.

    Google Scholar 

  • Colquitt, J. A., Noe, R. A., & Jackson, C. L. (2002). Justice in teams: Antecedents and consequences of procedural justice climate. Personnel Psychology, 55, 83–109.

    Google Scholar 

  • *Costa, A. C. (2003). Work team trust and effectiveness. Personnel Review, 32, 605–622.

    Google Scholar 

  • *Darr, W., & Johns, G. (2004). Political decision-making climates: Theoretical processes and multi-level antecedents. Human Relations, 57, 169–200.

    Google Scholar 

  • *de Jong, A., de Ruyter, K., & Lemmink, J. (2005). Service climate in self-managing teams: Mapping the linkage of team member perceptions and service performance outcomes in a business-to-business setting. Journal of Management Studies, 42, 1593–1620.

    Google Scholar 

  • *Dietz, J., Pugh, S. D., & Wiley, J. (2004). Service climate effects on customer attitudes: An examination of boundary conditions. Academy of Management Journal, 47, 81–92.

    Google Scholar 

  • *Ehrhart, M. G. (2004). Leadership and procedural justice climate as antecedents of unit-level organizational citizenship behavior. Personnel Psychology, 57, 61–94.

    Google Scholar 

  • *Eisenbeiss, S., Knippenberg, D. V., & Boerner, S. (2008). Transformational leadership and team innovation: Integrating team climate principles. Journal of Applied Psychology, 93, 1438–1446.

    Google Scholar 

  • *Elche, D., Ruiz-Palomino, P., & Linuesa-Langreo, J. (2020). Servant leadership and organizational citizenship behavior: The mediating effect of empathy and service climate. International Journal of Contemporary Hospitality Management, 32, 2035–2053.

    Google Scholar 

  • Enders, C., & Bandalos, D. (1999). The effects of heterogeneous item distributions on reliability. Applied Measurement in Education, 12, 133–150.

    Google Scholar 

  • *Frazier, M. L., & Bowler, W. M. (2015). Voice climate, supervisor undermining, and work outcomes: A group-level examination. Journal of Management, 41, 841–863.

    Google Scholar 

  • *Frazier, M. L., & Fainshmidt, S. (2012). Voice climate, work outcomes, and the mediating role of psychological empowerment: A multilevel examination. Group & Organization Management, 37, 691–715.

    Google Scholar 

  • *Gelade, G. A., & Young, S. (2005). Test of a service profit chain model in the retail-banking sector. Journal of Occupational & Organizational Psychology, 78, 1–22.

    Google Scholar 

  • *Getachew, D. S., & Zhou, E. (2018). The influences of transformational leadership on collective efficacy: The moderating role of perceived organizational support. The International Journal of Organizational Innovation, 10, 7–15.

    Google Scholar 

  • Ginsburg, L., & Oore, D. G. (2016). Patient safety climate strength: A concept that requires more attention. BMJ Quality & Safety, 25, 680–687.

    Google Scholar 

  • *Gonzalez, J. A., & Denisi, A. S. (2009). Cross-level effects of demography and diversity climate on organizational attachment and firm effectiveness. Journal of Organizational Behavior, 30, 21–40.

    Google Scholar 

  • *Gonzalez-Mulé, E., Courtright, S. H., DeGeest, D., Seong, J.-Y., & Hong, D.-S. (2016). Channeled autonomy: The joint effects of autonomy and feedback on team performance through organizational goal clarity. Journal of Management, 42, 2018–2033.

    Google Scholar 

  • *Gonzalez-Roma, V., & Hernandez, A. (2014). Climate uniformity: Its influence on team communication quality, task conflict, and team performance. Journal of Applied Psychology, 99, 1042–1058.

    Google Scholar 

  • Gonzalez-Roma, V., & Peiro, J. (2014). Climate and culture strength. In B. Schneider & K. M. Barbera (Eds.), The Oxford handbook of organizational climate and culture (pp. 496–531). Oxford University Press.

    Google Scholar 

  • Gonzalez-Roma, V., Peiro, J. M., & Tordera, N. (2002). An examination of the antecedents and moderator influences of climate strength. Journal of Applied Psychology, 87, 465–473.

    Google Scholar 

  • *Griffith, J. (2006). A compositional analysis of the organizational climate-performance relation: Public schools as organizations. Journal of Applied Social Psychology, 36, 1848–1880.

    Google Scholar 

  • Guolo, A., & Varin, C. (2017). Random-effects meta-analysis: The number of studies matters. Statistical Methods in Medical Research, 26, 1500–1518.

    PubMed  Google Scholar 

  • *Guerrero, E. G., Fenwick, K., & Kong, Y. (2017). Advancing theory development: Exploring the leadership–climate relationship as a mechanism of the implementation of cultural competence. Implementation Science, 12, 1–12.

    Google Scholar 

  • Gupta, V., Hanges, P. J., & Dorfman, P. (2002). Cultural clusters: Methodology and findings. Journal of World Business, 37, 11–15.

    Google Scholar 

  • Hare, A. P. (1981). Group size. American Behavioral Scientist, 24, 695–708.

    Google Scholar 

  • Hayes, T., Usami, S., Jacobucci, R., & McArdle, J. J. (2015). Using classification and regression trees (CART) and random forests to analyze attrition: Results from two simulations. Psychology and Aging, 30, 911–929.

    PubMed  PubMed Central  Google Scholar 

  • Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Academic Press.

    Google Scholar 

  • *Heled, E., Somech, A., & Waters, L. (2016). Psychological capital as a team phenomenon: Mediating the relationship between learning climate and outcomes at the individual and team levels. The Journal of Positive Psychology, 11, 303–314.

    Google Scholar 

  • Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21, 1539–1558.

    PubMed  Google Scholar 

  • *Hirst, G., Budhwar, P., Cooper, B. K., West, M., Long, C., Chongyuan, X., & Shipton, H. (2008). Cross-cultural variations in climate for autonomy, stress and organizational productivity relationships: A comparison of Chinese and UK manufacturing organizations. Journal of International Business Studies, 39, 1343–1358.

    Google Scholar 

  • *Hofmann, D. A., & Mark, B. (2006). An investigation of the relationship between safety climate and medication errors as well as other nurse and patient outcomes. Personnel Psychology, 59, 847–869.

    Google Scholar 

  • *Hofmann, D. A., & Stetzer, A. (1996). A cross-level investigation of factors influencing unsafe behaviors and accidents. Personnel Psychology, 49, 307–339.

    Google Scholar 

  • Hofstede, G. (2001). Culture’s consequences: Comparing values, behaviors, institutions and organizations across nations (2nd ed.). Sage.

    Google Scholar 

  • Hofstede, G., Hofstede, G. J., & Minkov, M. (2010). Cultures and organizations (3rd ed.). McGraw-Hill.

    Google Scholar 

  • Hothorn, T., Bretz, F., & Westfall, P. (2008). Simultaneous inference in general parametric models. Biometrical Journal, 50(3), 346–363.

    PubMed  Google Scholar 

  • *Huang, C.-C., & Hsieh, P.-N. (2017). Inspiring creativity in teams: Perspectives of transactive memory systems. Journal of Pacific Rim Psychology, 11, 1–14.

    Google Scholar 

  • *Huang, Y.-H., Verma, S. K., Chang, W.-R., Courtney, T. K., Lombardi, D. A., Brennan, M. J., & Perry, M. J. (2012). Supervisor vs. employee safety perceptions and association with future injury in US limited-service restaurant workers. Accident Analysis & Prevention, 47, 45–51.

    Google Scholar 

  • Huang, X., & van de Vliert, E. (2004). Job level and national culture as joint roots of job satisfaction. Applied Psychology: An International Review, 53, 329–348.

    Google Scholar 

  • Huedo-Medina, T. B., Sanchez-Meca, J., Marın-Martınez, F., & Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q Statistic or I2 Index? Psychological Methods, 11(2), 193–206.

    PubMed  Google Scholar 

  • *Hui, C. H., Chiu, W. C. K., Yu, P. L. H., Cheng, K., & Tse, H. (2007). The effects of service climate and the effective leadership behaviour of supervisors on frontline employee service quality: A multi-level analysis. Journal of Occupational & Organizational Psychology, 80, 151–172.

    Google Scholar 

  • *Huo, W., Cai, Z., Luo, J., Men, C., & Jia, R. (2016). Antecedents and intervention mechanisms: A multi-level study of R&D team’s knowledge hiding behavior. Journal of Knowledge Management, 20, 880–897.

    Google Scholar 

  • *Idris, M. A., Dollard, M. F., Coward, J., & Dormann, C. (2012). Psychosocial safety climate: Conceptual distinctiveness and effect on job demands and worker psychological health. Safety Science, 50, 19–28.

    Google Scholar 

  • James, L. R., Choi, C. C., Ko, C.-H.E., McNeil, P. K., Minton, M. K., Wright, M. A., & Kim, K.-i. (2008). Organizational and psychological climate: A review of theory and research. European Journal of Work and Organizational Psychology, 17, 5–32.

    Google Scholar 

  • James, D. L., Demaree, R. G., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85–98.

    Google Scholar 

  • James, L. A., & James, L. R. (1989). Integrating work environment perceptions: Exploration into the measurement of meaning. Journal of Applied Psychology, 74, 739–751.

    Google Scholar 

  • Jeanneret, P.R., Borman, W.C., Kubisiak, U.C., & Hanson, M.A. (1999). Generalized work activities. In N.G. Peterson, M.D. Mumford, W.C. Borman, P.R. Jeanneret, and E.A. Fleishman (Eds.), An occupational information system for the 21st century: The development of O*NET, 105–125. Washington, D.C.: American Psychological Association.

  • *Jiang, K., Chuang, C., & Chiao, Y. (2015). Developing collective customer knowledge and service climate: The interaction between service-oriented high-performance work systems and service leadership. Journal of Applied Psychology, 100, 1089–1106.

    Google Scholar 

  • *Jiang, W., & Gu, Q. (2016). How abusive supervision and abusive supervisory climate influence salesperson creativity and sales team effectiveness in China. Management Decision, 54, 455–475.

    Google Scholar 

  • *Johnson, A., Nguyen, H., Groth, M., & White, L. (2018). Reaping the rewards of functional diversity in healthcare teams: Why team processes improve performance. Group & Organization Management, 43, 440–474.

    Google Scholar 

  • *Johnson, J. W. (1996). Linking employee perceptions of service climate to customer satisfaction. Personnel Psychology, 49, 831–851.

    Google Scholar 

  • *Jones, G.D. (2017). Modeling effective work groups and teams: An assessment of the inclusion of social value orientation. Unpublished doctoral dissertation, University of Louisville: Louisville, KY. Retrieved from https://ir.library.louisville.edu/etd/2877

  • *Kao, R.-H. (2017). Task-oriented work characteristics, self-efficacy, and service-oriented organizational citizenship behavior: A cross-level analysis of the moderating effect of social work characteristics and collective efficacy. Personnel Review, 46, 718–739.

    Google Scholar 

  • *Kao, F.-H., & Cheng, B.-S. (2017). Proservice or antiservice employee behaviors: A multilevel ethics perspective. Human Performance, 30, 272–290.

    Google Scholar 

  • *Katz-Navon, T., Naveh, E., & Stern, Z. (2005). Safety climate in healthcare organizations: A multidimensional approach. Academy of Management Journal, 48, 1075–1089.

    Google Scholar 

  • *King, E. B., de Chermont, K., West, M., Dawson, J. F., & Hebl, M. (2007). How innovation can alleviate negative consequences of demanding work contexts: The influence of climate for innovation on organizational outcomes. Journal of Occupational and Organizational Psychology, 80, 631–645.

    Google Scholar 

  • Klein, K. J., Conn, A. B., Smith, D. B., & Sorra, J. S. (2001). Is everyone in agreement? An exploration of within-group agreement in employee perceptions of the work environment. Journal of Applied Psychology, 86, 3–16.

    Google Scholar 

  • *Koene, B., Vogelaar, A., & Soeters, J. L. (2002). Leadership effects on organizational climate and financial performance: Local leadership effect in chain organizations. The Leadership Quarterly, 13, 193–215.

    Google Scholar 

  • *Koopmann, J., Lanaj, K., Wang, M., Zhou, L., & Shi, J. (2016). Nonlinear effects of team tenure on team psychological safety climate and climate strength: Implications for average team member performance. Journal of Applied Psychology, 101, 940–957.

    Google Scholar 

  • Kopelman, R. E., Brief, A. P., & Guzzo, R. A. (1990). The role of climate and culture in productivity. In B. Schneider (Ed.), Organizational climate and culture (pp. 282–318). Jossey-Bass.

    Google Scholar 

  • *Kozlowski, S. W. J., & Hults, B. M. (1987). An exploration of climates for technical updating and performance. Personnel Psychology, 40, 539–563.

    Google Scholar 

  • Krackhardt, D., & Kilduff, M. (1990). Friendship patterns and culture: The control of organizational diversity. American Anthropologist, 92, 142–154.

    Google Scholar 

  • LeBreton, J. M., James, L. R., & Lindell, M. K. (2005). Recent issues regarding rWG, r*WG rWG(J), and r*WG(J). Organizational Research Methods, 8, 128–137.

    Google Scholar 

  • LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815–852.

    Google Scholar 

  • *Lee, J. (2014). Moderation effect of safety climate variability on the relationship between safety climate level and safety behavior and its boundary conditions. Doctoral Dissertations. 504. https://opencommons.uconn.edu/dissertations/504

  • *Lee, L.-Y., & Vaesna, S. (2013). The effects of social exchange perspective on employee creativity: A multilevel investigation. Psychology Research, 3, 660–678.

    Google Scholar 

  • Li, X., Dusseldorp, E., Liu, K., Claramunt, J., & Meulman, J. (2019). Meta-CART: A flexible approach to identify moderators in meta-analysis. R package version 2.0–1. https://CRAN.R-project.org/package=metacart

  • Li, X., Dusseldorp, E., & Meulman, J. J. (2017). Meta-CART: A tool to identify interactions between moderators in meta-analysis. British Journal of Mathematical and Statistical Psychology, 70(1), 118–136.

    Google Scholar 

  • *Liao, H., & Chuang, A. (2004). A multilevel investigation of factors influencing employee service performance and customer outcomes. Academy of Management Journal, 47, 41–58.

    Google Scholar 

  • *Lin, C-P., Liu, C-M., Liu, N-T., & Huang, H-T. (2018). Being excellent teams: Managing innovative climate, politics, and team performance. Total Quality Management & Business Excellence.

  • Lindell, M. K., & Brandt, C. J. (2000). Climate quality and climate consensus as mediators of the relationship between organizational antecedents and outcomes. Journal of Applied Psychology, 85, 331–348.

    Google Scholar 

  • *Ling, Q., Liu, F., & Wu, X. (2017). Servant versus authentic leadership: Assessing effectiveness in China’s hospitality industry. Cornell Hospitality Quarterly, 58, 53–68.

    Google Scholar 

  • *Linuesa-Langreo, J., Ruiz-Palomino, P., & Elche-Hortelano, D. (2017). New strategies in the new millennium: Servant leadership as enhancer of service climate and customer service performance. Frontiers in Psychology, 8,. https://doi.org/10.3389/fpsyg.2017.00786

  • *Loh, M.Y., Idris, M.A., Dollard, M.F., & Isahak, M. (2018). Psychosocial safety climate as a moderator of the moderators: Contextualizing JDR models and emotional demands effects. Journal of Occupational and Organizational Psychology.

  • Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Addison-Wesley Publishing Company Inc.

    Google Scholar 

  • *Luria, G. (2008). Climate strength: How leaders from consensus. The Leadership Quarterly, 19, 42–53.

    Google Scholar 

  • *Luria, G., & Yagil, D. (2008). Procedural justice, ethical climate and service outcomes in restaurants. International Journal of Hospitality Management, 27, 276–283.

    Google Scholar 

  • *Mascherek, A. C., & Schwappach, D. L. B. (2017). Patient safety climate profiles across time: Strength and level of safety climate associated with a quality improvement program in Switzerland – A cross-sectional survey study. PLoS ONE, 12(7), e0181410. https://doi.org/10.1371/journal.pone.0181410

    Article  PubMed  PubMed Central  Google Scholar 

  • *Maynard, M. T., Mathieu, J. E., Marsh, W. M., & Ruddy, T. R. (2007). A multilevel investigation of the influences of employees’ resistance to empowerment. Human Performance, 20, 147–171.

    Google Scholar 

  • *McKay, P. F., Avery, D. R., & Morris, M. A. (2009). A tale of two climates: Diversity climate from subordinates’ and managers’ perspectives and their role in store unit sales performance. Personnel Psychology, 62, 767–791.

    Google Scholar 

  • *Menges, J. I., Walter, F., Vogel, B., & Bruch, H. (2011). Transformational leadership climate: Performance linkages, mechanisms, and boundary conditions at the organizational level. The Leadership Quarterly, 22, 893–909.

    Google Scholar 

  • *Mohammadi, B. (2016). Empowerment climate, supervisory support for creativity and work role performance in new product development settings: A team level investigation. Unpublished masters thesis, Queensland University of Technology: Brisbane, Australia.

  • *Molina, A., Moliner, C., Martínez-Tur, V., Cropanzano, R., & Peiró, J. M. (2016). Validating justice climate and peer justice in a real work setting. Journal of Work and Organizational Psychology, 32, 191–205.

    Google Scholar 

  • *Moon, K. (2016). Determinants and consequences of organizational justice climate in the U.S. federal government: An organizational level, longitudinal study. Unpublished doctoral dissertation, University of Georgia: Athens, GA.

  • *Morrison, E. W., Wheeler-Smith, S. L., & Kamdar, D. (2011). Speaking up in groups: A cross-level study of group voice climate and voice. Journal of Applied Psychology, 96, 183–191.

    Google Scholar 

  • Mumford, M. D., & Peterson, N. G. (1999). The O*NET content model: Structural considerations in describing jobs. In N. G. Peterson, M. D. Mumford, W. C. Borman, P. R. Jeanneret, & E. A. Fleishman (Eds.), An occupational information system for the 21st century: The development of O*NET (pp. 21–30). American Psychological Association.

    Google Scholar 

  • *Myer, A. T., Thoroughgood, C. N., & Mohammed, S. (2016). Complementary or competing climates? Examining the interactive effect of service and ethical climates on company-level financial Performance. Journal of Applied Psychology, 101, 1178–1190.

    Google Scholar 

  • *Naumann, S., & Bennett, N. (2002). The effects of procedural justice climate on work group performance. Small Group Research, 33, 361–377.

    Google Scholar 

  • *Naveh, E., & Katz-Navon, T. (2015). A longitudinal study of an intervention to improve road safety climate: Climate as an organizational boundary spanner. Journal of Applied Psychology, 100, 216–226.

    Google Scholar 

  • *Naveh, E., Katz-Navon, T., & Stern, Z. (2005). Treatment errors in healthcare: A safety climate approach. Management Science, 51, 948–960.

    Google Scholar 

  • *Neal, A., & Griffin, M. A. (2006). A study of the lagged relationships among safety climate motivation, safety behavior, and accidents at the individual and group levels. Journal of Applied Psychology, 91, 946–953.

    Google Scholar 

  • *Neal, A., West, M. A., & Patterson, M. G. (2005). Do organizational climate and competitive strategy moderate the relationship between human resource management and productivity? Journal of Management, 31, 492–512.

    Google Scholar 

  • Newman, D. A., & Sin, H. (2020). Within-group agreement (rWG): Two theoretical parameters and their estimators. Organizational Research Methods, 23, 30–64.

    Google Scholar 

  • *Ning, N., Wang, C.J., Zhaohong, C., & Zheng, C.Z. (2017). The direct and moderating effect of learning orientation on individual performance in the banking industry in China: Contextualization of high-performance work systems. Asia Pacific Journal of Human Resources.

  • *Ostroff, C., Kinicki, A., & Clark, M. (2002). Substantive and operational issues of response bias across levels of analysis: An example of climate-satisfaction relationships. Journal of Applied Psychology, 87, 355–368.

    Google Scholar 

  • *Parker, A. W., Tones, M. J., & Ritchie, G. E. (2017). Development of a multilevel health and safety climate survey tool within a mining setting. Journal of Safety Research, 62, 173–180.

    PubMed  Google Scholar 

  • Parr, N.J., Loan, C.M., & Tanner-Smith, E.E. (2021). Using machine learning to identify and investigate moderators of alcohol use intervention effects in meta-analyses. Alcohol and Alcoholism. agab036, https://doi.org/10.1093/alcalc/agab036

  • *Patras, L., Martínez-Tura, V., Estredera, Y., Gracia, E., Molinera, C., & Peiró, J. M. (2018). Organizational performance focused on users’ quality of life: The role of service climate and “contribution-to-others” wellbeing beliefs. Research in Developmental Disabilities, 77, 114–123.

    PubMed  Google Scholar 

  • Patterson, M. G., West, M. A., Shackleton, V. J., Dawson, J. F., Lawthom, R., Maitlis, S., Robinson, D. L., & Wallace, A. M. (2005). Validating the organizational climate measure: Links to managerial practices, productivity and innovation. Journal of Organizational Behavior, 26, 379–408.

    Google Scholar 

  • *Pecino-Medina, V., Díaz-Fúnez, P. A., & Mañas-Rodríguez, M. A. (2017). Climate, stress and satisfaction: A multilevel study in the public sector. International Journal of Social Psychology, 32, 52–79.

    Google Scholar 

  • *Peng, J. C., Jien, J.-J., & Lin, J. (2016). Antecedents and consequences of psychological contract breach. Journal of Managerial Psychology, 31, 1312–1326.

    Google Scholar 

  • *Prentice, C., Ma, E., & Wong, I.A. (2018). Performance driven outcomes—the case of frontline employees in the hospitality sector. Journal of Hospitality Marketing & Management.

  • *Raes, E., Decuyper, S., Lismont, B., Van den Bossche, P., Kyndt, E., Demeyere, S., & Dochy, F. (2013). Facilitating team learning through transformational leadership. Instructional Science, 41, 287–305.

    Google Scholar 

  • Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. Routledge.

    Google Scholar 

  • Rentsch, J. R. (1990). Climate and culture: Interaction and qualitative differences in organizational meanings. Journal of Applied Psychology, 75, 668–681.

    Google Scholar 

  • *Riordan, C. M., Vandenberg, R. J., & Richardson, H. A. (2005). Employee involvement climate and organizational effectiveness. Human Resource Management, 44, 471–488.

    Google Scholar 

  • Roberson, Q. M. (2006). Justice in teams: The effects of interdependence and identification on referent choice and justice climate strength. Social Justice Research, 19, 323–344.

    Google Scholar 

  • Ryan, A. M., McFarland, L., Baron, H., & Page, R. (1999). An international look at selection practices: Nation and culture as explanations for variability in practice. Personnel Psychology, 52, 359–391.

    Google Scholar 

  • *Salanova, M., Agut, S., & Peiro, J. M. (2005). Linking organizational resources and work engagement to employee performance and customer loyalty: The mediation of service climate. Journal of Applied Psychology, 90, 1217–1227.

    Google Scholar 

  • *Sandell, K.J. (2015). Climate for workplace fun in a retail setting. Unpublished doctoral dissertation, Colorado State University, Fort Collins, CO.

  • *Schechter, C., & Qadach, M. (2016). Principals’ learning mechanisms: Exploring an emerging construct, leadership and policy in schools. Leadership and Policy in Schools, 15, 141–167.

    Google Scholar 

  • *Schneider, B., & Barbera, K. M. (2014). Introduction: The Oxford handbook of organizational climate and culture. In B. Schneider & K. M. Barbera (Eds.), The Oxford handbook of organizational climate and culture (pp. 3–20). Oxford University Press.

    Google Scholar 

  • Schneider, B., Ehrhart, M. G., Mayer, D. M., Saltz, J. L., & Niles-Jolly, K. (2005). Understanding organization-customer links in service settings. Academy of Management Journal, 48, 1017–1032.

    Google Scholar 

  • Schneider, B., Gonzalez-Roma, V., Ostroff, C., & West, M. A. (2017). Organizational climate and culture: Reflections on the history of the constructs in the Journal of Applied Psychology. Journal of Applied Psychology, 102, 468–482.

    Google Scholar 

  • Schneider, B., Gunnarson, S. K., & Niles-Jolly, K. (1994). Creating the climate and culture of success. Organizational Dynamics, 23, 17–29.

    Google Scholar 

  • Schneider, B., & Reichers, A. E. (1983). On the etiology of climates. Personnel Psychology, 36, 19–39.

    Google Scholar 

  • *Schneider, B., White, S. S., & Paul, M. C. (1998). Linking service climate and customer perceptions of service quality: Test of a causal model. Journal of Applied Psychology, 83, 150–163.

    Google Scholar 

  • *Schulte, M., Ostroff, C., Shmulyian, S., & Kinicki, A. (2009). Organizational climate configurations: Relationships to collective attitudes, customer satisfaction, and financial performance. Journal of Applied Psychology, 3, 618–634.

    Google Scholar 

  • *Shen, J., & Benson, J. (2016). When CSR as a social norm: How socially responsible human resource management affects employee work behavior. Journal of Management, 42, 1723–1746.

    Google Scholar 

  • *Shen,Y., Zhang, R.P., Koh, Y., Rowlinson, S., & Leicht, R.M. (2017). The effects of group safety climate on construction personnel’s safety behavior: A cross-level investigation. In A. Mahalingam, T. Shealy, & N. Gil (Eds.), Working Paper Proceedings of the 15th Engineering Project Organization Conference, 1–11. California: Stanford Sierra Camp.

  • *Shepherd, W. J., Ployhart, R. E., & Kautz, J. (2020). The neglected role of collective customer perceptions in shaping collective employee satisfaction, service climate, voluntary turnover, and involuntary turnover: A cautionary note. Journal of Applied Psychology, 105, 1327–1337.

    Google Scholar 

  • *Shih, H.-A., Chiang, Y.-H., & Chen, T.-J. (2012). Transformational leadership, trusting climate, and knowledge-exchange behaviors in Taiwan. The International Journal of Human Resource Management, 23, 1057–1073.

    Google Scholar 

  • Simon, H. A. (1973). The organization of complex systems. In H. H. Pattee (Ed.), Hierarchy theory (pp. 1–27). Braziller.

    Google Scholar 

  • *Simons, T., & Roberson, Q. (2003). Why managers should care about fairness: The effects of aggregate justice perceptions on organizational outcomes. Journal of Applied Psychology, 88, 432–443.

    Google Scholar 

  • Smith-Crowe, K., Burke, M. J., Kouchaki, M., Signal, S. M. (2013) Assessing interrater agreement via the average deviation index given a variety of theoretical and methodological problems. Organizational Research Methods, 16(1), 127–151. https://doi.org/10.1177/1094428112465898

  • *Sowinski, D. R., Fortmann, K. A., & Lezotte, D. V. (2008). Climate for service and the moderating effects of climate strength on customer satisfaction, voluntary turnover, and profitability. European Journal of Work and Organizational Psychology, 17, 73–88.

    Google Scholar 

  • *Spell, H. B., Eby, L. T., & Vandenberg, R. J. (2014). Developmental climate: A cross-level analysis of voluntary turnover and job performance. Journal of Vocational Behavior, 84, 283–292.

    PubMed  PubMed Central  Google Scholar 

  • *Spielfogel, J. E., Leathers, S. J., & Christian, E. (2016). Agency culture and climate in child welfare: Do perceptions vary by exposure to the child welfare system? Human Service Organizations: Management, Leadership & Governance, 40, 382–396.

    Google Scholar 

  • Strobl, C., Malley, J., & Tutz, G. (2009). An introduction to recursive partitioning: Rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychological Methods, 14, 323–348.

    PubMed  PubMed Central  Google Scholar 

  • *Tang, P. M. (2016). Ethical leadership in social enterprises: Multilevel investigation of its influence on team and individual prosocial voice. Unpublished masters thesis, Lingnan University: Hong Kong. Retrieved from http://commons.ln.edu.hk/mgt_etd/27

  • *Tang, C., & Naumann, S. (2016). Team diversity, mood, and team creativity: The role of team knowledge sharing in Chinese R&D teams. Journal of Management & Organization, 22, 420–434.

    Google Scholar 

  • Taras, V., Kirkman, B. L., & Steel, P. (2010). Examining the impact of culture‘s consequences: A three-decade, multi-level, meta-analytic review of Hofstede‘s cultural value dimensions. Journal of Applied Psychology, 95(3), 405–439.

    Google Scholar 

  • *Tesler, R. (2013). Alignment is not just for tires: The impact of leader-follower climate perceptions on learning and safety behaviors. Unpublished doctoral dissertation, The Pennsylvania State University, State College, PA.

  • *Tews, M. J., Hoefnagels, A., Jolly, P. M., & Stafford, K. (2021). Turnover among young adults In the hospitality industry: Examining the impact of fun in the workplace and training climate. Employee Relations: The International Journal, 43, 245–261.

    Google Scholar 

  • Therneau, T. M., & Atkinson, B. (2017). An introduction to recursive partitioning using the RPART routines. Mayo Foundation.

    Google Scholar 

  • *Tremblay, M. (2017). Humor in teams: Multilevel relationships between humor climate, inclusion, trust, and citizenship behaviors. Journal of Business and Psychology, 32, 363–378.

    Google Scholar 

  • *Tremblay, M., Gaudet, M. C., & Parent-Rocheleau, X. (2018). Good things are not eternal: How consideration leadership and initiating structure influence the dynamic nature of organizational justice and extra-role behaviors at the collective level. Journal of Leadership & Organizational Studies, 25, 211–232.

    Google Scholar 

  • *Tse, H. H. M., Dasborough, M. T., & Ashkanasy, N. M. (2008). A multi-level analysis of team climate and interpersonal exchange relationships at work. The Leadership Quarterly, 19, 195–211.

    Google Scholar 

  • Tung, R. L., & Baumann, C. (2009). Comparing the attitudes toward money, material possessions and savings of overseas Chinese vis-a`-vis Chinese in China: Convergence, divergence or cross-vergence, vis-a`-vis “one size fits all” human resource management policies and practices. International Journal of Human Resource Management, 20, 2382–2401.

    Google Scholar 

  • Tung, R. L., & Verbeke, A. (2010). Beyond Hofstede and GLOBE: Improving the quality of cross-cultural research. Journal of International Business Studies, 41, 1259–1274.

    Google Scholar 

  • *a Van De Voorde, K., Van Veldhoven, M., & Paauwe, J. (2010). Time precedence in the relationship between organizational climate and organizational performance: A cross-lagged study at the business unit level. The International Journal of Human Resource Management, 21, 1712–1732.

    Google Scholar 

  • *a Van De Voorde, K., Van Veldhoven, M., & Paauwe, J. (2014). Relationships between work unit climate and labour productivity in the financial sector: A longitudinal test of the mediating role of work satisfaction. European Journal of Work and Organizational Psychology, 23(2), 295–309.

    Google Scholar 

  • van Lissa, C. J. (2020). Small sample meta-analyses: Exploring heterogeneity using MetaForest. In R. van de Schoot & M. Miocevic (Eds.), Small sample size solutions: A guide for applied researchers and practitioners (pp. 186–202). Routledge.

    Google Scholar 

  • *Van Woerkom, M., & Meyers, M. (2015). My strengths count! Effects of a strengths-based psychological climate on positive affect and job performance. Human Resource Management, 54, 81–103.

    Google Scholar 

  • *Vera, M., Martınez, I. M., Lorente, L., & Chambel, M. A. (2016). The role of co-worker and supervisor support in the relationship between job autonomy and work engagement among Portuguese nurses: A multilevel study. Social Indicators Research, 12, 1143–1156.

    Google Scholar 

  • Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48.

    Google Scholar 

  • *Vogus, T. J., Cull, M. J., Hengelbrok, N. E., Modell, S. J., & Epstein, R. A. (2016). Assessing safety culture in child welfare: Evidence from Tennessee. Children and Youth Services Review, 65, 94–103.

    Google Scholar 

  • *Wallace, C., & Chen, G. (2006). A multilevel integration of personality, climate, self-regulation, and performance. Personnel Psychology, 59, 529–557.

    Google Scholar 

  • Wallace, J. C., Edwards, B. D., Paul, J., Burke, M. J., Christian, M., & Eissa, G. (2016). Change the referent? A meta-analytic investigation of direct and referent-shift consensus models for organizational climate. Journal of Management, 42, 838–861.

    Google Scholar 

  • *Wallace, J. C., Johnson, P. D., Mathe, K., & Paul, J. (2011). Structural and psychological empowerment climates, performance, and the moderating role of shared felt accountability: A managerial perspective. Journal of Applied Psychology, 96, 840–850.

    Google Scholar 

  • *Walumba, F. O., Peterson, S. J., Avolio, B. J., & Hartnell, C. A. (2010). An investigation of the relationships among leader and follower psychological capital, service climate, and job performance. Personnel Psychology, 63, 937–963.

    Google Scholar 

  • *Wang, P., & Rode, J. C. (2010). Transformational leadership and follower creativity: The moderating effects of identification with leader and organizational climate. Human Relations, 63, 1105–1128.

    Google Scholar 

  • *Wang, Z., Chen, Y.-f.N., Tjosvold, D., & Shi, K. (2010). Cooperative goals and team agreeableness composition for constructive controversy in China. Asia Pacific Journal of Management, 27, 139–153.

    Google Scholar 

  • *Williams, N. J., Glisson, C., Hemmelgarn, A., & Green, P. (2017). Mechanisms of change in the ARC organizational strategy: Increasing mental health clinicians’ EBP adoption through improved organizational culture and capacity. Administration Policy in Mental Health, 44, 269–283.

    PubMed  Google Scholar 

  • *Winkler, R.B. (2004). Organizational climate in a health care setting and its relationship to patient safety. Unpublished dissertation, DePaul University, Chicago, IL.

  • *Wood, S., Stride, C., & Johnson, S. (2012). Personal and team morale in health services: Distinct and measurable concepts. Journal of Health Management, 14, 535–557.

    Google Scholar 

  • *Woznyj, H.M., Heggestad, E.D., Kennerly, S., & Yap, T.L. (2018). Climate and organizational performance in long-term care facilities: The role of affective commitment. Journal of Occupational and Organizational Psychology.

  • Yohannes, Y., & Webb, P. (1999). Classification and regression trees, CART: A user manual for identifying indicators of vulnerability to famine and chronic food insecurity (No. 593–2016–39922). Washington, D.C.: International Food Policy Research Institute (IFPRI).

  • *Yu, T., de Ruyter, K., Patterson, P., & Chen, C.-F. (2018). The formation of a cross-selling initiative climate and its interplay with service climate. European Journal of Marketing, 52, 1457–1484.

    Google Scholar 

  • *Yusof, N.A.M., Jusoh, W.J.W., & Hashim, K.S.H. (2020). Service climate and service behaviour to customer satisfaction: A multilevel study of Muslim-friendly hotel. Journal of Halal Industry & Services, 3 (Special Issue), 1–6.

  • *Zappala, S., Martinez-Tur, V., & Mariani, M. G. (2018). Service climate in organizations: Validating the Italian version of the service climate scale (ISCS). TPM, 25, 5–20.

    Google Scholar 

  • *Zhang, Y., & Begley, T. M. (2011). Perceived organizational climate, knowledge transfer and innovation in China-based research and development companies. The International Journal of Human Resource Management, 22, 34–56.

    Google Scholar 

  • *Zhang, X., & Kwan, H. K. (2018). Team learning goal orientation and innovation: Roles of transactive memory system and task interdependence. GSL Journal of Business Management and Administration Affairs, 1(109), 1–10.

    Google Scholar 

  • *Zhang, N., & Zhang, J. (2016). Chinese insurance agents in “bad barrels”: A multilevel analysis of the relationship between ethical leadership, ethical climate and business ethical sensitivity. Springerplus, 5, 2078.

    PubMed  PubMed Central  Google Scholar 

  • Zohar, D., & Tenne-Gazit, O. (2008). Transformational leadership and group interaction as climate antecedents: A social network analysis. Journal of Applied Psychology, 93, 744–757.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Michael J. Burke.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 4

Table 4 Illustration of variations in context and methodology associated with the reporting of mean rWG

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Burke, M.J., Smith-Crowe, K., Burke, M.I. et al. The Relative Importance and Interaction of Contextual and Methodological Predictors of Mean rWG for Work Climate. J Bus Psychol (2022). https://doi.org/10.1007/s10869-021-09789-6

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10869-021-09789-6

Keywords

  • Within-group agreement
  • rWG
  • CART
  • Multilevel research
  • Meta-analysis