WHAT IS COMMON METHOD VARIANCE?

A large number of papers submitted to JIBS use data collected from a survey instrument. When self-report questionnaires are used to collect data at the same time from the same participants, common method variance (CMV) may be a concern. This concern is strongest when both the dependent and focal explanatory variables are perceptual measures derived from the same respondent (Podsakoff & Organ, 1986).

CMV is “variance that is attributable to the measurement method rather than to the constructs the measures represent” (Podsakoff, MacKenzie, Lee, & Podsakoff, 2003: 879). CMV creates a false internal consistency, that is, an apparent correlation among variables generated by their common source. For example, this could occur if a researcher asks respondents to evaluate an MNE's organizational capabilities and the firm's international performance in the same survey. In such cases, self-report data can create false correlations if the respondents have a propensity to provide consistent answers to survey questions that are otherwise not related. Thus, common methods can cause systematic measurement errors that either inflate or deflate the observed relationships between constructs, generating both Type I and Type II errors.

Scholarly views of CMV differ. Campbell (1982: 692), a former editor of the Journal of Applied Psychology, provides a strongly negative assessment (italics added): “If there is no evident construct validity for the questionnaire measure or no variables that are measured independently of the questionnaire, I am biased against the study and believe that it contributes very little”. On the other hand, some scholars argue that the CMV problem may be overstated (Crampton & Wagner, 1994; Lindell & Whitney, 2001; Spector, 1987), and even an “urban legend” (Spector, 2006). A recent exhaustive review of research on CMV in behavioral research reaches a more balanced conclusion: “common method variance is often a problem and researchers need to do whatever they can to control for it” (Posdsakoff et al., 2003: 900).

The specific details of the research methodology are clearly relevant in determining the likelihood and degree of common method bias. Podsakoff et al. (2003) explore four general sources of CMV: the use of a common rater, the manner in which items are presented to respondents, the context in which items on a questionnaire are placed, and the contextual influences (time, location and media) used to measure the constructs.Footnote 1 Some CMV sources may be more problematic than others, for example, perceptual data from single raters may be more worrisome than the manner in which items are presented in the survey instrument. A manuscript that suffers from more potential sources of CMV should, in general, be more problematic than one with fewer sources. The most worrisome example of CMV, according to the authors (2003: 885), occurs when “the data for both the predictor and criterion variable are obtained from the same person in the same measurement context using the same item context and similar item characteristics”.

The JIBS editors see CMV as a potentially serious concern for researchers using survey-based data,Footnote 2 especially where the dependent and independent variables are perceptual and from the same source.Footnote 3 JIBS editors and reviewers must therefore assess not only whether CMV exists, but also how likely it is to generate Type I and Type II errors. Our position is to inform potential contributors that they should, prior to journal submission, deal with potential CMV biases in their research both seriously and explicitly.

HOW CAN CMV BE ADDRESSED BY RESEARCHERS?

In general, four approaches have been recommended in the literature as methods that researchers should use to avoid or correct CMV (see, for example, Podsakoff et al., 2003):

  1. 1

    The obvious strategy is, of course, to avoid any potential CMV in the research design stage by using other sources of information for some of the key measures. In particular, if possible, the dependent variable should be constructed using information from different sources than the independent variables.

  2. 2

    A number of procedural remedies in designing and administering the questionnaire, from mixing the order of the questions to using different scale types, can reduce the likelihood of CMV.

  3. 3

    Complicated specifications of regression models reduce the likelihood of CMV. Specifically, respondents are unlikely to be guided by a cognitive map that includes difficult-to-visualize interaction and non-linear effects. This is less likely the more complicated the model.

  4. 4

    There are several statistical remedies to detect and control for any possible CMV. A post hoc Harman one-factor analysis is often used to check whether variance in the data can be largely attributed to a single factor. Additionally, other statistical procedures can be applied to partial out common factors or to control for them.

Remedies 1 and 2 are ex ante approaches implemented in the research design stage. Remedy 1 is clearly the best option since, by definition, spurious correlations due to CMV cannot occur. Remedy 2 can also ex ante reduce the likelihood of the consistency motive and theory-in-use biases in the informant responses. Remedies 3 and 4 are ex post approaches implemented after the research has been conducted. Remedy 3 implies that the likelihood of CMV can be reduced by specifying complex relationships that are unlikely to be part of the respondents’ cognitive maps, while remedy 4 corrects for CMV through a variety of statistical procedures.

We first discuss these four basic remedies below and then offer our advice to international business scholars on how to handle the possibility of CMV in their research.

REMEDIES 1 AND 2: AVOID CMV IN THE EX ANTE RESEARCH DESIGN STAGE

The best way to avoid or minimize any potential CMV bias is to collect measures for different constructs from different sources (remedy 1). Ideally, the dependent variable(s) are collected from a different source than the independent variables are collected from.Footnote 4 For example, subsidiary-level autonomy or decision-making power of a subsidiary-level manager could be measured with a local survey, whereas a subsidiary performance measure could be constructed from information provided by the multinational headquarters or from a local archival source. If it is not possible to obtain data from different sources, another possibility is to collect data at different points in time. Preferably, all this is part of the ex ante research design. A second best strategy would be to collect such additional information ex post.

Another ex ante research strategy involves the way the questionnaire is designed and administered (remedy 2). Respondents should be assured of the anonymity and confidentiality of the study, that there are no right or wrong answers, and that they should answer as honestly as possible.Footnote 5 Moreover, more fact-based questionnaire items are less likely to be associated with CMV. Podsakoff et al. (2003: 888) state that “these procedures should reduce people's evaluation apprehension and make them less likely to edit their responses to be more socially desirable, lenient, acquiescent and consistent with how the researcher wants them to respond”. Additionally, great care must be taken to systematically examine the construction of items so as to ensure that ambiguous, vague and unfamiliar terms are not included, and that the questionnaire as a whole and the individual items are formulated as concisely as possible (see Harrison, McClaughlin, & Coalter, 1996; Lindell & Whitney, 2001; Podsakoff et al., 2003). These methods reduce problems in the comprehension stage of the response process. Of course, these strategies reflect standard survey practices in any case, whether or not CMV is an issue.

Specifically related to CMV, Podsakoff et al. (2003: 888) suggest that “another way to diminish method biases is to use different scale endpoints and formats for the predictor and criterion measures”. This should reduce method biases caused by commonalities in scale endpoints and anchor effects. In a recently published JIBS article on the international location decision, for instance, Galan, González-Benito, and Zuñiga-Vincente (2007) measured the dependent variable in two different ways so as to avoid potential CMV with independent variables that were operationalized as perception-based measures.

Moreover, counterbalancing the order of questions relating to different scales and constructs makes CMV less likely, as the respondent cannot then easily combine related items to cognitively “create” the correlation needed to produce a CMV-biased pattern of responses (Murray, Kotabe, & Zhou, 2005). One option is to randomize the order of the questions using survey software. For instance, in a JIBS article examining the influence of parent control on conflict in international joint ventures, Barden, Steensma, and Lyles (2005) used different response anchors across measured constructs, separated measurements in time, and manipulated the order of questionnaire items in such a way that CMV across dependent, independent and control variables became very unlikely.

REMEDIES 3 AND 4: DEAL WITH CMV IN THE EX POST STATISTICAL ANALYSES

CMV is more likely to emerge in models that are overly simple. A third approach is therefore to specify relationships among the dependent and independent variables that are not so simple that these relationships are likely to be part of the individual raters’ cognitive maps. In this context, Harrison et al. (1996: 248) refer to the cognitive miser principle. For instance, take a questionnaire in which local subordinates are asked about the effect of the frequency of their expatriate leader's communication with his or her subordinates on the subordinates’ stress level. Suppose that the theory predicts that this effect is non-linearly moderated by the subordinates’ locus of control trait. Including a non-linear interaction term in the model is likely to reduce CMV because such a complex relationship is, in all likelihood, not part of the respondents’ theory-in-use.

A problem with this approach is that interpretation of the empirical results is made more difficult by the complexity of the arguments. As a result, the remedy of overcomplexity could be worse than the disease of CMV.Footnote 6 Basically, adding complexity such as mediating, moderating and/or non-linear effects makes sense only if guided by a good theory. In the end, sound theory that directs design and method is, of course, the bottom line that characterizes all good research, be it survey-based or not.

The fourth remedy is to apply ex post statistical approaches. Indeed, there are quite a few of them; here, we only briefly refer to some of the more popular ones since there are several other papers with more details (please refer to the references attached to this Letter). Perhaps the most common but ineffective response by authors to address CMV (other than ignoring it) is to rely on Harman's single-factor test to assert that their research is not pervasively affected by CMV. This method loads all items from each of the constructs into an exploratory factor analysis to see whether one single factor does emerge or whether one general factor does account for a majority of the covariance between the measures; if not, the claim is that CMV is not a pervasive issue. However, Podsakoff et al. (2003) explain that this claim is likely to be incomplete because Harman's test is insensitive. It is unlikely that a single-factor model will fit the data, and there is no useful guideline as to what would be the acceptable percentage of explained variance of a single-factor model. The JIBS team therefore believes that simply reporting seemingly reassuring outcomes from Harman's single-factor test is insufficient to prove that CMV is not a pervasive issue.

Lindell and Whitney (2001), Podsakoff et al. (2003) and Malhotra, Kim, and Patil (2006) review several statistical methods that are more sophisticated than Harman's test, which can be used to test and possibly control for CMV. Different statistical remedies are available for different types of research settings and different sources of CMV. Promising statistical remedies include a partial correlation procedure and a direct measure of a latent common method factor. The former method partials out the first unrotated factor from the exploratory factor analysis, and then continues to determine whether the theoretical relationships among the variables of interest do still hold. The latter method allows questionnaire items to load on their theoretical constructs, as well as on a latent CMV factor, and examines the significance of theoretical constructs with or without the common factor method. Both methods have their own limitations, however, one of which is the assumption that the sources of CMV can be well identified and validly measured.Footnote 7 A recommended solution is to use multiple remedies, not just one remedy, in order to assuage the various concerns about CMV.

COULD CMV BE A PROBLEM IN JIBS?

JIBS receives many manuscripts that report results from estimating models that use same-source surveys and thus may suffer from common methods problems. However, while psychology journals have worried about CMV since the early 1980s, and have implemented techniques to handle this problem since at least the mid-1990s, the common methods issue has been slow to be recognized, and is much less addressed, by international business scholars. Best practices in the “macro” disciplines appear to have lagged behind those in the “micro” disciplines, probably because the micro areas (e.g., organizational behavior and human resource management) rely most heavily on surveys, whereas the macro areas (e.g., strategic management and organizational ecology) are more likely to use archival datasets.

In March 2009, we reviewed all the articles published in JIBS between 2000 and the present for evidence of potential sources of CMV. Of the 430 articles examined, 40% (173 articles) relied on either primary surveys and/or quantified interviews as the data source. The 173 articles were then characterized by potential sources of common methods bias. Almost all the articles (167 articles) contained one or more sources of CMV, and most appeared to have multiple sources. Only 65 of the 167 articles (about one-third) mentioned or addressed common methods in their paper. Of this group, half the articles (32) used Harman's single-factor test or something similar to test for CMV. Fifteen articles used another approach. Only nine articles used both Harman's test and at least one other correction method to control for CMV.

These statistics, of course, tell us only how frequently common methods appear in recently published JIBS articles, not the magnitude of the potential bias from CMV in these articles. Previous research estimating the magnitude of the effects did not include JIBS articles; see for example, Doty and Glick (1998) and Cote and Buckley (1987). So, the most we can say is there may be a problem based on frequency of usage of common methods, but at present we have no estimates of the magnitude of the problem.

Based on this short survey, it appears that common method bias has not been recognized nor addressed by most IB scholars, even in JIBS, the top journal in the field of international business. We recognize, of course, that standards for rigor in empirical work are continually rising. What were acceptable methodological practices even five years ago can easily and rapidly become unacceptable as social science scholars better understand the limitations of their empirical techniques and develop more rigorous methods for identifying and correcting for potential biases in their work. The purpose of our Letter from the Editors is therefore not to criticize earlier research, but rather to encourage IB scholars to implement current best practices in research methods. We argue that the hurdle barrier must now be set higher in JIBS vis à vis CMV. It is time for IB scholars to address, and reduce or offset where feasible, the use of common methods in their empirical work.

RECOMMENDATIONS

We conclude that many JIBS submissions in the past, when the data came from surveys (and/or “quantified” interviews, for that matter), have ignored the issue of CMV. Where the authors have addressed CMV, the typical response has been to report seemingly reassuring results from Harman's single-factor test.

The JIBS editors believe this approach has been insufficient. For the current editorial team, it is now standard practice to return a manuscript to the author when it appears to suffer from common method bias and the issue has been ignored in the manuscript. The desk rejection letter asks the author to perform validity checks and resolve any CMV issues before resubmitting the manuscript.

Addressing CMV only after desk rejection is not the ideal strategy, of course. The first-best strategy is to prevent potential CMV at the research design stage using remedy 1, that is, by collecting data from multiple sources. Ex ante, before running any analyses, the collection of key information from other sources should be planned, using where possible archival data and multiple respondents. Alternatively, additional information can be collected afterwards. An example of remedy 1 is Carraher, Sullivan, and Crocitto (2008), who surveyed expatriate employees but obtained their expat performance measure from company records filed by the individual's supervisor rather than from the expat respondents themselves.

In addition, we recommend that the survey questionnaire be carefully designed, applying all or a large subset of the procedural remedies listed above (remedy 2). Depending upon the nature of the questionnaire, tailor-made CMV measures can be included, a well-known example being social desirability scales. Ex post, in the empirical stage, options are to run Harman's single-factor test, specify a complex model (including interaction and non-linear terms), and explicitly control for or partial out CMV statistically (remedies 3 and 4). The options are summarized in Figure 1.

Figure 1
figure 1

Approaches for handling common method variance (CMV).

Full size image

More often than not, a perfect solution is out of reach. If ex ante methods are not doable, the JIBS editors recommend that IB scholars use multiple ex post procedural remedies including possibly a more complex model specification, and partialing out or controlling for CMV (remedies 2, 3 and 4). What we ask for is that CMV-related methodological issues should be discussed carefully and explicitly in any manuscript submitted to JIBS that uses single-respondent data. This will often imply the need to apply a number of the remedies referred to above. While the problems with CMV were not well understood by IB scholars in the past and, as a result, there were many JIBS articles published that might suffer from multiple sources of common methods bias, the standards have changed and IB scholars must adopt current best practices.

Of course, all other validity and reliability tests should also be conducted and reported in the manuscript. In singling out CMV in this Letter from the Editors we do not want to give JIBS authors the impression that they should ignore other validity and reliability tests of their quantitative and qualitative research methods – they should not. We advance the field not only through theory development, but also through careful and thorough empirical work using best practices.

As international business researchers, we also recognize that sometimes common methods cannot be totally avoided, for example, if the research probes into difficult waters where data of any kind are scarce such as in severely understudied parts of the world (Africa, the Middle East), or where the research undertaken is so novel or insightful that this may be considered over standard methodological considerations. Editors and reviewers should not reject innovative manuscripts that push the boundaries of our knowledge of international business solely on the grounds of common methods. In such situations, the gains in creativity and impact might outweigh the loss of methodological purity.

Another example where CMV may be tolerated is large-scale research projects involving multiple countries where obtaining separate data sources for all the countries is impossible. Such large-scale projects typically deal with potential national differences in response bias by using within-subject standardization, mean centering the scores of individuals, or using regression approaches to control for national differences in response biases. These studies also typically link some sort of aggregate nation scores to a separate source archival predictor or criterion. Cross-country comparative studies where the intent of the study is to treat cognitive structures and processes of a population as a dependent variable might well want to use same-source correlations as a good indicator of shared aspects of cognition, with a national indicator as a separate-source predictor of these same-source correlations.

Lastly, we want to make it clear to the JIBS community that it is not our intent in this Letter from the Editors to privilege IB researchers who use large, readily available datasets such as Compustat, Orbis or SDC Platinum. Rather, we want to encourage primary and qualitative research in international business – including surveys – but at the same time increase awareness among IB researchers of potential CMV biases so that they can be avoided in the design stage. Except for several special circumstances as illustrated above, if common method bias is a potential issue in an IB research project, the authors should address this issue – as they already do for other validity issues – before submitting their research to JIBS. We are looking for more careful examination from all IB researchers in proving the robustness of their results, and hope this Letter from the Editors will help raise the quality standards for empirical research by future IB scholars.