- 233 Downloads
The term “nomological net” has been coined in the seminal paper by Cronbach and Meehl (1955) on construct validity (see also American Psychological Association 1954). Cronbach and Meehl introduced the idea of construct validity to validate theoretical attributes or qualities (i.e., constructs) for which there is no adequate criterion or which cannot be defined operationally, for example, personality traits or intelligence. The concept of construct validity as defined by Cronbach and Meehl did not only refer to measures of constructs, as did the earlier validity concepts of content validity or predictive validity, but intertwined the construct validation of measures with theory testing. According to construct validity theory, a construct is implicitly defined by its position in a network of other constructs that is deduced from theory and based on scientific laws – the “nomological net” (Cronbach and Meehl 1955). The laws in the nomological net or network (nomological: Greek for lawful) relate different constructs to each other (i.e., theoretical relations), at least some of these constructs to observations and the different observations to each other (i.e., empirical relations). In a nutshell, a nomological network can be understood as a system of scientific laws that relates constructs to each other and to observations. Campbell and Fiske (1959) extended construct validity theory by pointing out that theoretical as well as empirical relations should not only focus on convergent validity of related attributes but also on discriminant validity of unrelated attributes.
The Nomological Net Idea in Behavioral Science
The concept of the nomological network has been highly influential in research in the behavioral sciences and is still widely used. For example, a recent PsychINFO search in May 2016 revealed 655 journal articles published in the new millennium in the field of differential or personality psychology and assessment that apply the terms “nomological net” or “nomological network” (Search string: “nomological net” OR “nomological network” AND “personality differences” OR “individual differences” OR “differential psychology” OR “measurement” OR “assessment” OR “diagnosis” OR “testing” OR “psychometrics”). Most of these publications, however, are not concerned with the concept of the nomological network per se but rather use this term to frame their research (e.g., “investigation of the nomological network of construct x”). Scientific research and debates on the concept of the nomological network as such have typically been motivated to clarify the concept of construct validity and the practice of construct validation.
If constructs are defined by their position in a nomological net, the availability of such a lawful network of relations deduced from theory is a precondition for construct validation of measures and theories. Psychological theories often lack this theoretical precision. This has led to a dissociation between construct validity qua theory and the practice of construct validation (Brennan 2013), a weakening of the theory testing part of construct validity (Colliver et al. 2012), and a renewed discussion of the concept of validity as such (Borsboom et al. 2004; Embretson 2007; Newton and Shaw 2013; Special Issue on Validity of the Journal of Educational Measurement, 2013, 50/1).
Nomological Nets and Construct Validation
The nomological network idea provides no framework for addressing practical validation issues. Nevertheless, it helps to refine the construct validation process. Specifically, Cronbach (1988) contrasted programs of strong and weak construct validation. Strong programs are based on fully developed formal theories (i.e., nomological nets) and deductive theory testing, while weak programs are based on less developed theories that – put to the extreme – would allow interpreting any relation as validation evidence (“anything goes”). Strong and weak programs combine in construct validation in “an iterative process in which tests of partially developed theories provide information that leads to theory refinement and elaboration, which in turn provides a sounder basis for subsequent construct and theory validation research” (Strauss and Smith 2009, p. 9; see already Cronbach and Meehl 1955, for a discussion of these top-down and bottom-up processes in construct validation). In doing so, construct validation becomes an open-ended process in which validity is an overall evaluative judgment of the degree to which theoretical arguments and empirical findings support the plausibility and appropriateness of interpretations and uses of test scores (Messick 1995). Kane (2001, 2013) offered a pragmatic argument-based approach to construct validation that should avoid the extremes of the strong and weak program, thus, fitting better to actual research practice. In the argument-based approach, construct validity is established through theoretical and empirical evidence for a specific and clearly proposed use or interpretation of a measure instead of rigorous theory or nomological network testing (cf. Kane 2013).
In construct validation studies, convergent and discriminant relations are typically reported as correlations; researchers rarely refer to logical arguments or experimental results. Correlations can be estimated within a latent variable framework, e.g., by confirmatory factor analysis or structural equation modeling. When using such confirmatory methods, the nomological network idea guides psychological research in differential and personality psychology by pinpointing the importance of theory in the formulation of hypotheses about convergent and discriminant construct relations, by linking constructs to observations, by distinguishing between latent relations and observed relations, by distinguishing between conceptual and empirical overlap, or by distinguishing between theoretically and operationally defined constructs. There are various tools available to visually display network relations (e.g., Epskamp et al. 2012), as well as methods to evaluate construct validity based on convergent and discriminant construct relations (Westen and Rosenthal 2003).
First, convergent validity arguments are frequently based on correlations. High correlations of two or more measures of the same construct are interpreted in support of the convergent validity of a measure. However, no consensus has yet been reached on what constitutes a high enough correlation or how to deal with inconsistent correlational findings. Correlations are also influenced by the psychometric properties of a measure, the sample, or the method of assessment (e.g., tests, self-report). Further, for construct validation, there is the need to differentiate between the level of observations and the level of constructs. These aspects are not consistently taken into account in current validation studies (Schweizer 2012). Taken together, these issues undermine the idea of convergent validity as a vague and somewhat indetermined concept (Schweizer 2012).
Second, when measures of different constructs are not meaningfully correlated, this is typically interpreted as supporting the discriminant validity of these measures. However, many validation studies lack a clear theoretical rationale for selecting constructs for discriminant relations (Ziegler et al. 2013). Frequently, theoretically unrelated constructs are chosen. However, to strongly support the construct validity of measures of (new) constructs, it is most informative to investigate relations between different but closely related constructs (Shaffer et al. 2015; Ziegler et al. 2013). And again, there is the need to differentiate between the level of observations and the level of constructs (see Shaffer et al. 2015, for a guideline for conducting a discriminant validation study that takes these aspects into account).
Third, the nomological net relates theoretical constructs to observations assessed with a certain method. Methods refer to key factors that define the measurement process. That is, so-called method factors (e.g., rater response styles, characteristics of the item wording, high- vs. low-stakes measurement contexts) may introduce systematic variance over and above variance attributable to the target construct. Method factors may threaten the construct validity of a measure, particularly, because method variance has been estimated to make up between 18 and 32 percent of the total item variance (Podsakoff et al. 2012). Further, the nature of method variance remains elusive as theories explaining the phenomena producing method variance are scarce (Ziegler et al. 2013). Podsakoff et al. (2012) present an overview of procedural and statistical approaches that may help to minimize the impact of method variance.
Fourth, Embretson (1983) differentiates two components of construct validity: nomothetic span and construct representation. While nomothetic span comprises convergent and discriminant relations of a measure, construct representation refers to a cognitive theory that explains response behavior for that measure. Nomological nets include laws that relate constructs to observations, that is, construct representation; however, most studies that use the nomological network idea focus on convergent and discriminant relations or nomothetic span and neglect construct representation. But if we lack a theory of response behavior, that is, if we cannot explain our data, an important precondition for interpreting nomothetic span is missing. Borsboom et al. (2004) therefore argue for a shift to an attribute-based view of measurement that assigns validity to a measure only if theoretical and empirical arguments support the assumption that an attribute causes the measurement outcomes. In this respect, rational or theory-based item and test construction as well as scaling and scoring of test behavior become of paramount importance (Brennan 2013).
The idea of the nomological net was introduced to guide construct validation. To this end, the network in its strong form specifies the laws that explain to what extent and why theoretical constructs are related with each other and with corresponding measures. In its strong form, the network also informs on the circumstances (i.e., moderator variables) when these relations can or cannot be observed. Given its iterative nature, the nomological network idea underscores that theory development hinges on both clear construct definitions (see Podsakoff et al. 2016, for guidelines) and the development of excellent measures.
- American Psychological Association. (1954). Technical recommendations for psychological tests and diagnostic techniques. Psychological Bulletin, 51(2, Suppl.).Google Scholar
- Brennan, L. R. (2013). Commentary on “Validating the Interpretations and Uses of Test Scores”. Journal of Educational Measurement (Special Issue: Validity), 50, 74–83. doi:10.1111/jedm.12001.Google Scholar
- Cronbach, L. J. (1988). Five perspectives on the validity argument. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 3–17). Hillsdale: Erlbaum.Google Scholar
- Embretson, S. E. (2007). Construct validity: A universal validity system or just another test evaluation procedure? Educational Researcher, 36, 449–455. doi:10.3102/0013189X07311600.Google Scholar
- Epskamp, S., Cramer, A. O. J., Waldorp, L. J., Schmittmann, V. D., & Borsboom, D. (2012). qgraph: Network visualizations of relationships in psychometric data. Journal of Statistical Software, 48, 1–18. doi:10.18637/jss.v048.i04.Google Scholar
- Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2016). Recommendations for creating better concept definitions in the organizational, behavioral, and social sciences. Organizational Research Methods. doi:10.1177/1094428115624965. Published online before print.Google Scholar
- Shaffer, J. A., DeGeest, D., & Li, A. (2015). Tackling the problem of construct proliferation: A guide to assessing the discriminant validity of conceptually related constructs. Organizational Research Methods. doi:10.1177/1094428115598239. Published online before print.Google Scholar