Abstract
There has been growing criticism of the established practice of automatically including control variables into analyses, especially with survey studies. Several authors have explained the pitfalls of improper use and have provided some best practice advice. I build upon this foundation in suggesting a programmatic approach to the use of control variables that can provide evidence to support or refute feasible explanations for why two or more variables are related. The hierarchical iterative control (HIC) approach begins by establishing a connection between two or more variables and then hierarchically adds control variables to rule in or out their possible influence. The HIC approach involves conducting a series of studies to iteratively test relationships among target variables, utilizing a variety of control variable strategies involving multiple methods. A 7-step programmatic approach is described beginning with development of the research question and background literature review and then conducting empirical tests in a hierarchical (within a study) and iterative (across studies) manner.
Similar content being viewed by others
Notes
Thanks to one of my reviewers for suggesting this.
References
Atinc, G. M., Simmering, M. J., & Kroll, M. J. (2012). Control variable use and reporting in macro and micro management research. Organizational Research Methods, 15(1), 57–74. https://doi.org/10.1177/1094428110397773.
Becker, T. E. (2005). Potential problems in the statistical control of variables in organizational research: A qualitative analysis with recommendations. Organizational Research Methods, 8(3), 274–289.
Becker, T. E., Atinc, G., Breaugh, J. A., Carlson, K. D., Edwards, J. R., & Spector, P. E. (2016). Statistical control in correlational studies: 10 essential recommendations for organizational researchers. Journal of Organizational Behavior, 37(2), 157–167. https://doi.org/10.1002/job.2053.
Bernerth, J. B., & Aguinis, H. (2016). A critical review and best-practice recommendations for control variable usage. Personnel Psychology, 69(1), 229–283. https://doi.org/10.1111/peps.12103.
Boughzala, I., de Vreede, T., Nguyen, C., & de Vreede, G.-J. (2014, 6-9 Jan. 2014). Towards a maturity model for the assessment of ideation in crowdsourcing projects. Paper presented at the 2014 47th Hawaii International Conference on System Sciences.
Bowling, N. A., Alarcon, G. M., Bragg, C. B., & Hartman, M. J. (2015). A meta-analytic examination of the potential correlates and consequences of workload. Work and Stress, 29(2), 95–113. https://doi.org/10.1080/02678373.2015.1033037.
Breaugh, J. A. (2006). Rethinking the control of nuisance variables in theory testing. Journal of Business and Psychology, 20(3), 429–443. https://doi.org/10.1007/s10869-005-9009-y.
Breaugh, J. A. (2008). Important considerations in using statistical procedures to control for nuisance variables in non-experimental studies. Human Resource Management Review, 18(4), 282–293. https://doi.org/10.1016/j.hrmr.2008.03.001.
Burks, B. (1926). On the inadequacy of the partial and multiple correlation technique. Journal of Educational Psychology, 17(9), 625–630.
Carlson, K. D., & Wu, J. (2012). The illusion of statistical control: Control variable practice in management research. Organizational Research Methods, 15(3), 413–435. https://doi.org/10.1177/1094428111428817.
Clark, A., Oswald, A., & Warr, P. (1996). Is job satisfaction U-shaped in age? Journal of Occupational and Organizational Psychology, 69, 57–81.
Clogg, C. C., Petkova, E., & Shihadeh, E. S. (1992). Statistical methods for analyzing collapsibility in regression models. Journal of Educational Statistics, 17(1), 51–74. https://doi.org/10.3102/10769986017001051.
Crits-Christoph, P., Gallop, R., Gaines, A., Rieger, A., & Connolly Gibbons, M. B. (2018). Instrumental variable analyses for causal inference: Application to multilevel analyses of the alliance–outcome relation. Psychotherapy Research, 30, 53–67. https://doi.org/10.1080/10503307.2018.1544724.
Crowne, D. P., & Marlowe, D. (1964). The approval motive. New York: John Wiley.
Judge, T. A. (1993). Does affective disposition moderate the relationship between job satisfaction and voluntary turnover? Journal of Applied Psychology, 78(3), 395–401.
Kraemer, H. C., Stice, E., Kazdin, A., Offord, D., & Kupfer, D. (2001). How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. The American Journal of Psychiatry, 158(6), 848–856. https://doi.org/10.1176/appi.ajp.158.6.848.
Lefkowitz, J. (2000). The role of interpersonal affective regard in supervisory performance ratings: A literature review and proposed causal model. Journal of Occupational and Organizational Psychology, 73(1), 67–85. https://doi.org/10.1348/096317900166886.
MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000). Equivalence of the mediation, confounding and suppression effect. Prevention Science, 1(4), 173–181.
Malone, T., & Lusk, J. L. (2018). An instrumental variable approach to distinguishing perceptions from preferences for beer brands. Managerial and Decision Economics, 39(4), 403–417. https://doi.org/10.1002/mde.2913.
Meehl, P. E. (1971). High school yearbooks: A reply to Schwarz. Journal of Abnormal Psychology, 77(2), 143–148.
Moorman, R. H., & Podsakoff, P. M. (1992). A meta-analytic review and empirical test of the potential confounding effects of social desirability response sets in organizational behaviour research. Journal of Occupational and Organizational Psychology, 65(2), 131–149.
O’Connell, B. J. (1991). An examination of work related social support in a longitudinal study controlling for negative affectivity and transient mood. In Unpublished Doctoral Dissertation. Tampa: University of South Florida.
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903.
Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539–569. https://doi.org/10.1146/annurev-psych-120710-100452.
Quinn, R. P., Staines, G. L., & McCullough, M. R. (1974). Job satisfaciton: Is there a trend? Manpower research monograph no. 30 U.S. Department of Labor. Washington: Government Printing Office.
Seeber, I., Bittner, E., Briggs, R. O., de Vreede, T., de Vreede, G.-J., Elkins, A., Maier, R., Merz, A. B., Oeste-Reiß, S., Randrup, N., Schwabe, G., & Söllner, M. (2020). Machines as teammates: A research agenda on AI in team collaboration. Information & Management, 57(2), 103174. https://doi.org/10.1016/j.im.2019.103174.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632.
Spector, P. E. (2017). The lost art of discovery: The case for inductive methods in occupational health science and the broader organizational sciences. Occupational Health Science, 1(1), 11–27. https://doi.org/10.1007/s41542-017-0001-5.
Spector, P. E. (2019). Do not cross me: Optimizing the use of cross-sectional designs. Journal of Business and Psychology, 34, 125–137. https://doi.org/10.1007/s10869-018-09613-8.
Spector, P. E., & Brannick, M. T. (2011). Methodological urban legends: The misuse of statistical control variables. Organizational Research Methods, 14(2), 287–305. https://doi.org/10.1177/1094428110369842.
Spector, P. E., & Brannick, M. T. (2016). Methodological urban legends: The misuse of statistical control variables. In G. J. Boyle, J. G. O’Gorman, G. J. Fogarty, G. J. Boyle, J. G. O’Gorman, & G. J. Fogarty (Eds.), Work and organisational psychology: Research methodology; assessment and selection; Organisational change and development; human resource and performance management; emerging trends: Innovation/globalisation/technology (pp. 63–86). Thousand Oaks, CA, US: Sage Publications, Inc..
Spector, P. E., Chen, P. Y., & O’Connell, B. J. (2000a). A longitudinal study of relations between job stressors and job strains while controlling for prior negative affectivity and strains. Journal of Applied Psychology, 85(2), 211–218. https://doi.org/10.1037/0021-9010.85.2.211.
Spector, P. E., & Meier, L. L. (2014). Methodologies for the study of organizational behavior processes: How to find your keys in the dark. Journal of Organizational Behavior, 35(8), 1109–1119. https://doi.org/10.1002/job.1966.
Spector, P. E., & Nixon, A. E. (2019). How often do I agree: An experimental test of item format method variance in stress measures. Occupational Health Science, 3, 125–143.
Spector, P. E., Rosen, C. C., Richardson, H. A., Williams, L. J., & Johnson, R. E. (2019). A new perspective on method variance: A measure-centric approach. Journal of Management, 45(3), 855–880. https://doi.org/10.1177/0149206316687295.
Spector, P. E., Zapf, D., Chen, P. Y., & Frese, M. (2000b). Why negative affectivity should not be controlled in job stress research: Don’t throw out the baby with the bath water. Journal of Organizational Behavior, 21(1), 79–95. https://doi.org/10.1002/%28SICI%291099-1379%28200002%2921:1%3C79::AID-JOB964%3E3.0.CO;2-G.
Watson, D., Pennebaker, J. W., & Folger, R. (1986). Beyond negative affectivity: Measuring stress and satisfaction in the workplace. Journal of Organizational Behavior Management, 8(2), 141–157. https://doi.org/10.1300/J075v08n02_09.
Weitz, J. (1952). A neglected concept in the study of job satisfaction. Personnel Psychology, 5, 201–205. https://doi.org/10.1111/j.1744-6570.1952.tb01012.x.
Wright, J. D., & Hamilton, R. F. (1978). Work satisfaction and age: Some evidence for the ‘job change’ hypothesis. Social Forces, 56(4), 1140–1158. https://doi.org/10.2307/2577515.
Acknowledgments
Thanks to Steven Rogelberg for convincing me that there was something new to say about control variables and to Fred Oswald, Scott Tonidandel, and Jeremy Dawson for their incredibly helpful feedback on earlier versions of the paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Spector, P.E. Mastering the Use of Control Variables: the Hierarchical Iterative Control (HIC) Approach. J Bus Psychol 36, 737–750 (2021). https://doi.org/10.1007/s10869-020-09709-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10869-020-09709-0