Mastering the Use of Control Variables: the Hierarchical Iterative Control (HIC) Approach

Abstract

There has been growing criticism of the established practice of automatically including control variables into analyses, especially with survey studies. Several authors have explained the pitfalls of improper use and have provided some best practice advice. I build upon this foundation in suggesting a programmatic approach to the use of control variables that can provide evidence to support or refute feasible explanations for why two or more variables are related. The hierarchical iterative control (HIC) approach begins by establishing a connection between two or more variables and then hierarchically adds control variables to rule in or out their possible influence. The HIC approach involves conducting a series of studies to iteratively test relationships among target variables, utilizing a variety of control variable strategies involving multiple methods. A 7-step programmatic approach is described beginning with development of the research question and background literature review and then conducting empirical tests in a hierarchical (within a study) and iterative (across studies) manner.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

Notes

  1. 1.

    Thanks to one of my reviewers for suggesting this.

References

  1. Atinc, G. M., Simmering, M. J., & Kroll, M. J. (2012). Control variable use and reporting in macro and micro management research. Organizational Research Methods, 15(1), 57–74. https://doi.org/10.1177/1094428110397773.

    Article  Google Scholar 

  2. Becker, T. E. (2005). Potential problems in the statistical control of variables in organizational research: A qualitative analysis with recommendations. Organizational Research Methods, 8(3), 274–289.

    Article  Google Scholar 

  3. Becker, T. E., Atinc, G., Breaugh, J. A., Carlson, K. D., Edwards, J. R., & Spector, P. E. (2016). Statistical control in correlational studies: 10 essential recommendations for organizational researchers. Journal of Organizational Behavior, 37(2), 157–167. https://doi.org/10.1002/job.2053.

    Article  Google Scholar 

  4. Bernerth, J. B., & Aguinis, H. (2016). A critical review and best-practice recommendations for control variable usage. Personnel Psychology, 69(1), 229–283. https://doi.org/10.1111/peps.12103.

    Article  Google Scholar 

  5. Boughzala, I., de Vreede, T., Nguyen, C., & de Vreede, G.-J. (2014, 6-9 Jan. 2014). Towards a maturity model for the assessment of ideation in crowdsourcing projects. Paper presented at the 2014 47th Hawaii International Conference on System Sciences.

  6. Bowling, N. A., Alarcon, G. M., Bragg, C. B., & Hartman, M. J. (2015). A meta-analytic examination of the potential correlates and consequences of workload. Work and Stress, 29(2), 95–113. https://doi.org/10.1080/02678373.2015.1033037.

    Article  Google Scholar 

  7. Breaugh, J. A. (2006). Rethinking the control of nuisance variables in theory testing. Journal of Business and Psychology, 20(3), 429–443. https://doi.org/10.1007/s10869-005-9009-y.

    Article  Google Scholar 

  8. Breaugh, J. A. (2008). Important considerations in using statistical procedures to control for nuisance variables in non-experimental studies. Human Resource Management Review, 18(4), 282–293. https://doi.org/10.1016/j.hrmr.2008.03.001.

    Article  Google Scholar 

  9. Burks, B. (1926). On the inadequacy of the partial and multiple correlation technique. Journal of Educational Psychology, 17(9), 625–630.

    Article  Google Scholar 

  10. Carlson, K. D., & Wu, J. (2012). The illusion of statistical control: Control variable practice in management research. Organizational Research Methods, 15(3), 413–435. https://doi.org/10.1177/1094428111428817.

    Article  Google Scholar 

  11. Clark, A., Oswald, A., & Warr, P. (1996). Is job satisfaction U-shaped in age? Journal of Occupational and Organizational Psychology, 69, 57–81.

    Article  Google Scholar 

  12. Clogg, C. C., Petkova, E., & Shihadeh, E. S. (1992). Statistical methods for analyzing collapsibility in regression models. Journal of Educational Statistics, 17(1), 51–74. https://doi.org/10.3102/10769986017001051.

    Article  Google Scholar 

  13. Crits-Christoph, P., Gallop, R., Gaines, A., Rieger, A., & Connolly Gibbons, M. B. (2018). Instrumental variable analyses for causal inference: Application to multilevel analyses of the alliance–outcome relation. Psychotherapy Research, 30, 53–67. https://doi.org/10.1080/10503307.2018.1544724.

    Article  PubMed  Google Scholar 

  14. Crowne, D. P., & Marlowe, D. (1964). The approval motive. New York: John Wiley.

    Google Scholar 

  15. Judge, T. A. (1993). Does affective disposition moderate the relationship between job satisfaction and voluntary turnover? Journal of Applied Psychology, 78(3), 395–401.

    Article  Google Scholar 

  16. Kraemer, H. C., Stice, E., Kazdin, A., Offord, D., & Kupfer, D. (2001). How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. The American Journal of Psychiatry, 158(6), 848–856. https://doi.org/10.1176/appi.ajp.158.6.848.

    Article  PubMed  Google Scholar 

  17. Lefkowitz, J. (2000). The role of interpersonal affective regard in supervisory performance ratings: A literature review and proposed causal model. Journal of Occupational and Organizational Psychology, 73(1), 67–85. https://doi.org/10.1348/096317900166886.

    Article  Google Scholar 

  18. MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000). Equivalence of the mediation, confounding and suppression effect. Prevention Science, 1(4), 173–181.

    Article  Google Scholar 

  19. Malone, T., & Lusk, J. L. (2018). An instrumental variable approach to distinguishing perceptions from preferences for beer brands. Managerial and Decision Economics, 39(4), 403–417. https://doi.org/10.1002/mde.2913.

    Article  Google Scholar 

  20. Meehl, P. E. (1971). High school yearbooks: A reply to Schwarz. Journal of Abnormal Psychology, 77(2), 143–148.

    Article  Google Scholar 

  21. Moorman, R. H., & Podsakoff, P. M. (1992). A meta-analytic review and empirical test of the potential confounding effects of social desirability response sets in organizational behaviour research. Journal of Occupational and Organizational Psychology, 65(2), 131–149.

    Article  Google Scholar 

  22. O’Connell, B. J. (1991). An examination of work related social support in a longitudinal study controlling for negative affectivity and transient mood. In Unpublished Doctoral Dissertation. Tampa: University of South Florida.

    Google Scholar 

  23. Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903.

    Article  Google Scholar 

  24. Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539–569. https://doi.org/10.1146/annurev-psych-120710-100452.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Quinn, R. P., Staines, G. L., & McCullough, M. R. (1974). Job satisfaciton: Is there a trend? Manpower research monograph no. 30 U.S. Department of Labor. Washington: Government Printing Office.

    Google Scholar 

  26. Seeber, I., Bittner, E., Briggs, R. O., de Vreede, T., de Vreede, G.-J., Elkins, A., Maier, R., Merz, A. B., Oeste-Reiß, S., Randrup, N., Schwabe, G., & Söllner, M. (2020). Machines as teammates: A research agenda on AI in team collaboration. Information & Management, 57(2), 103174. https://doi.org/10.1016/j.im.2019.103174.

    Article  Google Scholar 

  27. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.

    Google Scholar 

  28. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632.

    Article  PubMed  Google Scholar 

  29. Spector, P. E. (2017). The lost art of discovery: The case for inductive methods in occupational health science and the broader organizational sciences. Occupational Health Science, 1(1), 11–27. https://doi.org/10.1007/s41542-017-0001-5.

    Article  Google Scholar 

  30. Spector, P. E. (2019). Do not cross me: Optimizing the use of cross-sectional designs. Journal of Business and Psychology, 34, 125–137. https://doi.org/10.1007/s10869-018-09613-8.

    Article  Google Scholar 

  31. Spector, P. E., & Brannick, M. T. (2011). Methodological urban legends: The misuse of statistical control variables. Organizational Research Methods, 14(2), 287–305. https://doi.org/10.1177/1094428110369842.

    Article  Google Scholar 

  32. Spector, P. E., & Brannick, M. T. (2016). Methodological urban legends: The misuse of statistical control variables. In G. J. Boyle, J. G. O’Gorman, G. J. Fogarty, G. J. Boyle, J. G. O’Gorman, & G. J. Fogarty (Eds.), Work and organisational psychology: Research methodology; assessment and selection; Organisational change and development; human resource and performance management; emerging trends: Innovation/globalisation/technology (pp. 63–86). Thousand Oaks, CA, US: Sage Publications, Inc..

    Google Scholar 

  33. Spector, P. E., Chen, P. Y., & O’Connell, B. J. (2000a). A longitudinal study of relations between job stressors and job strains while controlling for prior negative affectivity and strains. Journal of Applied Psychology, 85(2), 211–218. https://doi.org/10.1037/0021-9010.85.2.211.

    Article  PubMed  Google Scholar 

  34. Spector, P. E., & Meier, L. L. (2014). Methodologies for the study of organizational behavior processes: How to find your keys in the dark. Journal of Organizational Behavior, 35(8), 1109–1119. https://doi.org/10.1002/job.1966.

    Article  Google Scholar 

  35. Spector, P. E., & Nixon, A. E. (2019). How often do I agree: An experimental test of item format method variance in stress measures. Occupational Health Science, 3, 125–143.

    Article  Google Scholar 

  36. Spector, P. E., Rosen, C. C., Richardson, H. A., Williams, L. J., & Johnson, R. E. (2019). A new perspective on method variance: A measure-centric approach. Journal of Management, 45(3), 855–880. https://doi.org/10.1177/0149206316687295.

    Article  Google Scholar 

  37. Spector, P. E., Zapf, D., Chen, P. Y., & Frese, M. (2000b). Why negative affectivity should not be controlled in job stress research: Don’t throw out the baby with the bath water. Journal of Organizational Behavior, 21(1), 79–95. https://doi.org/10.1002/%28SICI%291099-1379%28200002%2921:1%3C79::AID-JOB964%3E3.0.CO;2-G.

    Article  Google Scholar 

  38. Watson, D., Pennebaker, J. W., & Folger, R. (1986). Beyond negative affectivity: Measuring stress and satisfaction in the workplace. Journal of Organizational Behavior Management, 8(2), 141–157. https://doi.org/10.1300/J075v08n02_09.

    Article  Google Scholar 

  39. Weitz, J. (1952). A neglected concept in the study of job satisfaction. Personnel Psychology, 5, 201–205. https://doi.org/10.1111/j.1744-6570.1952.tb01012.x.

    Article  Google Scholar 

  40. Wright, J. D., & Hamilton, R. F. (1978). Work satisfaction and age: Some evidence for the ‘job change’ hypothesis. Social Forces, 56(4), 1140–1158. https://doi.org/10.2307/2577515.

    Article  Google Scholar 

Download references

Acknowledgments

Thanks to Steven Rogelberg for convincing me that there was something new to say about control variables and to Fred Oswald, Scott Tonidandel, and Jeremy Dawson for their incredibly helpful feedback on earlier versions of the paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Paul E. Spector.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Spector, P.E. Mastering the Use of Control Variables: the Hierarchical Iterative Control (HIC) Approach. J Bus Psychol (2020). https://doi.org/10.1007/s10869-020-09709-0

Download citation

Keywords

  • Causal inference
  • Control variable
  • Method variance
  • Philosophy of science research design
  • Research methodology