Do Not Cross Me: Optimizing the Use of Cross-Sectional Designs

Abstract

The cross-sectional research design, especially when used with self-report surveys, is held in low esteem despite its widespread use. It is generally accepted that the longitudinal design offers considerable advantages and should be preferred due to its ability to shed light on causal connections. In this paper, I will argue that the ability of the longitudinal design to reflect causality has been overstated and that it offers limited advantages over the cross-sectional design in most cases in which it is used. The nature of causal inference from a philosophy of science perspective is used to illustrate how cross-sectional designs can provide evidence for relationships among variables and can be used to rule out many potential alternative explanations for those relationships. Strategies for optimizing the use of cross-sectional designs are noted, including the inclusion of control variables to rule out spurious relationships, the addition of alternative sources of data, and the incorporation of experimental methods. Best practice advice is offered for the use of both cross-sectional and longitudinal designs, as well as for authors writing and for reviewers evaluating papers that report results of cross-sectional studies.

This is a preview of subscription content, access via your institution.

References

  1. Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2010). On making causal claims: A review and recommendations. The Leadership Quarterly, 21(6), 1086–1120. https://doi.org/10.1016/j.leaqua.2010.10.010.

    Article  Google Scholar 

  2. Berofsky, B. (1966). Causality and general laws. The Journal of Philosophy, 63(6), 148–157. https://doi.org/10.2307/2024170.

    Article  Google Scholar 

  3. Bolino, M. C. (1999). Citizenship and impression management: Good soldiers or good actors? The Academy of Management Review, 24(1), 82–98. https://doi.org/10.2307/259038.

    Article  Google Scholar 

  4. Brief, A. P., Burke, M. J., George, J. M., Robinson, B. S., & Webster, J. (1988). Should negative affectivity remain an unmeasured variable in the study of job stress? Journal of Applied Psychology, 73(2), 193–198.

    Article  PubMed  Google Scholar 

  5. Chen, P. Y., & Spector, P. E. (1991). Negative affectivity as the underlying cause of correlations between stressors and strains. Journal of Applied Psychology, 76(3), 398–407. https://doi.org/10.1037/0021-9010.76.3.398.

    Article  PubMed  Google Scholar 

  6. Cole, D. A., Martin, N. C., & Steiger, J. H. (2005). Empirical and conceptual problems with longitudinal trait-state models: Introducing a trait-state-occasion model. Psychological Methods, 10(1), 3–20. https://doi.org/10.1037/1082-989X.10.1.3.

    Article  PubMed  Google Scholar 

  7. Dalal, R. S. (2005). A meta-analysis of the relationship between organizational citizenship behavior and counterproductive work behavior. Journal of Applied Psychology, 90(6), 1241–1255. https://doi.org/10.1037/0021-9010.90.6.1241.

    Article  PubMed  Google Scholar 

  8. Frese, M., & Zapf, D. (1988). Methodological issues in the study of work stress: Objective vs subjective measurement of work stress and the question of longitudinal studies. In C. L. Cooper & R. Payne (Eds.), Causes, coping and consequences of stress at work (pp. 375–411). Oxford, England: John Wiley & Sons.

    Google Scholar 

  9. Glick, W. H., Huber, G. P., Miller, C. C., Doty, D. H., & Sutcliffe, K. M. (1990). Studying changes in organizational design and effectiveness: Retrospective event histories and periodic assessments. Organization Science, 1(3), 293–312. https://doi.org/10.2307/2635007.

    Article  Google Scholar 

  10. Glick, W. H., Jenkins, G., & Gupta, N. (1986). Method versus substance: How strong are underlying relationships between job characteristics and attitudinal outcomes? Academy of Management Journal, 29(3), 441–464.

    Article  Google Scholar 

  11. Griffin, R. W. (1991). Effects of work redesign on employee perceptions, attitudes, and behaviors: A long-term investigation. Academy of Management Journal, 34(2), 425–435. https://doi.org/10.2307/256449.

    Article  Google Scholar 

  12. Hausman, D. M., & Woodward, J. (1999). Independence, invariance and the causal Markov condition. The British Journal for the Philosophy of Science, 50(4), 521–583. https://doi.org/10.1093/bjps/50.4.521.

    Article  Google Scholar 

  13. Illari, P., & Russo, F. (2014). Causality: Philosophical theory meets scientific practice. Oxford, UK: Oxford University Press.

    Google Scholar 

  14. Imai, K., Keele, L., & Tingley, D. (2010). A general approach to causal mediation analysis. Psychological Methods, 15(4), 309–334. https://doi.org/10.1037/a0020761.

    Article  PubMed  Google Scholar 

  15. Kraemer, H. C., Stice, E., Kazdin, A., Offord, D., & Kupfer, D. (2001). How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. The American Journal of Psychiatry, 158(6), 848–856. https://doi.org/10.1176/appi.ajp.158.6.848.

    Article  PubMed  Google Scholar 

  16. Meehl, P. E. (1971). High school yearbooks: A reply to Schwarz. Journal of Abnormal Psychology, 77(2), 143–148.

    Article  Google Scholar 

  17. Mitchell, T. R., & James, L. R. (2001). Building better theory: Time and the specification of when things happen. The Academy of Management Review, 26(4), 530–547. https://doi.org/10.2307/3560240.

    Article  Google Scholar 

  18. Nixon, A. E., Mazzola, J. J., Bauer, J., Krueger, J. R., & Spector, P. E. (2011). Can work make you sick? A meta-analysis of the relationships between job stressors and physical symptoms. Work & Stress, 25(1), 1–22. https://doi.org/10.1080/02678373.2011.569175.

    Article  Google Scholar 

  19. Pearl, J. (2014). Interpretation and identification of causal mediation. Psychological Methods, 19(4), 459–481. https://doi.org/10.1037/a0036434.

    Article  PubMed  Google Scholar 

  20. Pindek, S., & Spector, P. E. (2016). Organizational constraints: A meta-analysis of a major stressor. Work & Stress, 30(1), 7–25. https://doi.org/10.1080/02678373.2015.1137376.

    Article  Google Scholar 

  21. Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539–569. https://doi.org/10.1146/annurev-psych-120710-100452.

    Article  PubMed  Google Scholar 

  22. Schwarzmüller, T., Brosi, P., & Welpe, I. M. (2018). Sparking anger and anxiety: Why intense leader anger displays trigger both more deviance and higher work effort in followers. Journal of Business and Psychology, 33(6), 761–777. https://doi.org/10.1007/s10869-017-9523-8.

    Article  Google Scholar 

  23. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.

    Google Scholar 

  24. Spector, P. E. (1992). A consideration of the validity and meaning of self-report measures of job conditions. In C. L. Cooper & I. T. Robertson (Eds.), International Review of Industrial and Organizational Psychology (pp. 123–151). West Sussex, UK: John Wiley.

    Google Scholar 

  25. Spector, P. E. (2006). Method variance in organizational research: Truth or urban legend? Organizational Research Methods, 9(2), 221–232. https://doi.org/10.1177/1094428105284955.

    Article  Google Scholar 

  26. Spector, P. E., Bauer, J. A., & Fox, S. (2010). Measurement artifacts in the assessment of counterproductive work behavior and organizational citizenship behavior: Do we know what we think we know? Journal of Applied Psychology, 95(4), 781–790. https://doi.org/10.1037/a0019477.

    Article  PubMed  Google Scholar 

  27. Spector, P. E., & Brannick, M. T. (2011). Methodological urban legends: The misuse of statistical control variables. Organizational Research Methods, 14(2), 287–305. https://doi.org/10.1177/1094428110369842.

    Article  Google Scholar 

  28. Spector, P. E., Fox, S., & Van Katwyk, P. T. (1999). The role of negative affectivity in employee reactions to job characteristics: Bias effect or substantive effect? Journal of Occupational and Organizational Psychology, 72(2), 205–218. https://doi.org/10.1348/096317999166608.

    Article  Google Scholar 

  29. Spector, P. E., & Meier, L. L. (2014). Methodologies for the study of organizational behavior processes: How to find your keys in the dark. Journal of Organizational Behavior, 35(8), 1109–1119. https://doi.org/10.1002/job.1966.

    Article  Google Scholar 

  30. Spector, P. E., & Pindek, S. (2016). The future of research methods in work and occupational health psychology. Applied Psychology: An International Review, 65(2), 412–431. https://doi.org/10.1111/apps.12056.

    Article  Google Scholar 

  31. Spector, P. E., Rosen, C. C., Richardson, H. A., Williams, L. J., & Johnson, R. E. (2017). A new perspective on method variance: A measure-centric approach. Journal of Management, 0(0), 0149206316687295. https://doi.org/10.1177/0149206316687295.

    Article  Google Scholar 

  32. Spector, P. E., Yang, L.-Q., & Zhou, Z. E. (2015). A longitudinal investigation of the role of violence prevention climate in exposure to workplace physical violence and verbal abuse. Work & Stress, 29(4), 325–340. https://doi.org/10.1080/02678373.2015.1076537.

    Article  Google Scholar 

  33. Stone-Romero, E. F., & Rosopa, P. J. (2008). The relative validity of inferences about mediation as a function of research design characteristics. Organizational Research Methods, 11(2), 326–352. https://doi.org/10.1177/1094428107300342.

    Article  Google Scholar 

  34. Tuma, N. B., & Hannan, M. T. (1984). Social dynamics models and methods. Saint Louis, US: Elsevier.

    Google Scholar 

  35. Watson, D., Pennebaker, J. W., & Folger, R. (1986). Beyond negative affectivity: Measuring stress and satisfaction in the workplace. Journal of Organizational Behavior Management, 8(2), 141–157. https://doi.org/10.1300/J075v08n02_09.

    Article  Google Scholar 

  36. Woodward, J. (2003). Making things happen: A theory of causal explanation. Oxford: New York City.

    Google Scholar 

  37. Woodward, J. (2017). Scientific explanation. In E. N. Zalta (Ed.), Stanford encyclopedia of philosophy. Retrieved from https://plato.stanford.edu/archives/fall2017/entries/scientific-explanation. Accessed 28 Dec 2018.

  38. Zapf, D., Dormann, C., & Frese, M. (1996). Longitudinal studies in organizational stress research: A review of the literature with reference to methodological issues. Journal of Occupational Health Psychology, 1(2), 145–169. https://doi.org/10.1037/1076-8998.1.2.145.

    Article  PubMed  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Paul E. Spector.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Spector, P.E. Do Not Cross Me: Optimizing the Use of Cross-Sectional Designs. J Bus Psychol 34, 125–137 (2019). https://doi.org/10.1007/s10869-018-09613-8

Download citation

Keywords

  • Causal inference
  • Causality
  • Cross-sectional design
  • Longitudinal design
  • Method variance
  • Philosophy of science
  • Research design
  • Research methodology