Journal of Business and Psychology

, Volume 17, Issue 2, pp 245–260 | Cite as

Understanding Self-Report Bias in Organizational Behavior Research

Article

Abstract

Self-report and mono-method bias often threaten the validity of research conducted in business settings and thus hinder the development of theories of organizational behavior. This paper outlines a conceptual framework for understanding factors that influence the motivation of an employee to bias his or her responses to questions posed by organizational researchers. Using a longitudinal, multitrait-multimethod dataset, we illustrate various aspects of the problem and argue that traditional approaches for controlling self-report bias do not adequately prevent the problem. The results suggest the need for developing a theory of method effects and companion analytic techniques to improve the accuracy of psychological research in business settings.

self-report bias organizational behavior research method bias construct validity 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

REFERENCES

  1. Becker, T. E., & Vance, R. J. (1992). Construct validity of three types of organizational citizenship behavior using a direct product model.Paper presented at the 1992 Academy of Management Meeting, Las Vegas (August).Google Scholar
  2. Borman, W. C. (1991). Job behavior, performance, and effectiveness. In M. D. Dunnette & L. M. (Eds.), Handbook of industrial and organizational psychology (pp. 271–326). Newbury Park, CA: Sage.Google Scholar
  3. Browne, M. W. (1984). The decomposition of multitrait-multimethod matrices.British Journal of Mathematical and Statistical Psychology, 37, 1–21.Google Scholar
  4. Browne, M. W. (1993). Models for multitrait-multimethod matrices. In R. Steyer, K.F. Wender, & K.F. Widamen (Eds.), Psychometric Methodology (pp. 570–584). Stuttgart and New York: Gustav Fisher Verlag.Google Scholar
  5. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by multitrait-multimethod matrix.Psychological Bulletin,56,81–105.Google Scholar
  6. Crowne, D. P., & Marlowe. D. (1964). The approval motive: Studies in evaluative dependence. New York: Wiley.Google Scholar
  7. Donaldson, S. I. (1993). Effects of lifestyle and stress on the employee and organization: Implications for promoting health at work.Anxiety, Stress, and Coping,6,155–177. 259 STEWART I. DONALDSON AND ELISA J. GRANT-VALLONEGoogle Scholar
  8. Donaldson, S. I. (1995). Worksite health promotion: A theory-driven, empirically based perspective. In L. R. Murphy, J. J. Hurrel, S. L. Sauter, & G. P. Keita (Eds.), Job stress interventions (pp. 73–90). Washington, DC: American Psychological Association.Google Scholar
  9. Donaldson, S. I., & Blanchard, A. L. (1995). The seven health practices, well-being, and performance at work: Evidence for the value of reaching small and underserved worksites. Preventive Medicine,24,270–277.Google Scholar
  10. Donaldson, S. I., Dent. C. W., Sussman, S., Stoddard, J. J., & Severson, H. H. (1996). The organizational implications of smokeless tobacco use in the Lumber Mill Industry. Addictive Behaviors,21,259–267.Google Scholar
  11. Donaldson, S. I., Ensher, E. A., & Grant-Vallone, E. J. (2000). Longitudinal examination of mentoring relationships on organizational commitment and citizenship behavior. Journal of Career Development, 26(4),233–249.Google Scholar
  12. Donaldson, S. I., Gooler, L. E., & Weiss, R. (1998). Promoting health and well-being through work: Science and practice. In X. B. Arriaga & S. Oskamp (Eds.), Addressing community problems: Psychological research and intervention (pp. 160–194).Newbury Park: Sage.Google Scholar
  13. Donaldson, S. I., Graham, J. W., Piccinin, A. M., & Hansen, W. B. (1995). Resistance-skills training and onset of alcohol use: Evidence for beneficial and potentially harmful effects in public schools and in private catholic schools.Health Psychology,14, 291–300.Google Scholar
  14. Donaldson, S. I., & Klein, D. (1997). Creating healthful work environments for ethnically diverse employees working in small and medium-sized businesses: A non-profit industry/ community/university collaboration model.Employee Assistance Quarterly,13, 17–32.Google Scholar
  15. Donaldson, S. I., Thomas, C. W., & Graham, J. W. (2002). Understanding self-report bias in prevention research.Manuscript under review.Google Scholar
  16. Donaldson, S. I., Thomas, C. W., Graham, Au, J., & Hansen, W. B. (2000). Verifying drug prevention program effects using reciprocal best friend reports.Journal of Behavioral Medicine,23,221–234.Google Scholar
  17. Donaldson, S. I., & Weiss, R. (1998). Health, well-being, and organizational effectiveness in the virtual workplace. In M. Igbaria, & M. Tan,The virtual workplace (pp. 24–44). Harrisburg, PA: Idea Group Publishing.Google Scholar
  18. Ensher, E. A., Grant-Vallone, E. J., & Donaldson, S. I. (2001). Effects of perceived discrimination on job satisfaction, organizational commitment, organizational citizenship behavior, and grievances. Human Resource Development Quarterly, 12, 53–72.Google Scholar
  19. Fowler, F. Jr. (1995). Improving survey questions: Design and evaluation. Thousand Oaks, CA: Sage.Google Scholar
  20. Graham, J. W., & Collins, N. L. (1991). Controlling correlational bias via confirmatory factor analysis of MTMM data. Multivariate Behavioral Research,26,501–523.Google Scholar
  21. Graham, J.W., Collins, N.L., Donaldson, S.I., & Hansen, W.B. (1993). Understanding and controlling for response bias: Confirmatory factor analysis of multitrait-multimethod data. In R. Steyer, K.F. Wender, & K.F. Widamen (Eds.), Psychometric Methodology (pp. 585–590).Stuttgart and New York: Gustav Fisher Verlag.Google Scholar
  22. Howard, H. (1994). Why do people say nasty things about self-reports? Journal of Organizational Behavior,15, 399–404.Google Scholar
  23. Karasek, R. (1985). Job content questionnaire.Department of Industrial systems Engineering, University of Southern California, Los Angeles.Google Scholar
  24. Karasek, R., & Theorell, T. (1990). Healthy work: Stress, productivity, and the reconstruction of working life.New York: Basic Books.Google Scholar
  25. Lifestyle Assessment Questionnaire (1988). LAQ reliability/validity research efforts.Steven Point, WI: National Wellness Institute.Google Scholar
  26. McCall, M. M., & Bobko, P. (1990). Research methods in the service of discovery. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (pp. 381–418). Palo Alto, CA: Consulting Psychologists Press, Inc.Google Scholar
  27. McDowell, I. & Newell, C. (1987). Measuring health: A guide to rating scales and questionnaires. New York: Oxford University Press.Google Scholar
  28. Mersman, J. L., & Donaldson, S. I. (2000). Factors affecting the convergence of self-peer ratings on contextual and task performance.Human Performance, 13(3),299–322Google Scholar
  29. Moorman, R. H., & Podsakoff, P. M. (1992). A meta-analytic review and empirical test of 260 JOURNAL OF BUSINESS AND PSYCHOLOGY the potential confounding effects of social desirability response sets in organizational behavior research.Journal of Occupational and Organizational Psychology,65,131–149.Google Scholar
  30. Podsakoff, P.M., & Organ, D. W. (1986). Self-reports in organizational research: Problems and prospects.Journal of Management,12,531–544.Google Scholar
  31. Sacket, P. R., & Larson, J. R. (1990). Research strategies and tactics in industrial and organizational psychology. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (pp. 419–489).Palo Alto, CA: Consulting Psychologists Press.Google Scholar
  32. Schmitt, N. (1989). Editorial.Journal of Applied Psychology,74,843–845.Google Scholar
  33. Schmitt, N. (1994). Method bias: the importance of theory and measurement.Journal of Organizational Behavior,15,393–398.Google Scholar
  34. Schwartz, N. (1999). Self-reports: How the questions shape the answers.American Psychologist, 54, 93–105.Google Scholar
  35. Shadish, W. R. (1993). Critical multiplism: A research strategy and its attendant tactics. New Directions for Program Evaluation, 60,13–57.Google Scholar
  36. Smith, J. C. (1993). Understanding stress and coping.New York: Macmillan.Google Scholar
  37. Spector, P. E. (1987). Method variance as an artifact in self-reported affect and perceptions at work: Myth or significant problem? Journal of Applied Psychology,3,438–443.Google Scholar
  38. Spector, P. E. (1994). Using self-report questionnaires in OB research: a comment on the use of a controversial method.Journal of Organizational Behavior,15,385–392.Google Scholar
  39. Spector, P. E., & Brannick, M. T. (1995). The nature and effects of method variance in organizational research. In C. L. Cooper & I. T. Robertson (Eds.), International Review of Industrial and Organizational Psychology (pp. 210–245). New York: Wiley & Sons.Google Scholar
  40. Stone, A.A., Turkkan, J.S., Bachrach, C.A., Jobe, J.B., Kurtzman, H.S. & Cain, V.S. (2000). The science of self-report: Implications for research and practice.Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  41. Williams, L. J., & Brown, B. K. (1994). Method variance in organizational behavior and human resources research: Effects on correlations, path coefficients, and hypothesis testing.Organizational Behavior and Human Decision Processes,58,185–209.Google Scholar
  42. Williams, L. J., Cote, J. A., & Buckley, M. R. (1989). Lack of method variance in selfreported affect and perceptions at work: Reality or artifact? Journal of Applied Psychology, 74,462–468.Google Scholar
  43. Wyler, A. R., Masuda, M., & Holmes, T. H. (1968). The Seriousness of Illness Rating Scale. Journal of Psychosomatic Research, 11,363–374.Google Scholar
  44. Zerbe, W. J., & Paulhus, D. L. (1987). Socially desirable responding in organizational behavior: A reconception.Academy of Management Review, 12,250–254.Google Scholar

Copyright information

© Human Sciences Press, Inc. 2002

Authors and Affiliations

  • Stewart I. Donaldson
    • 1
  • Elisa J. Grant-Vallone
    • 2
  1. 1.Department of PsychologyClaremont Graduate UniversityClaremont
  2. 2.California State UniversitySan Marcos

Personalised recommendations