Advertisement

Journal of Business and Psychology

, Volume 34, Issue 4, pp 425–452 | Cite as

A Tale of Two Sample Sources: Do Results from Online Panel Data and Conventional Data Converge?

  • Sheryl L. WalterEmail author
  • Scott E. Seibert
  • Daniel Goering
  • Ernest H. O’BoyleJr
Original Paper

Abstract

Samples drawn from commercial online panel data (OPD) are becoming more prevalent in applied psychology research, but they remain controversial due to concerns with data quality. In order to examine the validity of OPD, we conduct meta-analyses of online panel samples and compare internal reliability estimates for scales and effect size estimates for IV–DV relations commonly found in the field with those based on conventionally sourced data. Results based on 90 independent samples and 32,121 participants show OPD has similar psychometric properties and produces criterion validities that generally fall within the credibility intervals of existing meta-analytic results from conventionally sourced data. We suggest that, with appropriate caution, OPD are suitable for many exploratory research questions in the field of applied psychology.

Keywords

Online panel data MTurk StudyResponse Meta-analysis 

References

The articles marked with an asterisk are included in the meta-analysis.

  1. *Alarcon, G. M. (2009). The development of the wright work engagement scale (Doctoral dissertation). Available from ProQuest Dissertations & Theses database. (UMI No. 3393395).Google Scholar
  2. *Badger, J. M. (2014). The formative nature of perceived person-environment fit (Doctoral dissertation). Available from ProQuest Dissertations & Theses database. (UMI No. 3615149).Google Scholar
  3. Baker, R., Blumberg, S. J., Brick, J. M., Couper, M. P., Courtright, M., Dennis, J. M., et al. (2010). Research synthesis AAPOR report on online panels. Public Opinion Quarterly, 74(4), 711–781.Google Scholar
  4. *Ballinger, G. A., Lehman, D. W., & Schoorman, F. D. (2010). Leader–member exchange and turnover before and after succession events. Organizational Behavior and Human Decision Processes, 113(1), 25–36.Google Scholar
  5. *Baratta, P. (2014). The “Noonday Demon”, weariness, inattention, or all of the above? Refining the definition and measurement of state boredom (Unpublished doctoral dissertation). The University of Guelph.Google Scholar
  6. *Basford, T. E., Offermann, L. R., & Behrend, T. S. (2014). Please accept my sincerest apologies: Examining follower reactions to leader apology. Journal of Business Ethics, 119(1), 99–117.Google Scholar
  7. *Bauer, J. A. (2013). An investigation of OCB demands and workplace behaviors. (Doctoral dissertation). Graduate theses and dissertations. https://scholarcommons.usf.edu/etd/4634
  8. Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011). The viability of crowdsourcing for survey research. Behavior Research Methods, 43(3), 800–813.Google Scholar
  9. Bergman, M. E., & Jean, V. A. (2016). Where have all the “workers” gone? A critical analysis of the unrepresentativeness of our samples relative to the labor market in the industrial–organizational psychology literature. Industrial and Organizational Psychology, 9(01), 84–113.Google Scholar
  10. Berry, C. M., Ones, D. S., & Sackett, P. R. (2007). Interpersonal deviance, organizational deviance, and their common correlates: A review and meta-analysis. Journal of Applied Psychology, 92(2), 410–424.Google Scholar
  11. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester: John Wiley & Sons, Ltd.Google Scholar
  12. Botella, J., Suero, M., & Gambara, H. (2010). Psychometric inferences from a meta-analysis of reliability and internal consistency coefficients. Psychological Methods, 15, 386–397.Google Scholar
  13. *Bowling, N. A., & Burns, G. N. (2010). A comparison of work-specific and general personality measures as predictors of work and non-work criteria. Personality and Individual Differences, 49(2), 95–101.Google Scholar
  14. *Bowling, N. A., Burns, G. N., & Beehr, T. A. (2010). Productive and counterproductive attendance behavior: An examination of early and late arrival to and departure from work. Human Performance, 23(4), 305–322.Google Scholar
  15. *Bowling, N. A., Burns, G. N., Stewart, S. M., & Gruys, M. L. (2011). Conscientiousness and agreeableness as moderators of the relationship between neuroticism and counterproductive work behaviors: A constructive replication. International Journal of Selection and Assessment, 19(3), 320–330.Google Scholar
  16. *Bowling, N. A., & Eschleman, K. J. (2010). Employee personality as a moderator of the relationships between work stressors and counterproductive work behavior. Journal of Occupational Health Psychology, 15(1), 91–103.Google Scholar
  17. *Bowling, N. A., & Michel, J. S. (2011). Why do you treat me badly? The role of attributions regarding the cause of abuse in subordinates’ responses to abusive supervision. Work & Stress, 25(4), 309–320.Google Scholar
  18. Brüggen, E., Wetzels, M., de Ruyter, K., & Schillewaert, N. (2011). Individual differences in motivation to participate in online panels: The effect on response rate and response quality perceptions. International Journal of Market Research, 53(3), 369–390.Google Scholar
  19. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk a new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5.Google Scholar
  20. *Bunk, J. A. (2006). The role of appraisals, emotions, and coping in understanding experiences of workplace incivility.Google Scholar
  21. *Burton, J. P. (2014). The role of job embeddedness in the relationship between bullying and aggression. European Journal of Work and Organizational Psychology, 24(4), 518–529.Google Scholar
  22. Callegaro, M., Villar, A., Yeager, D., & Krosnick, J. A. (2014). A critical review of studies investigating the quality of data obtained with online panels based on probability and nonprobability samples. Online panel research: A data quality perspective, pp. 23–53.Google Scholar
  23. *Carlsen, J. J. (2015). An investigation of work engagement as a moderator of the relationship between personality and work outcomes (Doctoral dissertation, San Diego State University).Google Scholar
  24. *Carsten, M. K., & Uhl-Bien, M. (2012). Follower beliefs in the co-production of leadership. Zeitschrift für Psychologie, 220(4), 210–220.Google Scholar
  25. *Castille, C. M. (2015). Bright or dark, or virtues and vices? A reexamination of the big five and job performance. Louisiana Tech University.Google Scholar
  26. Chiaburu, D. S., Oh, I. S., Berry, C. M., Li, N., & Gardner, R. G. (2011). The five-factor model of personality traits and organizational citizenship behaviors: A meta-analysis. Journal of Applied Psychology, 96(6), 1140.Google Scholar
  27. Choi, D., Oh, I. S., & Colbert, A. (2015). Understanding organizational commitment: A meta-analytic examination of the roles of the five-factor model of personality and culture. Journal of Applied Psychology, 100(5), 1542–1567.Google Scholar
  28. *Chung-Yan, G. A. (2010). The nonlinear effects of job complexity and autonomy on job satisfaction, turnover, and psychological well-being. Journal of Occupational Health Psychology, 15(3), 237.Google Scholar
  29. Cochran, M. N. (2014). Counterproductive work behaviors, justice, and affect: A meta-analysis (Unpublished doctoral dissertation). University of Central Florida, Orlando, FL.Google Scholar
  30. *Cochrum-Nguyen, F. L. (2013). Predicting job performance and job satisfaction: An examination of the five-factor model of personality, polychronicity and role overload (Doctoral dissertation, San Diego State University).Google Scholar
  31. *Cohen, T. R., Panter, A. T., & Turan, N. (2013). Predicting counterproductive work behavior from guilt proneness. Journal of Business Ethics, 114(1), 45–53.Google Scholar
  32. *Cohen, T. R., Panter, A. T., Turan, N., Morse, L., & Kim, Y. (2013). Agreement and similarity in self-other perceptions of moral character. Journal of Research in Personality, 47(6), 816–830.Google Scholar
  33. *Colbert, A. E., Bono, J. E., & Purvanova, R. (2008). Development of a relationship functions inventory: Assessing the functions of high-quality work relationships. Paper presented at the annual meeting of the Academy of Management, Anaheim, CA.Google Scholar
  34. Colquitt, J. A., Conlon, D. E., Wesson, M. J., Porter, C. O., & Ng, K. Y. (2001). Justice at the millennium: A meta-analytic review of 25 years of organizational justice research. Journal of Applied Psychology, 86(3), 425.Google Scholar
  35. Cook, T. D., & Campbell, D. T. (1976). Four kinds of validity. Handbook of industrial and organizational psychology (pp. 224–246).Google Scholar
  36. *Costa, J. B. (2015). Coping through counterproductive work behaviors: An examination of how employees deal with emotional labor (Master’s thesis, Roosevelt University).Google Scholar
  37. *Credé, M., Harms, P., Niehorster, S., & Gaye-Valentine, A. (2012). An evaluation of the consequences of using short measures of the Big Five personality traits. Journal of Personality and Social Psychology, 102(4), 874.Google Scholar
  38. *Dahling, J. J., & Thompson, M. N. (2013). Detrimental relations of maximization with academic and career attitudes. Journal of Career Assessment, 21(2), 278–294.Google Scholar
  39. Dalal, R. S. (2005). A meta-analysis of the relationship between organizational citizenship behavior and counterproductive work behavior. Journal of Applied Psychology, 90(6), 1241–1255.Google Scholar
  40. *Decker, C., & Van Quaquebeke, N. (2015). Getting respect from a boss you respect: How different types of respect interact to explain subordinates’ job satisfaction as mediated by self-determination. Journal of Business Ethics, 131(3), 543–556.Google Scholar
  41. DeGroot, T., Kiker, D. S., & Cross, T. C. (2000). A meta-analysis to review organizational outcomes related to charismatic leadership. Canadian Journal of Administrative Sciences/Revue Canadienne des Sciences de l’Administration, 17(4), 356–372.Google Scholar
  42. DeSimone, J. A., Harms, P. D., & DeSimone, A. J. (2015). Best practice recommendations for data screening. Journal of Organizational Behavior, 36(2), 171–181.Google Scholar
  43. Dillman, D, A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Hoboken, NJ: John Wiley & Sons, Inc.Google Scholar
  44. Duhachek, A., Coughlan, A. T., & Iacobucci, D. (2005). Results on the standard error of the coefficient alpha index of reliability. Marketing Science, 24, 294–301.Google Scholar
  45. *Duniewicz, K. (2015). Don't get mad, get even: how employees abused by their supervisor retaliate against the organization and undermine their spouses. FIU Electronic Theses and Dissertations. Paper 1848. https://digitalcommons.fiu.edu/etd/1848.Google Scholar
  46. *Eschleman, K. J., Bowling, N. A., & Judge, T. A. (2015). The dispositional basis of attitudes: A replication and extension of Hepler and Albarracín (2013). Journal of Personality and Social Psychology, 108(5), e1–e15.Google Scholar
  47. Feitosa, J., Joseph, D. L., & Newman, D. A. (2015). Crowdsourcing and personality measurement equivalence: A warning about countries whose primary language is not English. Personality and Individual Differences, 75, 47–52.Google Scholar
  48. *Ferris, D. L., Johnson, R. E., Rosen, C. C., Djurdjevic, E., Chang, C. H. D., & Tan, J. A. (2013). When is success not satisfying? Integrating regulatory focus and approach/avoidance motivation theories to explain the relation between core self-evaluation and job satisfaction. Journal of Applied Psychology, 98(2), 342–353.Google Scholar
  49. Fisher, R. (1955). Statistical methods and scientific induction. Journal of the Royal Statistical Society. Series B (Methodological), 69–78.Google Scholar
  50. Fisher, G. G., & Sandell, K. (2015). Sampling in industrial–organizational psychology research: Now what? Industrial and Organizational Psychology, 8(02), 232–237.Google Scholar
  51. Fleischer, A., Mead, A. D., & Huang, J. (2015). Inattentive responding in MTurk and other online samples. Industrial and Organizational Psychology, 8(02), 196–202.Google Scholar
  52. *Gabler, C. B., Nagy, K. R., & Hill, R. P. (2014). Causes and consequences of abusive supervision in sales management: A tale of two perspectives. Psychology & Marketing, 31(4), 278–293.Google Scholar
  53. *Gangadharan, A. (2014). Can I smile with spirit? Towards a process model associating workplace spirituality and emotional labor (Doctoral dissertation). Available from ProQuest Dissertations & Theses database (UMI No. 3642187).Google Scholar
  54. *Giacopelli, N. M., Simpson, K. M., Dalal, R. S., Randolph, K. L., & Holland, S. J. (2013). Maximizing as a predictor of job satisfaction and performance: A tale of three scales. Judgment and Decision making, 8(4), 448–469.Google Scholar
  55. Gillespie, M. A., Gillespie, J. Z., Brodke, M. H., & Balzer, W. K. (2016). The importance of sample composition depends on the research question. Industrial and Organizational Psychology, 9(01), 207–211.Google Scholar
  56. *Goo, W. (2015). Employee needs and job-related opportuniites: From the person-environment fit framework (Doctoral dissertation). Available from ProQuest Dissertations & Theses database. (UMI No. 3681240).Google Scholar
  57. Goodman, J. K., & Paolacci, G. (2017). Crowdsourcing consumer research. Journal of Consumer Research, 44(1), 196–210.Google Scholar
  58. Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004). Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. American Psychologist, 59(2), 93.Google Scholar
  59. Greco, L., O’Boyle, E., Cockburn, B., & Yuan, Z. (2015). Raising the .70 bar: A meta-analytic study of coefficient alpha. Working Paper.Google Scholar
  60. Griffeth, R. W., Hom, P. W., & Gaertner, S. (2000). A meta-analysis of antecedents and correlates of employee turnover: Update, moderator tests, and research implications for the next millennium. Journal of Management, 26(3), 463–488.Google Scholar
  61. *Hannah, S. T., Jennings, P. L., Bluhm, D., Peng, A. C., & Schaubroeck, J. M. (2014). Duty orientation: Theoretical development and preliminary construct testing. Organizational Behavior and Human Decision Processes, 123(2), 220–238.Google Scholar
  62. Harms, P. D., & DeSimone, J. A. (2015). Caution! MTurk workers ahead—fines doubled. Industrial and Organizational Psychology, 8(02), 183–190.Google Scholar
  63. *Hausknecht, J. P., Sturman, M. C., & Roberson, Q. M. (2011). Justice as a dynamic construct: Effects of individual trajectories on distal work outcomes. Journal of Applied Psychology, 96(4), 872–880.Google Scholar
  64. Hershcovis, M. S., Turner, N., Barling, J., Arnold, K. A., Dupré, K. E., Inness, M., Mirielle LeBlanc, M., & Sivanathan, N. (2007). Predicting workplace aggression: A meta-analysis. Journal of Applied Psychology, 92(1), 228–238.Google Scholar
  65. Highhouse, S., & Gillespie, J. Z. (2009). Do samples really matter that much. Statistical and methodological myths and urban legends: Doctrine, verity and fable in the organizational and social sciences (pp. 247–265).Google Scholar
  66. Hillygus, D. S., Jackson, N., & Young, M. (2014). Professional respondents in non-probability online panels. Online panel research: A data quality perspective, 1. 219–237.Google Scholar
  67. *Holtz, B. C., & Harold, C. M. (2013a). Effects of leadership consideration and structure on employee perceptions of justice and counterproductive work behavior. Journal of Organizational Behavior, 34(4), 492–519.Google Scholar
  68. *Holtz, B. C., & Harold, C. M. (2013b). Interpersonal justice and deviance the moderating effects of interpersonal justice values and justice orientation. Journal of Management, 39(2), 339–365.Google Scholar
  69. Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012). Detecting and deterring insufficient effort responding to surveys. Journal of Business and Psychology, 27(1), 99–114.Google Scholar
  70. Huang, J. L., Liu, M., & Bowling, N. A. (2015). Insufficient effort responding: Examining an insidious confound in survey data. Journal of Applied Psychology, 100(3), 828–845.Google Scholar
  71. Jackson, T. A., Meyer, J. P., & Wang, X. H. F. (2013). Leadership, commitment, and culture: A meta-analysis. Journal of Leadership & Organizational Studies, 20(1), 84–106.Google Scholar
  72. *Jenkins, J. S., Heneghan, C. J., Bailey, S. F., & Barber, L. K. (2014). The work–family interface as a mediator between job demands and employee behaviour. Stress and Health.Google Scholar
  73. *Jeon, G. (2011). Equity sensitivity versus equity preference: Validating a new viewpoint on equity sensitivity (Unpublished doctoral dissertation). University of Illionois at Urbana-Champaign.Google Scholar
  74. *Johnson, V. A., Beehr, T. A., & O’Brien, K. E. (2015). Determining the relationship between employee psychopathy and strain: Does the type of psychopathy matter? International Journal of Stress Management, 22(2), 111–136.Google Scholar
  75. *Johnston-Fisher, J. (2014). Testing a multi-level mediation model of workgroup incivility: The role of civility climate and group norms for civility (Unpublished master’s thesis). Western Kentucky University.Google Scholar
  76. *Joseph, D. L. (2011). Emotional intelligence, leader-member exchange, and behavioral engagement: Considering mediation and reciprocity effects (Doctoral dissertation, University of Illinois at Urbana-Champaign).Google Scholar
  77. Judge, T. A., Heller, D., & Mount, M. K. (2002). Five-factor model of personality and job satisfaction: A meta-analysis. Journal of Applied Psychology, 87(3), 530–541.Google Scholar
  78. *Kiffin-Petersen, S. A., Jordan, C. L., & Soutar, G. N. (2011). The Big Five, emotional exhaustion and citizenship behaviors in service settings: The mediating role of emotional labor. Personality and Individual Differences, 50(1), 43–48.Google Scholar
  79. *Krischer, M. M., Penney, L. M., & Hunter, E. M. (2010). Can counterproductive work behaviors be productive? CWB as emotion-focused coping. Journal of Occupational Health Psychology, 15(2), 154.Google Scholar
  80. *Lambert, L. S., Tepper, B. J., Carr, J. C., Holt, D. T., & Barelka, A. J. (2012). Forgotten but not gone: An examination of fit between leader consideration and initiating structure needed and received. Journal of Applied Psychology, 97(5), 913.Google Scholar
  81. Landers, R. N., & Behrend, T. S. (2015). An inconvenient truth: Arbitrary distinctions between organizational, mechanical turk, and other convenience samples. Industrial and Organizational Psychology, 1–23.Google Scholar
  82. *Lee, J. (2012). The effects of leadership behavior on workplace harassment, employee outcomes, and organizational effectiveness in small businesses (Doctoral dissertation). Available from ABI/INFORM Global; ProQuest Dissertations & Theses database (UMI No. 3489453).Google Scholar
  83. *Long, C. P., Bendersky, C., & Morrill, C. (2011). Fairness monitoring: Linking managerial controls and fairness judgments in organizations. Academy of Management Journal, 54(5), 1045–1068.Google Scholar
  84. *Long, E. C., & Christian, M. S. (2015). Mindfulness buffers retaliatory responses to injustice: A regulatory approach. Journal of Applied Psychology, 100(5), 1409–1422.Google Scholar
  85. *Lusin, J. M. (2014). Employee perceptions of authentic leadership and outcomes of planned organizational change (Doctoral dissertation). Available from ProQuest Dissertations & Theses database (UMI No. 3615782).Google Scholar
  86. Mackey, J. D., Frieder, R. E., Brees, J. R., & Martinko, M. J. (2015). Abusive supervision: A meta-analysis and empirical review. Journal of Management, 0149206315573997.Google Scholar
  87. Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44(1), 1–23.Google Scholar
  88. *Mayer, D. M., Thau, S., Workman, K. M., Van Dijke, M., & De Cremer, D. (2012). Leader mistreatment, employee hostility, and deviant behaviors: Integrating self-uncertainty and thwarted needs perspectives on deviance. Organizational Behavior and Human Decision Processes, 117(1), 24–40.Google Scholar
  89. McGonagle, A. K. (2015). Participant motivation: A critical consideration. Industrial and Organizational Psychology, 8(02), 208–214.Google Scholar
  90. Meade, A. W. (2010). A taxonomy of effect size measures for the differential functioning of items and scales. Journal of Applied Psychology, 95(4), 728.Google Scholar
  91. Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437.Google Scholar
  92. *Meyer, R. D., Dalal, R. S., José, I. J., Hermida, R., Chen, T. R., Vega, R. P., et al. (2014). Measuring job-related situational strength and assessing its interactive effects with personality on voluntary work behavior. Journal of Management, 40(4), 1010–1041.Google Scholar
  93. *Michel, J. S., & Clark, M. A. (2009). Has it been affect all along? A test of work-to-family and family-to-work models of conflict, enrichment, and satisfaction. Personality and Individual Differences, 47(3), 163–168.Google Scholar
  94. *Michel, J. S., Newness, K., & Duniewicz, K. (2016). How abusive supervision affects workplace deviance: A moderated-mediation examination of aggressiveness and work-related negative affect. Journal of Business and Psychology, 31(1), 1–22.Google Scholar
  95. *Mullins, A. K. (2015). The dimensionality of destructive leadership: Toward an integration of the bright and dark sides. North Carolina State University.Google Scholar
  96. *Murphy, S. L. (2015). Individual adaptability as a predictor of job performance. Louisiana Tech University.Google Scholar
  97. *Nichols, A. L., & Cottrell, C. A. (2014). What do people desire in their leaders? The role of leadership level on trait desirability. The Leadership Quarterly, 25(4), 711–729.Google Scholar
  98. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.Google Scholar
  99. *O’Brien, K. E. (2008). A stressor-strain model of organizational citizenship behavior and counterproductive work behavior (Doctoral dissertation). Available from ProQuest Dissertations & Theses database (UMI No. 3347361).Google Scholar
  100. *O’Boyle, E. H. (2010). A test of the general CWB-OCB emotion model (Doctoral dissertation). Available from ABI/INFORM Global; ProQuest Dissertations & Theses database (UMI No. 3411997).Google Scholar
  101. Paolacci, G., & Chandler, J. (2014). Inside the turk understanding mechanical turk as a participant pool. Current Directions in Psychological Science, 23(3), 184–188.Google Scholar
  102. Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision making, 5(5), 411–419.Google Scholar
  103. *Penney, L. M., Hunter, E. M., & Perry, S. J. (2011). Personality and counterproductive work behaviour: Using conservation of resources theory to narrow the profile of deviant employees. Journal of Occupational and Organizational Psychology, 84(1), 58–77.Google Scholar
  104. *Petersen, N. L. (2015). Retaliatory behavior as a response to executive compensation. Bowling Green State University.Google Scholar
  105. *Porter, C., Woo, S. E., & Tak, J. (2015). Developing and validating short form protean and boundaryless career attitudes scales. Journal of Career Assessment, 1069072714565775.Google Scholar
  106. Postoaca, A. (2006). Launching the bottle: The rhetoric of the online researcher. The anonymous elect: market research through online access panels (pp. 67–107).Google Scholar
  107. *Powell, N. C. (2013). Responding to abusive supervision: Opposing arguments for the role of social class in predicting workplace deviance. (Masters thesis). Available from https://uwaterloo.ca.
  108. *Ramirez, S. A. (2015). Impulsive and premeditated counterproductive work behaviors and the moderating effects of self-monitoring and core self-evaluation. North Carolina State University.Google Scholar
  109. Ran, S., Liu, M., Marchiondo, L. A., & Huang, J. L. (2015). Difference in response effort across sample types: Perception or reality? Industrial and Organizational Psychology, 8(02), 202–208.Google Scholar
  110. *Richards, D. A., & Schat, A. C. (2011). Attachment at (not to) work: Applying attachment theory to explain individual behavior in organizations. Journal of Applied Psychology, 96(1), 169–182.Google Scholar
  111. *Rosen, C. C., Slater, D. J., & Johnson, R. E. (2013). Let’s make a deal development and validation of the ex post I-deals scale. Journal of Management, 39(3), 709–742.Google Scholar
  112. Sackett, P. R., & Larson Jr, J. R. (1990). Research strategies and tactics in industrial and organizational psychology.Google Scholar
  113. *Salvaggio, T. (2014). Towards a more complete understanding of the effects of the virtual environment on the relationship between leadership style and outcomes of LMX relationships (Doctoral dissertation). Available from ProQuest Dissertations & Theses database (UMI No. 3690538).Google Scholar
  114. Schmidt, F. L., & Hunter, J. E. (2014). Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Thousand Oaks, CA: Sage.Google Scholar
  115. Schmidt, F. L., Shaffer, J. A., & Oh, I.-S. (2008). Increased accuracy for range restriction corrections: Implications for the role of personality and general mental ability in job and training performance. Personnel Psychology, 61, 827–868.Google Scholar
  116. *Schultz, L. A. (2009). Exploring the relationship between the positive and negative sides of the work-family interface: The role of enrichment in buffering the effects of time-, strain-, and behavior-based conflict (Doctoral dissertation). Available from ProQuest Dissertations & Theses database (UMI No. 3378859).Google Scholar
  117. *Scott, K. A., & Zweig, D. (2008, May). Dispositional predictors of organizational cynicism. In ASAC (Vol. 29, No. 5).Google Scholar
  118. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton: Mifflin and Company.Google Scholar
  119. *Shao, P. (2010). Ethics-based leadership and employee ethical behavior: Examining the mediating role of ethical regulatory focus (Doctoral dissertation). Available from ProQuest Dissertations & Theses database (UMI No. 3430591).Google Scholar
  120. *Shao, P., Resick, C. J., & Hargis, M. B. (2011). Helping and harming others in the workplace: The roles of personal values and abusive supervision. Human Relations, 64(8), 1051–1078.Google Scholar
  121. *Sharif, M. M., & Scandura, T. A. (2014). Do perceptions of ethical conduct matter during organizational change? Ethical leadership and employee involvement. Journal of Business Ethics, 124(2), 185–196.Google Scholar
  122. Sharpe Wessling, K., Huber, J., & Netzer, O. (2017). MTurk character misrepresentation: Assessment and solutions. Journal of Consumer Research, 44(1), 211–230.Google Scholar
  123. Smith, R. & Hofma Brown, H. (2006). Panel and data quality: Comparing metrics and assessing claims. In Proceedings of the ESOMAR Panel Research Conference. Barcelona: ESOMAR.Google Scholar
  124. Smith, N. A., Sabat, I. E., Martinez, L. R., Weaver, K., & Xu, S. (2015). A convenient solution: Using MTurk to sample from hard-to-reach populations. Industrial and Organizational Psychology, 8(02), 220–228.Google Scholar
  125. Sparrow, N. (2007). Quality issues in online research. Journal of Advertising Research-New York, 47(2), 179.Google Scholar
  126. Sprouse, J. (2011). A validation of Amazon Mechanical Turk for the collection of acceptability judgments in linguistic theory. Behavior Research Methods, 43(1), 155–167.Google Scholar
  127. *Sprung, J. M., & Jex, S. M. (2012). Work locus of control as a moderator of the relationship between work stressors and counterproductive work behavior. International Journal of Stress Management, 19(4), 272–291.Google Scholar
  128. *Tepper, B. J., Carr, J. C., Breaux, D. M., Geider, S., Hu, C., & Hua, W. (2009). Abusive supervision, intentions to quit, and employees’ workplace deviance: A power/dependence analysis. Organizational Behavior and Human Decision Processes, 109(2), 156–167.Google Scholar
  129. *Tepper, B. J., Mitchell, M. S., Haggard, D. L., Kwan, H. K., & Park, H. M. (2015). On the exchange of hostility with supervisors: An examination of self-enhancing and self-defeating perspectives. Personnel Psychology, 68(4), 723–758.Google Scholar
  130. *Thau, S., Bennett, R. J., Mitchell, M. S., & Marrs, M. B. (2009). How management style moderates the relationship between abusive supervision and workplace deviance: An uncertainty management theory perspective. Organizational Behavior and Human Decision Processes, 108(1), 79–92.Google Scholar
  131. *Thau, S., & Mitchell, M. S. (2010). Self-gain or self-regulation impairment? Tests of competing explanations of the supervisor abuse and employee deviance relationship through perceptions of distributive justice. Journal of Applied Psychology, 95(6), 1009.Google Scholar
  132. *Thompson, C. N. (2008). Personal characteristics and the impact of transformational leadership behaviors on follower outcomes (Doctoral dissertation). Available from ABI/INFORM Global; ProQuest Dissertations & Theses database (UMI No. 3375475).Google Scholar
  133. Thoresen, C. J., Kaplan, S. A., Barsky, A. P., Warren, C. R., & de Chermont, K. (2003). The affective underpinnings of job perceptions and attitudes: A meta-analytic review and integration. Psychological Bulletin, 129(6), 914–945.Google Scholar
  134. *Toaddy, S. (2012). Validation of a measure of external organizational justice. North Carolina State University.Google Scholar
  135. Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58, 6–20.Google Scholar
  136. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70.Google Scholar
  137. *van Prooijen, J. W., & de Vries, R. E. (2016). Organizational conspiracy beliefs: Implications for leadership styles and employee outcomes. Journal of Business and Psychology, 31(4), 479–491.Google Scholar
  138. Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48.Google Scholar
  139. *Vogel, R. M., & Mitchell, M. S. (2015). The motivational effects of diminished self-esteem for employees who experience abusive supervision. Journal of Management.Google Scholar
  140. *Wall, A. (2014). Common method variance: An experimental manipulation (Doctoral dissertation). Available from ProQuest Dissertations & Theses database (UMI No. 3662469).Google Scholar
  141. *Wilson, L. M. (2015). An examination of the moderating effects of work centrality on the relationships between person-group fit and work-related outcomes (Doctoral dissertation, San Diego State University).Google Scholar
  142. *Wiltshire, J., Bourdage, J. S., & Lee, K. (2014). Honesty-humility and perceptions of organizational politics in predicting workplace outcomes. Journal of Business and Psychology, 29(2), 235–251.Google Scholar
  143. Wood, J. A. (2008). Methodology for dealing with duplicate study effects in a meta-analysis. Organizational Research Methods, 11(1), 79–95.Google Scholar
  144. *Wynne, K. T. (2012). Profiling leaders: Using a profiling approach to examine the effects of multifactor leadership on follower deviance (Master’s thesis). Available from ProQuest Dissertations & Theses database (UMI No. 1518818).Google Scholar
  145. Zimmerman, R. D. (2008). Understanding the impact of personality traits on individuals' turnover decisions: A meta-analytic path model. Personnel Psychology, 61(2), 309–348.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Management and EntrepreneurshipIndiana UniversityBloomingtonUSA
  2. 2.Department of ManagementUniversity of OregonEugeneUSA
  3. 3.Department of Management and OrganizationsUniversity of IowaIowa CityUSA

Personalised recommendations