Skip to main content
SpringerLink
Log in

Seeking Convergence Between Experimental and Survey Methods for Investigating Organizational Behaviour in a Virtually Connected World

  • Chapter
  • First Online:
Towards an Integrative Psychological Science

  • 515 Accesses

Abstract

Select scholarly reviews suggest that experimental research has been losing favour in organizational behaviour (OB) research in the past three decades. We suggest that this is primarily due to sampling, setting, and selection bias issues. To counteract these challenges, we revisit the merits of experimental research especially with an emphasis on the advantages conferred by causality and efficacy. The drawback of the artificiality of the lab setting is addressed with the recommendation for a greater use of Web-based experimentation in organizational behaviour. This is especially relevant in the contemporary virtually connected world where much work is carried out via technology-enabled communication channels. Sampling, experimental manipulations, dependent variables measurement, manipulation checks, and ethical considerations of Web-based experimentation are examined in detail with examples from the OB literature.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • Anderson, N., Ones, D. S., Sinangil, H. K., & Viswesvaran, C. (2002). International collaboration on the handbook of Industrial, Work and Organizational (IWO) psychology: Editorial perspectives. The Industrial-Organizational Psychologist, 40, 106–115.

    Google Scholar 

  • Asch, S. E. (1951). Effects of group pressure upon the modification and distortion of judgments. In H. Guetzkow (Ed.), Groups, leadership and men; research in human relations (pp. 177–190). Carnegie Press.

    Google Scholar 

  • Avolio, B. J., Sosik, J. J., Kahai, S. S., & Baker, B. (2014). E-leadership: Re-examining transformations in leadership source and transmission. The Leadership Quarterly, 25(1), 105–131.

    Article  Google Scholar 

  • Banaji, M. R., & Hardin, C. D. (1996). Automatic stereotyping. Psychological Science, 7(3), 136–141.

    Article  Google Scholar 

  • Banerjee, A. V., & Duflo, E. (2009). The experimental approach to development economics. Annual Review of Economics, Annual Reviews, 1(1), 151–178.

    Google Scholar 

  • Benartzi, S., Beshears, J., Milkman, K. L., Sunstein, C. R., Thaler, R. H., Shankar, M., Tucker-Ray, W., Congdon, W. J., & Galing, S. (2017). Should governments invest more in nudging? Psychological Science, 28(8), 1041–1055.

    Google Scholar 

  • Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014). Separating the shirkers from the workers? Making sure respondents pay attention on self-administered surveys. American Journal of Political Science, 58(3), 739–753.

    Article  Google Scholar 

  • Birnbaum, M. H. (2000). Decision making in the lab and on the Web. In Psychological experiments on the internet. Academic.

    Google Scholar 

  • Birnbaum, M. H. (2004). Human research and data collection via the internet. Annual Review of Psychology, 55, 803–832.

    Article  PubMed  Google Scholar 

  • Bond, R. M., Fariss, C. J., Jones, J. J., Kramer, A. D., Marlow, C., Settle, J. E., & Fowler, J. H. (2012). A 61-million-person experiment in social influence and political mobilization. Nature, 489(7415), 295.

    Article  PubMed  Google Scholar 

  • Braddy, P. W., Meade, A. W., & Kroustalis, C. M. (2008). Online recruiting: The effects of organizational familiarity, website usability, and website attractiveness on viewers’ impressions of organizations. Computers in Human Behavior, 24(6), 2992–3001.

    Article  Google Scholar 

  • Brown, K. G., Charlier, S. D., Rynes, S. L., & Hosmanek, A. (2013). What do we teach in organizational behavior? An analysis of MBA syllabi. Journal of Management Education, 37(4), 447–471.

    Google Scholar 

  • Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data. Perspectives on Psychological Science, 6(1), 3–5.

    Article  PubMed  Google Scholar 

  • Cascio, W.F., & Aguinis, H. (2008). Research in industrial and organizational psychology from 1963 to 2007: Changes, choices, and trends. Journal of Applied Psychology, 93(5), 1062–1081.

    Google Scholar 

  • Cohen, J. (1977). Statistical power analysis for the behavioral sciences (Rev). Lawrence Erlbaum Associates Inc.

    Google Scholar 

  • Colquitt, J. A. (2008). From the editors publishing laboratory research in AMJ: A question of when not if. Academy of Management Journal, 51(4), 616–620.

    Google Scholar 

  • Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Houghton Mifflin.

    Google Scholar 

  • Danziger, K. (2000). Making social psychology experimental: A conceptual history, 1920–1970. Journal of the History of Behavioral Sciences, 36(4), 329–247.

    Article  Google Scholar 

  • Davidson, D. (1967). Causal relations. Journal of Philosophy, 64(21), 691–703.

    Article  Google Scholar 

  • Dobbin, F., & Kalev, A. (2018). Why doesn't diversity training work? The challenge for industry and academia. Anthropology Now, 10(2), 48–55. https://doi.org/10.1080/19428200.2018.1493182

  • Dodds, P.S., Muhamad, R., &Watts, D. J. (2003). An experimental study of search in global social networks. Science, 301(5634), 827–829.

    Google Scholar 

  • Dunnette, M. D., & Hough, L. (1990). Handbook of industrial and organizational psychology (2nd ed.) Consulting Psychologists Press.

    Google Scholar 

  • Eden, D. (2017). Field experiments in organizations. Annual Review of Organizational Psychology and Organizational Behavior, 4, 91–122.

    Article  Google Scholar 

  • Ferrer, R. A., Grenen, E. G., & Taber, J. M. (2015). Effectiveness of internet-based affect induction procedures: A systematic review and meta-analysis. Emotion, 15(6), 752.

    Article  PubMed  Google Scholar 

  • Flick, C. (2016). Informed consent and the Facebook emotional manipulation study. Research Ethics, 12(1), 14–28.

    Article  Google Scholar 

  • Fraley, R. C. (2004). How to conduct behavioral research over the Internet: A beginner’s guide to HTML and CGI/Perl. Guilford Press.

    Google Scholar 

  • Golembiewski, R. T. (Ed.). (2000). Handbook of organizational consultation. Marcel Dekker.

    Google Scholar 

  • Gordon, M. E., Slade, L. A., & Schmitt, N. (1986). Student guinea pigs: Porcine predictors and particularistic phenomena. The Academy of Management Review, 12(1), 160–163.

    Article  Google Scholar 

  • Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review, 102(1), 4–27.

    Google Scholar 

  • Greenberg, J., & Tomlinson, E. C. (2004). Situated experiments in organizations: Transplanting the lab to the field. Journal of Management, 30(5), 703–724.

    Article  Google Scholar 

  • Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998). Measuring individual differences in implicit cognition: The implicit association task. Journal of Personality and Social Psychology, 74, 1464–1480.

    Article  PubMed  Google Scholar 

  • Guzman, F. A., & Espejo, A. (2018). Introducing changes at work: How voice behavior relates to management innovation. Journal of Organizational Behavior, 40, 73–90.

    Article  Google Scholar 

  • Hafenbrack, A., & Vohs, K. (2018). Mindfulness meditation impairs task motivation but not performance. Organizational Behavior and Human Decision Processes, 147, 1–15.

    Article  Google Scholar 

  • Hauser, O. P., Linos, E., & Rogers, T. (2017). Innovation with field experiments: Studying organizational behaviors in actual organizations. Research in Organizational Behavior, 37, 185–198.

    Article  Google Scholar 

  • Henshel, R. L. (1980). The purposes of laboratory experimentation and the virtues of deliberate artificiality. Journal of Experimental Social Psychology, 16(5), 466–478.

    Article  Google Scholar 

  • Johnson, E. J., & Goldstein, D. G. (2003). Do defaults save lives? Science, 302, 1338–1339.

    Google Scholar 

  • Katz, D., & Kahn, R. L. (1978). The social psychology of organizations. Wiley.

    Google Scholar 

  • Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111(24), 8788–8790.

    Article  Google Scholar 

  • Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., & Couper, M. (2004). Psychological research online: Report of board of scientific affairs’ advisory group on the conduct of research on the internet. American Psychologist, 59(2), 105.

    Article  Google Scholar 

  • Kuhn, T. S. (1970). The structure of scientific revolutions (2nd ed.). The University of Chicago Press.

    Google Scholar 

  • Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863.

    Google Scholar 

  • Levitt, S. D., & List, J. A. (2007). What do laboratory experiments measuring social preferences reveal about the real world? Journal of Economic Perspectives, 21(2), 153–174.

    Google Scholar 

  • Lewin, K., Lippitt, R., & White, R. K. (1939). Patterns of aggressive behavior in experimentally created “social climates.” The Journal of Social Psychology, 10, 271–299. https://doi.org/10.1080/00224545.1939.9713366

    Article  Google Scholar 

  • Lupoli, M., Levine, E., & Greenberg, A. (2018). Paternalistic lies. Organizational Behavior and Human Decision Processes, 146, 31–50.

    Article  Google Scholar 

  • Luthans, F., Luthans, B. C., & Luthans, K. W. (2015). Organizational behavior: An evidence- based approach. Information Age Publishing (IAP).

    Google Scholar 

  • McGraw, K. O., & Wong, S. P. (1992). A common language effect size statistic. Psychological Bulletin, 111(2), 361–365.

    Article  Google Scholar 

  • McNemar, Q. (1946). Opinion-attitude methodology. Psychological Bulletin, 43(4), 289–374.

    Article  PubMed  Google Scholar 

  • Milgram, S. (1967). The small-world problem. Psychology Today, 1(1), 61–67.

    Google Scholar 

  • Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231–259.

    Article  Google Scholar 

  • Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002). E-research: Ethics, security, design, and control in psychological research on the internet. Journal of Social Issues, 58(1), 161–176.

    Article  Google Scholar 

  • Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45(4), 867–872.

    Article  Google Scholar 

  • Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on amazon mechanical Turk. Judgment and Decision Making, 5(5), 411–419.

    Google Scholar 

  • Peterson, R. A. (2001). On the use of college students in social science research: Insights from a second-order meta-analysis. Journal of Consumer Research, 28(3), 450–461.

    Article  Google Scholar 

  • Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903.

    Article  Google Scholar 

  • Reips, U.-D. (2008). How Internet-mediated research changes science. In Psychological aspects of cyberspace: Theory, research, applications (pp. 268–294). Cambridge University Press.

    Google Scholar 

  • Reips, U.-D., & Krantz, J. H. (2010). Conducting true experiments on the web. In Advanced methods for conducting online behavioral research (pp. 193–216). American Psychological Association.

    Google Scholar 

  • Reips, U.-D. (2002). Standards for Internet-based experimenting. Experimental Psychology, 49(4), 243–256.

    PubMed  Google Scholar 

  • Rogelberg, S.G. (2002). Handbook of research methods in industrial and organizational psychology. Blackwell Publishing.

    Google Scholar 

  • Scandura, T., & Williams, E. A. (2000). Research methodology in management: Current practices, trends, and implications for future research. Academy of Management Journal, 43(5), 1248–1264.

    Google Scholar 

  • Schilke, O., Levine, S. S., Kacperczyk, O., & Zucker, L. G. (2019). Special issue on experiments in organizational theory. Organization Science, 30(1), 232–234.

    Article  Google Scholar 

  • Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54(2), 93–105.

    Article  Google Scholar 

  • Singh, R. P., & Schick, A. G. (2007). Organizational behavior: Where does it fit in today's management curriculum? Journal of Education for Business, 82(6), 349–356.

    Google Scholar 

  • Soman, D. (2015). The last mile: Creating social and economic value from behavioral insights. University of Toronto Press.

    Google Scholar 

  • Sonnenfield, J. A. (1985). Shedding light on the Hawthorne studies. Journal of Organizational Behavior, 6(2), 111–130.

    Google Scholar 

  • ten Brinke, L., & Adams, G. S. (2015). Saving face? When emotion displays during public apologies mitigate damage to organizational performance. Organizational Behavior and Human Decision Processes, 130, 1–12.

    Google Scholar 

  • Thaler, R. H., & Sunstein, C. R. (2008). Nudge: Improving decisions about health, wealth, and happiness. Yale University Press.

    Google Scholar 

  • Thibaut, J. W. (1959/2017). The social psychology of groups. Routledge.

    Google Scholar 

  • Travers, J., & Milgram, S. (1969). An experimental study of the small world problem. Sociometry, 32(4), 425–443.

    Article  Google Scholar 

  • Tripathi, R., Cervone, D., & Savani, K. (2018). Are the motivational effects of autonomy-supportive conditions universal? Contrasting results among Indians and Americans. Personality and Social Psychology Bulletin, 44(9), 1287–1301.

    Article  PubMed  Google Scholar 

  • Watts, D. J. (2004). Six degrees: The science of a connected age. WW Norton & Company.

    Google Scholar 

  • Williams, C. (2007). Research methods. Journal of Business & Economic Research, 5(3), 65–72.

    Google Scholar 

  • Williams, K. D., Cheung, C. K. T., & Choi, W. (2000). Cyberostracism: Effects of being ignored over the internet. Journal of Personality and Social Psychology, 79, 748–762.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Organizational Behaviour and Human Resources Management, Indian Institute of Management Bangalore, Bannerghatta Road, Bangalore, 560076, India

    Ritu Tripathi

  2. Behavioural Sciences Lab, Indian Institute of Management Bangalore, Bannerghatta Road, Bangalore, 560076, India

    Naureen Bhullar

Authors
  1. Ritu Tripathi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Naureen Bhullar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ritu Tripathi .

Editor information

Editors and Affiliations

  1. Department of Psychology, University of Allahabad, Prayagraj, Uttar Pradesh, India

    R.C. Tripathi

  2. Centre of Behavioural and Cognitive Sciences, University of Allahabad, Prayagraj, Uttar Pradesh, India

    Bhoomika R. Kar

  3. Department of Psychology, University of Allahabad, Prayagraj, Uttar Pradesh, India

    Namita Pande

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Tripathi, R., Bhullar, N. (2022). Seeking Convergence Between Experimental and Survey Methods for Investigating Organizational Behaviour in a Virtually Connected World. In: Tripathi, R., Kar, B.R., Pande, N. (eds) Towards an Integrative Psychological Science. Springer, Singapore. https://doi.org/10.1007/978-981-16-9565-0_3

Download citation

Publish with us

Policies and ethics

Search

Navigation