Practical advice for conducting ethical online experiments and questionnaires for United States psychologists

Abstract

It is increasingly easy and, therefore, increasingly common to conduct experiments and questionnaire studies in online environments. However, the online environment is not a data collection medium that is familiar to many researchers or to many research methods instructors. Because of this, researchers have received little information about how to address ethical issues when conducting online research. Researchers need practical suggestions on how to translate federal and professional ethics codes into this new data collection medium. This article assists United States psychologists in designing online studies that meet accepted standards for informed consent, deception, debriefing, the right to withdraw, security of test materials, copyright of participants’ materials, confidentiality and anonymity, and avoiding harm.

References

  1. Adobe (2008). How to prevent caching of swf files. Retrieved September 29, 2008, from www.adobe.com/go/tn_14743.

  2. American Bar Association (2005). Digital signature guidelines tutorial. Retrieved September 29, 2008, from www.abanet.org/scitech/ec/isc/dsg-tutorial.html.

  3. American Psychological Association (2002). Ethical principles of psychologists and code of conduct. American Psychologist, 57, 1060–1073.

    Article  Google Scholar 

  4. American Registry for Internet Numbers (1997–2007). ARIN WHOIS database search. Retrieved September 29, 2008, from ws.arin.net/whois.

  5. Andrew, R., Foley, R., McLellan, D.., & Turnbull, R. (2004). ASP Web development with Macromedia Dreamweaver MX 2004. London: Apress.

    Google Scholar 

  6. Barbaro, M., & Zeller, T., Jr. (2006, August 9). A face is exposed for AOL Searcher No. 4417749. New York Times.

  7. Barchard, K. A., & Pace, L. A. (in press). Evaluating the effectiveness of collaborative computer-intensive projects. Interactive Learning Environments.

  8. Birnbaum, M. H. (2004a). Human research and data collection via the Internet. Annual Review of Psychology, 55, 803–832.

    PubMed  Article  Google Scholar 

  9. Birnbaum, M. H. (2004b). Methodological and ethical issues in conducting social psychology research via the Internet. In C. Sansone, C. C. Morf, & A. T. Panter (Eds.), Handbook of methods in social psychology (pp. 359–382). Thousand Oaks, CA: Sage.

    Google Scholar 

  10. Birnbaum, M. H., & Reips, U.-D. (2005). Behavioral research and data collection via the Internet. In R. W. Proctor & K.-P. L. Vu (Eds.), The handbook of human factors in Web design (pp. 471–492). Mahwah, NJ: Erlbaum.

    Google Scholar 

  11. Boothroyd, R. A. (2000). The impact of research participation on adults with severe mental illness. Mental Health Services Research, 2, 213–222.

    Article  Google Scholar 

  12. Buchanan, T., Johnson, J. A., & Goldberg, L. E. (2005). Implementing a five-factor personality inventory for use on the Internet. European Journal of Psychological Assessment, 21, 115–127.

    Article  Google Scholar 

  13. Burns, J. (2008). New window: No title bar. Retrieved September 29, 2008, from www.htmlgoodies.com/beyond/javascript/article.php/3471181.

  14. Byrom, J., & Medway, D. (2004). Cyber solutions to remote problems? Online trading in British overseas territories—A review and research agenda. International Review of Retail, Distribution, & Consumer Research, 14, 71–82.

    Article  Google Scholar 

  15. Carnegie Mellon University (2002). Home computer security. Retrieved September 29, 2008, from www.cert.org/homeusers/ HomeComputerSecurity/.

  16. Code of Federal Regulations, Part 46: Protection of Human Subjects of 2001, 45 U.S.C.A. §46. Retrieved September 29, 2008, from www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.htm.

  17. Coomber, R. (1997). Using the Internet for survey research. Sociological Research Online, 2. Retrieved September 29, 2008, from www .socresonline.org.uk/2/2/coomber.htm.

  18. Copyright Law of the United States of America of 2003, U.S.C.A. §106 et seq. Retrieved September 29, 2008, from www.copyright .gov/title17/circ92.pdf.

  19. The Counter (2008). Statistics for March, 2008. Retrieved September 29, 2008, from www.thecounter.com/stats/.

  20. Curran, K., Walters, N., & Robinson, D. (2007). Investigating the problems faced by older adults and people with disabilities in online environments. Behaviour & Information Technology, 26, 447–453.

    Article  Google Scholar 

  21. Dickinson, A., Arnott, R., & Prior, S. (2007). Methods for human-computer interaction research with older people. Behaviour & Information Technology, 26, 343–352.

    Article  Google Scholar 

  22. Electronic Signatures in Global and National Commerce Act (2000). 15 U.S.C.A. §101. Retrieved September 29, 2008, from frwebgate. access.gpo.gov/cgi-bin/getdoc.cgi?dbname=106_cong_public_laws&docid=f:publ229.106.pdf.

  23. Ess, C. (2002). Introduction: Special issue on Internet research ethics. Ethics & Information Technology, 4, 177–188.

    Article  Google Scholar 

  24. Ess, C. (2005). “Lost in translation”?: Intercultural dialogues on privacy and information ethics. Ethics & Information Technology, 7, 1–6.

    Article  Google Scholar 

  25. Ess, C. (2007). Internet research ethics. In A. Joinson, K. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford handbook of Internet psychology (pp. 487–502). Oxford: Oxford University Press.

    Google Scholar 

  26. Ess, C., & the Association of Internet Researchers (2002). Ethical decision-making and Internet research: Recommendations from the aoir ethics working committee. Retrieved September 29, 2008, from aoir.org/reports/ethics.pdf.

  27. Eysenbach, G., & Till, J. (2001). Information in practice: Ethical issues in qualitative research on Internet communities. British Medical Journal, 323, 1103–1105.

    PubMed  Article  Google Scholar 

  28. Flicker, S., Haans, D., & Skinner, H. (2004). Ethical dilemmas in research on Internet communities. Qualitative Health Research, 14, 124–134.

    PubMed  Article  Google Scholar 

  29. Frankel, M. S., & Siang, S. (1999). Ethical and legal aspects of human subjects research on the Internet. Washington, DC: American Association for the Advancement of Science.

    Google Scholar 

  30. Frick, A., Bächtiger, M.-T., & Reips, U.-D. (2001). Financial incentives, personal information and dropout in online studies. In U.-D. Reips & M. Bošnjak (Eds.), Dimensions of Internet science (pp. 209–219). Lengerich, Germany: Pabst.

    Google Scholar 

  31. Fricker, R. D., & Schonlau, M. (2002). Advantages and disadvantages of Internet research surveys: Evidence from the literature. Field Methods, 14, 347–367.

    Article  Google Scholar 

  32. Glaser, J., Dixit, J., & Green, D. P. (2002). Studying hate crimes with the Internet: What makes racists advocate racial violence? Journal of Social Issues, 58, 177–193.

    Article  Google Scholar 

  33. Göritz, A. S. (2006). The induction of mood via the WWW. Motivation & Emotion, 31, 35–47.

    Article  Google Scholar 

  34. Granello, D. H., & Wheaton, J. E. (2004). Online data collection: Strategies for research. Journal of Counseling & Development, 82, 387–393.

    Google Scholar 

  35. Hanover College (2008). Psychological research on the net. Retrieved September 29, 2008, from psych.hanover.edu/research/exponnet.html.

  36. JavaScript Kit (1997–2008). Windows and JavaScript. Retrieved September 29, 2008, from www.javascriptkit.com/javatutors/window1 .shtml.

  37. Kaplowitz, M. D., Hadlock, T. D., & Levine, R. (2004). A comparison of Web and mail survey response rates. Public Opinion Quarterly, 68,94–1011.

    Article  Google Scholar 

  38. Koster, M. (2008). A standard for robot exclusion. Retrieved September 29, 2008, from www.robotstxt.org/wc/norobots.html.

  39. Krantz, J. H., & Dalal, R. (2000). Validity of Web-based psychological research. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 35–60). San Diego: Academic Press.

    Chapter  Google Scholar 

  40. Kraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., & Couper, M. (2004). Psychological research online: Report of board of scientific affairs’ advisory group on the conduct of research on the Internet. American Psychologist, 59, 105–117.

    PubMed  Article  Google Scholar 

  41. Mangan, M. A., & Reips, U.-D. (2007). Sleep, sex, and the Web: Surveying the difficult-to-reach clinical population suffering from sexsomnia. Behavior Research Methods, 39, 233–236.

    PubMed  Article  Google Scholar 

  42. Massachusetts Institute of Technology (2005). Kerberos: The network authentication protocol. Retrieved September 29, 2008, from web.mit.edu/kerberos/.

  43. Mathy, R. M., Kerr, D. L., & Haydin, B. M. (2003). Methodological rigor and ethical considerations in Internet-mediated research. Psychotherapy: Theory, Research, Practice, Training, 40, 77–85.

    Article  Google Scholar 

  44. McCabe, S. E. (2004). Comparison of Web and mail surveys in collecting illicit drug use data: A randomized experiment. Journal of Drug Education, 34, 61–72.

    PubMed  Article  Google Scholar 

  45. Meyerson, P., & Tryon, W. W. (2003). Validating Internet research: A test of the psychometric equivalence of Internet and in-person samples. Behavior Research Methods, Instruments, & Computers, 35, 614–620.

    Article  Google Scholar 

  46. Musch, J., & Reips, U.-D. (2000). A brief history of Web experimenting. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 61–87). San Diego: Academic Press.

    Chapter  Google Scholar 

  47. National Bioethics Advisory Commission (1998). Research involving persons with mental disorders that may affect decision-making capacity. Retrieved September 29, 2008, from bioethics.georgetown .edu/nbac/capacity/TOC.htm.

  48. Nosek, B. A., Banaji, M. R., & Greenwald, A. G. (2002). eResearch: Ethics, security, design, and control in psychological research on the Internet. Journal of Social Issues, 58, 161–176.

    Article  Google Scholar 

  49. Pace, L. A., & Livingston, M. M. (2005). Protecting human subjects in Internet research. Electronic Journal of Business Ethics & Organizational Studies, 10, 35–41.

    Google Scholar 

  50. PCI Security Standards Council (2008). Payment card industry (PCI) data security standard, version 1.1. Retrieved September 29, 2008, from www.pcisecuritystandards.org/security_standards/pci_ dss.shtml.

  51. Peden, B. F., & Flashinski, D. P. (2004). Virtual research ethics: A content analysis of surveys and experiments online. In E. A. Buchanan (Ed.), Readings in virtual research ethics: Issues and controversies (pp. 1–26). Hershey, PA: Information Science Publishing.

    Google Scholar 

  52. PHD Software Systems (1996–2002). Robots.txt tutorial. Retrieved September 29, 2008, from www.santarosa.edu/~dpearson/mirrored_pages/SearchEngineWorld.com/robots_tutorial.htm.

  53. Reips, U.-D. (1995–2008). The Web experimental psychology lab. Retrieved September 29, 2008, from www.psychologie.unizh.ch/sowi/ Ulf/Lab /WebExpPsyLab.html.

  54. Reips, U.-D. (1997). Das psychologische experimentieren im Internet [Psychological experimenting on the Internet]. In B. Batinic (Ed.), Internet für Psychologen (pp. 245–265). Göttingen: Hogrefe.

    Google Scholar 

  55. Reips, U.-D. (1999). Online research with children. In U.-D. Reips, B. Batinic, W. Bandilla, M. Bosnjak, L. Gräf, K. Moser, & A. Werner (Eds.), Current Internet science: Trends, techniques, results. Zürich: Online Press.

    Google Scholar 

  56. Reips, U.-D. (2000). The Web experiment method: Advantages, disadvantages, and solutions. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 89–117). San Diego: Academic Press.

    Chapter  Google Scholar 

  57. Reips, U.-D. (2001–2008). The Web experiment list. Retrieved September 29, 2008, from genpsylab-wexlist.unizh.ch/.

  58. Reips, U.-D. (2002a). Internet-based psychological experimenting: Five dos and five don’ts. Social Science Computer Review, 20, 241–249.

    Google Scholar 

  59. Reips, U.-D. (2002b). Standards for Internet-based experimenting. Experimental Psychology, 49, 243–256.

    PubMed  Google Scholar 

  60. Roberts, L. W., & Roberts, B. (1999). Psychiatric research ethics: An overview of evolving guidelines and current ethical dilemmas in the study of mental illness. Biological Psychiatry, 46, 1025–1038.

    PubMed  Article  Google Scholar 

  61. Schmidt, W. C. (1997). World-Wide Web survey research: Benefits, potential problems, and solutions. Behavior Research Methods, Instruments, & Computers, 29, 274–279.

    Article  Google Scholar 

  62. Social Psychology Network (1996–2008). Online social psychology studies. Retrieved September 29, 2008, from www.socialpsychology. org/expts.htm.

  63. Stanton, J. M., & Rogelberg, S. G. (2001). Using Internet/Intranet Web pages to collect organizational research data. Organizational Research Methods, 4, 200–217.

    Article  Google Scholar 

  64. Taylor, L. (2002). Secure FTP 101. Retrieved September 29, 2008, from www.intranetjournal.com/articles/200208/se_08_14_02a.html.

  65. Tschabitscher, H. (2008). Top 16 free email services. Retrieved September 29, 2008, from email.about.com/cs/freeemailreviews/tp/free_email.htm.

  66. University of Mississippi (2000). PsychExperiments: Psychology experiments on the Internet. Available at psychexps.olemiss.edu/.

  67. University of New Hampshire Institutional Review Board for the Protection of Human Subjects in Research (2005). Guidelines for conducting Web-based survey research. Retrieved September 29, 2008, from www.unh.edu/osr/compliance/support/internet_research.pdf.

  68. VeriSign (1995–2008). SSL information center. Retrieved September 29, 2008, from www.verisign.com/ssl/ssl-information-center/index.html.

  69. Walstrom, M. K. (2004). Ethics and engagement in communication scholarship: Analyzing public, online support groups as researcher/ participant-experiencer. In E. A. Buchanan (Ed.), Readings in virtual research ethics: Issues and controversies (pp. 174–202). Hershey, PA: Information Science Publishers.

    Google Scholar 

  70. Web Content Accessibility Guidelines Working Group (1994–2007). How to meet WCAG 2.0: A customizable list of WCAG 2.0 requirements (success criteria) and techniques [Draft]. Retrieved September 29, 2008, from www.w3.org/WAI/WCAG20/quickref/.

  71. Wood, R. T. A., Griffiths, M. D., & Eatough, V. (2004). Online data collection from video game players: Methodological issues. CyberPsychology & Behavior, 7, 511–518.

    Google Scholar 

  72. Youd, D. (1996). What is a digital signature? An introduction to digital signatures. Retrieved September 29, 2008, from www.youdzone.com/signature.html.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Kimberly A. Barchard.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Barchard, K.A., Williams, J. Practical advice for conducting ethical online experiments and questionnaires for United States psychologists. Behavior Research Methods 40, 1111–1128 (2008). https://doi.org/10.3758/BRM.40.4.1111

Download citation

Keywords

  • Online Study
  • Internet Protocol Address
  • Online Research
  • Copyright Holder
  • Data Withdrawal