Differences in responses to Web and paper surveys among school professionals
This experiment investigated the effects of survey administration mode (Web vs. paper and pencil) and survey length (short or long) on the responses of a large sample of primary and secondary school professionals in the U.S. The 812 participants in this study were part of an initial random sample of 1,000 individuals representing 5.81% of the membership of a national professional organization. The participants were randomly assigned to each of the four treatment conditions. Results indicated (1) substantially lower response rates for Web surveys than for same-length paper surveys; (2) a higher response rate for short surveys than for long surveys with paper, but not Web, questionnaires; and (3) a younger age for Web respondents, as compared with their paper counterparts. In light of prior research, we suggest that paper-and-pencil methods be used for surveying professionals in primary and secondary school settings.
- Bachmann, D. P., Elfrink, J., & Vazzana, G. (2000). E-mail and snail mail face off in rematch. Marketing Research, 11, 10–15.
- Birnbaum, M. H. (1999). Testing critical properties of decision making on the Internet. Psychological Science, 10, 399–407. doi:10.1111/1467-9280.00176 CrossRef
- Cobanoglu, C., Warde, B., & Moreo, P. J. (2001). A comparison of mail, fax, and Web-based survey methods. International Journal of Market Research, 43, 441–455.
- Cook, C., Heath, F., & Thompson, R. L. (2000). A meta-analysis of response rates in Web- or Internet-based surveys. Educational & Psychological Measurement, 60, 821–836. doi:10.1177/00131640021970934 CrossRef
- Couper, M. P. (2005). Technology trends in survey data collection. Social Science Computer Review, 23, 486–501. doi:10.1177/ 0894439305278972 CrossRef
- Curtis, M. J., Hunley, S. A., Walker, K. J., & Baker, A. C. (1999). Demographic characteristics and professional practices in school psychology. School Psychology Review, 28, 104–116.
- Curtis, M. J., Lopez, A. D., Batsche, G. M., & Smith, J. C. (2006, March). School psychology 2005: A national perspective. Paper presented at the meeting of the National Association of School Psychologists, Anaheim, CA.
- Dillman, D. (2000). Main and Internet surveys: The tailored design method (2nd ed.). New York: Wiley.
- Diment, K., & Garrett-Jones, S. (2007). How demographic characteristics affect mode preference in a postal/Web mixed-mode survey of Australian researchers. Social Science Computer Review, 25, 410–417. Retrieved May 4, 2009, from http://ro.uow.edu.au/commpapers/304/. doi:10.1177/0894439306295393 CrossRef
- Fagan, T. K., Gorin, S., & Tharinger, D. (2000). The National Association of School Psychologists and the Division of School Psychology-APA: Now and beyond. School Psychology Review, 29, 525–535.
- Feldt, L. S., & Ankenmann, R. D. (1999). Determining sample size for a test of the equality of alpha coefficients when the number of parttests is small. Psychological Methods, 4, 366–377. doi:10.1037/1082-989X.4.4.366 CrossRef
- Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004). Should we trust Web-based studies? A comparative analysis of six preconceptions about Internet questionnaires. American Psychologist, 59, 93–104. doi:10.1037/0003-066X.59.2.93 CrossRef
- Heberlein, T. A., & Baumgartner, R. (1978). Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literature. American Sociological Review, 43, 447–462. CrossRef
- Hosmer, D. W., & Lemeshow, S. (2000). Applied logistic regression (2nd ed.). New York: Wiley. CrossRef
- Individuals With Disabilities Education Improvement Act of 2004. 20 U.S.C. 1400.
- Kaplowitz, M. D., Hadlock, T. D., & Levine, R. (2004). A comparison of Web and mail survey response rates. Public Opinion Quarterly, 68, 94–101. doi:10.1093/poq/nfh006 CrossRef
- LaRose, R., Mastro, D., & Eastin, M. S. (2001). Understanding Internet usage: A social-cognitive approach to uses and gratifications. Social Science Computer Review, 19, 395–413. CrossRef
- Marcus, B., Bosnjak, M., Lindner, S., Pilischenko, S., & Schutz, A. (2007). Compensating for low topic interest and long surveys: A field experiment on nonresponse in Web surveys. Social Science Computer Review, 25, 372–383. doi:10.1177/0894439307297606 CrossRef
- McCabe, S. E., Couper, M. P., Cranford, J. A., & Boyd, C. J. (2006). Comparison of Web and mail surveys for studying secondary consequences associated with substance use: Evidence for minimal mode effects. Addictive Behaviors, 31, 162–168. doi:10.1016/ j.addbeh.2005.04.018 CrossRef
- McDonald, H., & Adam, S. (2003). A comparison of online and postal data collection methods in marketing research. Marketing Intelligence & Planning, 21, 85–95. doi:10.1108/02634500310465399 CrossRef
- Mertler, C. A. (2003). Patterns of response and nonresponse from teachers to traditional and Web surveys. Practical Assessment, Research & Evaluation, 8, Art. 22. Retrieved January 22, 2009, from http://PAREonline.net/getvn.asp?v=8&n=22.
- Microsoft Corporation (2003). Microsoft Office FrontPage (Version 2003) [Computer software]. Redmond, WA: Microsoft, Inc.
- Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: What can be done? Assessment & Evaluation in Higher Education, 33, 301–314. doi:10.1080/02602930701293231 CrossRef
- Pasveer, K. A., & Ellard, J. H. (1998). The making of a personality inventory: Help from the WWW. Behavior Research Methods, Instruments, & Computers, 30, 309–313. CrossRef
- Reips, U.-D. (2002). Standards for Internet-based experimenting. Experimental Psychology, 49, 243–256. doi:10.1026//1618-322.214.171.124
- Reips, U.-D., & Lengler, R. (2005). The Web Experiment List: A Web service for the recruitment of participants and archiving of Internet-based experiments. Behavior Research Methods, 37, 287–292. CrossRef
- Sheehan, K. B. (2001). E-mail survey response rates: A review. Journal of Computer-Mediated Communication, 6(2). Retrieved April 16, 2009, from www3.interscience.wiley.com.argo.library.okstate.edu/ cgi-bin/fulltext/120837811/HTMLSTARTW.
- Sheehan, K. B., & McMillan, S. J. (1999). Response variation in email surveys: An exploration. Journal of Advertising Research, 39, 45–54.
- Solomon, D. J. (2001). Conducting Web-based surveys. Practical Assessment, Research & Evaluation, 7, Art. 19. Retrieved January 22, 2009, from http://PAREonline.net/getvn.asp?v=7&n=19.
- Stanton, J. M., & Rogelberg, S. G. (2001). Using Internet/intranet Web pages to collect organizational research data. Organizational Research Methods, 4, 200–217. doi:10.1177/109442810143002 CrossRef
- Weible, R., & Wallace, J. (1998). Cyber research: The impact of the Internet on data collection. Marketing Research, 10, 19–25.
- Yetter, G., & Doll, B. (2007). The impact of logistical resources on pre-referral team acceptability. School Psychology Quarterly, 22, 340–357. doi:10.1037/1045-38126.96.36.1990 CrossRef
- Differences in responses to Web and paper surveys among school professionals
Behavior Research Methods
Volume 42, Issue 1 , pp 266-272
- Cover Date
- Print ISSN
- Online ISSN
- Additional Links
- Industry Sectors