Abstract
This study evaluated the psychometric equivalency of Web-based research. The Sexual Boredom Scale was presented via the World-Wide Web along with five additional scales used to validate it. A subset of 533 participants that matched a previously published sample (Watt & Ewing, 1996) on age, gender, and race was identified. An 8 × 8 correlation matrix from the matched Internet sample was compared via structural equation modeling with a similar 8 × 8 correlation matrix from the previously published study. The Internet and previously published samples were psychometrically equivalent. Co-efficient alpha values calculated on the matched Internet sample yielded reliability coefficients almost identical to those for the previously published sample. Factors such as computer administration and uncontrollable administration settings did not appear to affect the results. Demographic data indicated an overrepresentation of males by about 6% and Caucasians by about 13% relative to the U.S. Census (2000). A total of 2,230 participants were obtained in about 8 months without remuneration. These results suggest that data collection on the Web is (1) reliable, (2) valid, (3) reasonably representative, (4) cost effective, and (5) efficient.
Article PDF
Similar content being viewed by others
References
Bailey, R. D., Foote, W. E., &Throckmorton, B. (2000). Human sexual behavior: A comparison of college and Internet surveys. In M. H. Birnbaum (Ed.),Psychological experiments on the Internet (pp. 141–168). San Diego: Academic Press.
Batinic, B., Reips, U.-D., &Bosnjak, M. (Eds.) (2002).Online social sciences. Cambridge, MA: Hogrefe & Huber.
Birnbaum, M. H. (Ed.) (2000a).Introduction to behavioral research on the Internet. Upper Saddle River, NJ: Prentice-Hall.
Birnbaum, M. H. (2000b).Psychological experiments on the Internet. San Diego: Academic Press.
Browne, M. W. (1993). The use of causal indicators in covariance structure models: Some practical issues.Psychological Bulletin,114, 533–541.
Buchanan, T. (2000). Internet research: Self-monitoring and judgments of attractiveness.Behavior Research Methods, Instruments, & Computers,32, 521–527.
Buchanan, T., &Smith, J. L. (1999a). Research on the Internet: Validation of a World-Wide Web mediated personality scale.Behavior Research Methods, Instruments, & Computers,31, 565–571.
Buchanan, T., &Smith, J. L. (1999b). Using the Internet for psychological research: Personality testing on the World-Wide Web.British Journal of Psychology,90, 125–144.
Carlton, L. G., Chow, J. W., Ekkekakis, P., Shim, J., Ichiyama, R., &Carlton, M. J. (1999). A Web-based digitized video image system for the study of motor coordination.Behavior Research Methods, Instruments, & Computers,31, 57–62.
Cronk, B. C., &West, J. L. (2002). Personality research on the Internet: A comparison of Web-based and traditional instruments in take-home and in-class settings.Behavior Research Methods, Instruments, & Computers,34, 177–180.
Davis, R. N. (1999). Web-based administration of a personality questionnaire: Comparison with traditional methods.Behavior Research Methods, Instruments, & Computers,31, 572–577.
Diener, E., Emmons, R. A., Larsen, R. J., &Griffin, S. (1985). The satisfaction with life scale.Journal of Personality Assessment,49, 71–75.
Farmer, R., &Sundberg, N. D. (1986). Boredom proneness—The development and correlates of a new scale.Journal of Personality Assessment,50, 4–17.
Gartner Group (2000, October 30).Gartner says average U.S. Internet user is 41 years old with an income of $65,000 [Press release]. Stamford, CT: Author. Retrieved January 15, 2001, from http:// www.gartner.com/5_about/press_room/pr20001030a.html.
Ghiselli, E. E. (1964).Theory of psychological measurement. New York: McGraw-Hill.
Ghiselli, E. E., Campbell, J. P., &Zedeck, S. (1981).Measurement theory for the behavioral sciences. New York: W. H. Freeman.
Gordon, M. S., &Rosenblum, L. D. (2001). Audiovisual Speech Weblab: An Internet teaching and research laboratory.Behavior Research Methods, Instruments, & Computers,33, 267–269.
Honaker, L. M. (1988). The equivalency of computerized and conventional MMPI administration: A critical review.Clinical Psychology Review,8, 561–577.
Hudson, W. W., Harrison, D. F., &Crosscup, P. C. (1981). A short-form scale to measure sexual discord in dyadic relationships.Journal of Sex Research,17, 157–174.
Jerome, L. W., DeLeon, P. H., James, L. C., Folen, R., Earles, J., &Gedney, J. J. (2000). The coming of age of telecommunications in psychological research and practice.American Psychologist,55, 407–421.
Jöreskog, K. G., &Sörbom, D. (1996).LISREL 8 user’s reference guide. Chicago: Scientific Software International.
Krantz, J. H., Ballard, J., &Scher, J. (1997). Comparing the results of laboratory and World-Wide Web samples on the determinants of female attractiveness.Behavior Research Methods, Instruments, & Computers,29, 264–269.
Krantz, J. H., &Dalal, R. (2000). Validity of Web-based psychological research. In M. H. Birnbaum (Ed.),Psychological experiments on the Internet (pp. 35–60). San Diego: Academic Press.
Nolen-Hoeksema, S., &Morrow, J. (1991). A prospective study of depression and posttraumatic stress symptoms after a natural disaster: The 1989 Loma Prieta earthquake.Journal of Personality & Social Psychology,61, 115–121.
Pasveer, K. A., &Ellard, J. H. (1998). The making of a personality inventory: Help from the WWW.Behavior Research Methods, Instruments, & Computers,30, 309–313.
Reips, U.-D. (2001). The Web Experimental Psychology Lab: Five years of data collection on the Internet.Behavior Research Methods, Instruments, & Computers,33, 201–211.
Schmidt, W. C. (1997). World-Wide Web survey research made easy with WWW Survey Assistant.Behavior Research Methods, Instruments, & Computers,29, 303–304.
Smith, M. A., &Leigh, B. (1997). Virtual subjects: Using the Internet as an alternative source of subjects and research environment.Behavior Research Methods, Instruments, & Computers,29, 496–505.
Snell, W. E., &Papini, D. R. (1989). The sexuality scale: An instrument to measure sexual-esteem, sexual-depression, and sexual-preoccupation.Journal of Sex Research,26, 256–263.
Snyder, M. (1974). Self-monitoring of expressive behavior.Journal of Personality & Social Psychology,30, 526–537.
Snyder, M., Berscheid, E., &Glick, P. (1985). Focusing on the exterior and the interior: Two investigations of the initiation of personal relationships.Journal of Personality & Social Psychology,48, 1427–1439.
Snyder, M., &Gangestad, S. W. (1986). On the nature of self-monitoring: Matters of assessment, matters of validity.Journal of Personality & Social Psychology,51, 125–139.
Stanton, J. M. (1998). An empirical assessment of data collection using the Internet.Personnel Psychology,51, 709–725.
Sue, S. (1999). Science, ethnicity and bias: Where have we gone wrong?American Psychologist,54, 1070–1077.
U.S.Census Bureau (2000, November 2).U. S. Census Bureau: College enrollment of students 14 to 34 years old, by type of college, attendance status, age, and gender: October 1970 to 1998 [Report]. Washington, DC: Author. Retrieved January 22, 2001, from the World-Wide Web.
Watt, J. D., &Ewing, J. E. (1996). Toward the development and validation of a measure of sexual boredom.Journal of Sex Research,33, 57–66.
Zuckerman, M. (1979).Sensation seeking: Beyond the optimal level of arousal. Hillsdale, NJ: Erlbaum.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Meyerson, P., Tryon, W.W. Validating Internet research: A test of the psychometric equivalence of Internet and in-person samples. Behavior Research Methods, Instruments, & Computers 35, 614–620 (2003). https://doi.org/10.3758/BF03195541
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03195541