Synonyms
Definition
Pretesting is the stage in survey research when survey questions and questionnaires are tested on members of target population/study population, to evaluate the reliability and validity of the survey instruments prior to their final distribution. Pretesting is widely regarded as indispensable in survey questionnaire development and is also crucial to improve data collection for quality-of-life research. It incorporates a variety of methods or combinations of methods.
Description
Pretesting, probably as old as modern sample survey, has evidently become a well-established practice by the late 1930s or early 1940s, as indicated by the title “Pretesting of Questionnaire” of Sletto’s (1940) paper in American Sociological Review and the remarks made by Katz (1940):
“In social surveys, moreover, it is an accepted practice to pretest attitudinal material to insure the exclusion of questions which can not be answered meaningfully by respondents. And this...
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Andrews, F. M. (1984). Construct validity and error components of survey measures: A structural modeling approach. Public Opinion Quarterly, 48(2), 409–442.
Bassili, J. N., & Scott, B. S. (1996). Response latency as a signal to question problems in survey research. Public Opinion Quarterly, 60(3), 390–399.
Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71(2), 287–311.
Belson, W. A. (1981). The design and understanding of survey questions. Aldershot, UK: Gower.
Biemer, P. (2004). Modeling measurement error to identify flawed questions. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 225–246). New York: Wiley.
Converse, J. M., & Presser, S. (1986). Survey questions: Handcrafting the standardized questionnaire. Thousand Oaks, CA: Sage Publications Incorporated.
de Leeuw, E., Borgers, N., & Smits, A. (2004). Pretesting questionnaires for children and adolescents. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 409–429). New York: Wiley.
Draisma, S., & Dijkstra, W. (2004). Response latency and (para) linguistic expressions as indicators of response error. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 131–147). New York: Wiley.
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215.
Forsyth, B., Rothgeb, J. M., & Willis, G. B. (2004). Does pretesting make a difference? an experimental test. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 525–546). New York: Wiley.
Fowler, F. J. (2004). The case for more split-sample experiments in developing survey instruments. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 173–188). New York: Wiley.
Harkness, J., Pennell, B. E., & Schoua-Glusberg, A. (2004). Survey questionnaire translation and assessment. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 453–473). New York: Wiley.
Katz, D. (1940). Three criteria: Knowledge, conviction, and significance. Public Opinion Quarterly, 4(2), 277–284.
Loftus, E. (1984). Protocol analysis of responses to survey recall questions. In T. B. Jabine, M. L. Straf, J. M. Tanur, & R. Tourangeau (Eds.), Cognitive aspects of survey methodology: Building a bridge between disciplines (pp. 61–64). Washington, DC: National Academy Press.
Martin, E. (2004). Vignettes and respondent debriefing for questionnaire design and evaluation. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 149–171). New York: Wiley.
Moore, J., Pascale, J., Doyle, P., Chan, A., & Griffiths, J. K. (2004). Using field experiments to improve instrument design: The SIPP methods panel project. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 189–207). New York: Wiley.
Presser, S., & Blair, J. (1994). Survey pretesting: Do different methods produce different results? Sociological Methodology, 24, 73–104.
Presser, S., Couper, M. P., Lessler, J. T., Martin, E., Martin, J., Rothgeb, J. M., et al. (2004). Methods for testing and evaluating survey questions. Public Opinion Quarterly, 68(1), 109–130.
Reeve, B. B., & Mâsse, L. C. (2004). Item response theory modeling for questionnaire evaluation. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 247–273). New York: Wiley.
Rothgeb, J., Willis, G., & Forsyth, B. (2001). Questionnaire pretesting methods: Do different techniques and different organizations produce similar results. Paper presented at the Proceedings of the Annual Meeting of the American Statistical Association. August 5–9, 2001 (Accessed on May 8, 2013: http://www.amstat.org/sections/srms/Proceedings/y2001/Proceed/00476.pdf).
Saris, W. E., van der Veld, W., & Gallhofer, I. (2004). Development and improvement of questionnaires using predictions of reliability and validity. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 275–297). New York: Wiley.
Sletto, R. F. (1940). Pretesting of questionnaires. American Sociological Review, 5(2), 193–200.
Smith, T. W. (2004). Developing and evaluating cross-national survey instruments. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 431–452). New York: Wiley.
Tourangeau, R. (2004). Experimental design considerations for testing and evaluating questionnaires. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 209–224). New York: Wiley.
U.S. Bureau of the Census (2003). Census bureau standard: Pretesting questionnaires and related materials for surveys and censuses. Washington, DC: U.S. Bureau of the Census.
Van der Zouwen, J., & Smit, J. H. (2004). Evaluating survey questions by analyzing patterns of behavior codes and question-answer sequences: A diagnostic approach. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 109–130). New York: Wiley.
Willis, G., Schechter, S., & Whitaker, K. (1999). A comparison of cognitive interviewing, expert review, and behavior coding: What do they tell us. Paper presented at the Proceedings of the Section on Survey Research Methods, American Statistical Association. (Accessed on May 8, 2013: http://www.amstat.org/sections/srms/proceedings/papers/1999_006.pdf)
Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage Publications Incorporated.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media Dordrecht
About this entry
Cite this entry
Hu, S. (2014). Pretesting. In: Michalos, A.C. (eds) Encyclopedia of Quality of Life and Well-Being Research. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-0753-5_2256
Download citation
DOI: https://doi.org/10.1007/978-94-007-0753-5_2256
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-0752-8
Online ISBN: 978-94-007-0753-5
eBook Packages: Humanities, Social Sciences and LawReference Module Humanities and Social SciencesReference Module Business, Economics and Social Sciences