Survey Development for Adolescents Aged 11–16 Years: A Developmental Science Based Guide
Methodological knowledge on surveying young adolescents is scarce and researchers often rely on theories and methodological studies based on adult respondents. However, young adolescents are in the process of developing their cognitive, psychological, emotional and social skills, therefore present a unique set of considerations. Question characteristics including; question type and format, question difficulty, wording, ambiguity, the number of response options, and the inclusion of a neutral mid-point, play a pivotal role in the response quality of young adolescents. Failure to address these factors is likely to encourage young adolescents to use satisficing techniques. In this article, we provide a science based guide for developing surveys for use with adolescents aged 11–16 years. The guide considers the characteristics and developmental stages of adolescents as survey responders and incorporates advice on appropriate question characteristics, survey layout and question sequence, approaches to pre-testing surveys and mode of survey administration. The guide provides recommendations for developmentally appropriate survey design to improve response quality in survey research with young adolescents.
KeywordsSurvey development Adolescents Satisficing Respondent characteristic Question characteristic
AO created the first draft of the article, and JWS, JS and NB provided substantive feedback on subsequent drafts. After several iterations where all authors contributed, all authors approved the final version of the article.
There were no forms of financial support, funding, or involvement.
Compliance with Ethical Standards
Conflict of interest
The authors report no conflict of interests.
- Alm-Roijer, C., Stagmo, M., Uden, G., & Erhardt, L. (2004). Better knowledge improves adherence to lifestyle changes and medication in patients with coronary heart disease. European Journal of Cardiovascular Nursing, 3(4), 321–330. https://doi.org/10.1016/j.ejcnurse.2004.05.002.CrossRefPubMedPubMedCentralGoogle Scholar
- Arthur, A. M., Smith, M. H., White, A. S., Hawley, L., & Koziol, N. A. (2017). Age-sensitive instrument design for youth: A developmental approach. Retrieved from http://cyfs.unl.edu/resources/downloads/working-papers/MAP-working-paper-2017-1.pdf. Accessed November, 2 2017.
- Borgers, N., & Hox, J. J. (2000). Reliability of responses in questionnaire research with children. In The fifth international conference on logic and methodology. Cologne, Germany.Google Scholar
- Borgers, N., & Hox, J. J. (2001). Item nonresponse in questionnaire research with children. Journal of Official Statistics, 17(2), 321–335.Google Scholar
- Borgers, N., Hox, J. J., & Sikkel, D. (2004). Response effects in surveys on children and adolescents: The effect of number of response options, negative wording, and neutral mid-point. Quality and Quantity, 38(1), 17–33. https://doi.org/10.1023/b:ququ.0000013236.29205.a6.CrossRefGoogle Scholar
- Bowling, A. (2014). Research methods in health: Investigating health and health services (4th edn.). Maidenhead: McGraw-Hill Open University Press.Google Scholar
- Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions the definitive guide to questionnaire design—for market research, political polls, and social and health questionnaires. San Francisco: Jossey-Bass Inc.Google Scholar
- Bryman, A. (2012). Social research methods (4th edn.). Oxford: University Press.Google Scholar
- Colosi, R. (2005). Negatively worded questions cause respondent confusion. In U. B. o. t. Census (pp. 2896–2903). Suitland: ASA Section on Survey Research Methods.Google Scholar
- Cronbach, L. J. (1988). Five perspectives on validity argument. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 3–16). New York: Routledge.Google Scholar
- de Leeuw, E., Natacha, B., & Astrid, S. (2004). Pretesting questionnaires for children and adolescents. In S. Presser, J. M. Rothgeb, M. P. Couper, J. T. Lessler, E. Martin, J. Martin & E. Singer (Eds.), Methods for testing and evaluating survey questionnaires (pp. 409–429). New Jersey: John Wiley & Sons.CrossRefGoogle Scholar
- de Leeuw, E. D. (2011). Improving data quality when surveying children and adolescents: Cognitive and social development and its role in questionnaire construction and pretesting. In Report prepared for the Annual Meeting of the Academy of Finland: Research Programs Public Health, Finland.Google Scholar
- De Vaus, D. (2014). Surveys in social research. New York: Routledge.Google Scholar
- Deutskens, E., de Ruyter, K., Wetzels, M., & Oosterveld, P. (2004). Response rate and response quality of internet-based surveys: An experimental study. Marketing Letters, 15(1), 21–36. https://doi.org/10.1023/b:mark.0000021968.86465.00.CrossRefGoogle Scholar
- DeVellis, R. F. (2016). Scale development: Theory and applications (4th edn.). London: SAGE.Google Scholar
- Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone mail and mixed-mode surveys: The Tailored design method (4th edn.). New York: Wiley.Google Scholar
- DuBay, W. H. (2007). Smart language: Readers, readability, and the grading of text. Costa Mesa: Booksurge Publishing.Google Scholar
- Flesch, R. (1962). The art of plain talk. Revised edition. New York: Harper and Row.Google Scholar
- Fuchs, M. (2005). Children and adolescents as respondents. Experiments on question order, response order, scale effects and the effect of numeric values associated with response options. Journal of Official Statistics, 21(4), 701–725.Google Scholar
- Haeger, H., Lambert, A. D., Kinzie, J., & Gieser, J. (2012). Using cognitive interviews to improve survey instruments. New Orleans: The Association for Institutional Research Annual Forum.Google Scholar
- Harter, S. (2012a). Manual for the self-perception profile for adolescents. Denver, CO: University of Denver.Google Scholar
- Harter, S. (2012b). Manual for the self-perception profile for children. Denver, CO: University of Denver.Google Scholar
- Kincaid, J. P., Fishburne, R., Rogers, R., & Chissom, B. S. (1975). Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for navy enlisted personnel. Defense Technical Information Center. https://doi.org/10.21236/ada006655.CrossRefGoogle Scholar
- Kline, P. (2000). The handbook of psychological testing (2nd edn.). London: Routledge.Google Scholar
- Krosnick, J. A., & Presser, S. (2010). Question and questionnaire design. In J. D. Wright & P. V. Marsden (Ed.), Handbook of survey research (pp. 263–314). Bingley: Emerald Group Publishing Ltd.Google Scholar
- Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x.CrossRefGoogle Scholar
- Leshem, R. (2016). Brain development, impulsivity, risky decision making, and cognitive control: Integrating cognitive and socioemotional processes during adolescence—an introduction to the special issue. Developmental Neuropsychology, 41(1–2), 1–5. https://doi.org/10.1080/87565641.2016.1187033.CrossRefPubMedPubMedCentralGoogle Scholar
- Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 1–55.Google Scholar
- Lissitz, R. W., & Hou, X. (2012). Multiple choice items and constructed response items: Does it matter? Retrieved from http://www.education.umd.edu/MARC/multiplechoiceitemsandconstructedresponseitems.pdf. Accessed June 25, 2016.
- Mavletova, A., & Lynn, P. (2017). Data quality in the understanding society youth self-completion questionnaire. Understanding Society at the Institute for Social and Economic Research (No. 2017–8).Google Scholar
- McLaughlin, G. H. (1969). SMOG grading: A new readability formula. Journal of Reading, 12(8), 639–646.Google Scholar
- Mellor, D., & Moore, K. A. (2003). The questionnaire on teacher interaction: Assessing information transfer in single and multi-teacher environments. Journal of Classroom Interaction, 38(2), 29–35.Google Scholar
- Michalos, A. C., Creech, H., Swayze, N., Kahlke, M., Buckler, P., C., & Rempel, K. (2011). Measuring knowledge, attitudes and behaviours concerning sustainable development among tenth grade students in Manitoba. Social Indicators Research, 106(2), 213–238. https://doi.org/10.1007/s11205-011-9809-6.CrossRefGoogle Scholar
- Mitchell, M. L., & Jolley, J. M. (2013). Research design explained (8th edn.). Boston: Wadsworth Publishing Co.Google Scholar
- Moser, C., & Kalton, G. (1985). Survey methods in social investigation (2nd edn.). Aldershot: Gower.Google Scholar
- Nevo, B. (1985). Face validity revisited. Journal of Educational Measurement, 22(4), 287–293. https://doi.org/10.1111/j.1745-3984.1985.tb01065.x.CrossRefGoogle Scholar
- Osgood, C. E. E., Suci, G. J., & Tannenbaum, P. H. (1957). The measurement of meaning. Champaign: University of Illinois Press.Google Scholar
- Piaget, J. (1929). Child’s conception of the world (international library of psychology). London: Routledge and Kegan Paul.Google Scholar
- Pole, C., & Lampard, R. (2013). Practical social investigation. Qualitative and quantitative methods in social research. Harlow: Printice Hall.Google Scholar
- Quaigrain, K., & Arhin, A. K. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1). https://doi.org/10.1080/2331186x.2017.1301013.
- Rea, L. M., & Parker, R. A. (2005). Designing and conducting survey research: A comprehensive guide (3rd edn.). San Francisco: Jossey-Bass.Google Scholar
- Reja, U., Manfreda, K. L., Hlebec, V., & Vehovar, V. (2003). Open-ended vs. close-ended questions in web questionnaires. Developments in Applied Statistics, 19, 159–177.Google Scholar
- Robson, C. (2011). Real world research: A resource for users of social research methods in applied settings (3rd edn.). Chichester: Wiley.Google Scholar
- Rohrmann, B. (2007). Verbal qualifiers for rating scales: Sociolinguistic considerations and psychometric data. University of Melbourne/Australia. Retrieved from http://www.rohrmannresearch.net/pdfs/rohrmann-vqs-report.pdf. Accessed: 15 April, 2016.
- Rossi, P. H., Wright, J. D., & Anderson, A. B. (2013). Handbook of survey research. New York: Academic Press.Google Scholar
- Saffi, M. A. L., Macedo Junior Jacques de, L. J., Trojahn, M. M., Polanczyk, C. A., & Rabelo-Silva, E. (2013). Validity and reliability of a questionnaire on knowledge of cardiovascular risk factors for use in Brazil. Revista Da Escola de Enfermagem Da USP, 47(5), 1083–1089. https://doi.org/10.1590/s0080-623420130000500011.CrossRefGoogle Scholar
- Schuman, H., & Presser, S. (1996). Questions and answers in attitude surveys: Experiments on question form, wording, and context. New York: Academic Press.Google Scholar
- Seale, C. (2004). Social research methods: A reader. London: Routledge.Google Scholar
- Smith, K., & Platt, L. (2013). How do children answer questions about frequencies and quantities? Evidence from a large-scale field test. Centre for Longitudinal Studies, Institute of Education, University of London.Google Scholar
- Tan, C. L., Hassali, M. A., Saleem, F., Shafie, A. A., Aljadhey, H., & Gan, V. B. (2015). Development, test-retest reliability and validity of the pharmacy value-added services questionnaire (PVASQ). Pharmacy Practice, 13(3), 598. https://doi.org/10.18549/pharmpract.2015.03.598.CrossRefPubMedPubMedCentralGoogle Scholar
- Turner, J. (1975). Cognitive development. London: Methue.Google Scholar
- van Hattum, M. J. C., & de Leeuw, E. (1999). A disk by mail survey of pupils in primary schools: Data quality and logistics. Journal of Official Statistics, 15(3), 413–429.Google Scholar
- Wallace, C. M. (2009). Measuring changes in attitude, skill and knowledge of undergraduate nursing students after receiving an educational intervention in intimate partner violence.Google Scholar
- Willis, G. G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. In PsycEXTRA dataset. Thousand Oaks: SAGE Publications.Google Scholar
- Zanolin, M. E., Visentin, M., Trentin, L., Saiani, L., Brugnolli, A., & Grassi, M. (2007). A questionnaire to evaluate the knowledge and attitudes of health care providers on pain. Journal of Pain and Symptom Management, 33(6), 727–736. https://doi.org/10.1016/j.jpainsymman.2006.09.032.CrossRefPubMedPubMedCentralGoogle Scholar