Advertisement

Research in Higher Education

, Volume 46, Issue 2, pp 127–152 | Cite as

Non-response in student surveys: The Role of Demographics, Engagement and Personality

  • Stephen R. Porter
  • Michael E. Whitcomb
Article

Abstract

What causes a student to participate in a survey? This paper looks at participation across multiple surveys to understand survey non-response; by using multiple surveys we minimize the impact of survey salience. Students at a selective liberal arts college were administered four different surveys throughout the 2002–2003 academic year, and we use the number of surveys participated in to understand how student characteristics such as demographics, engagement and Holland personality type affect cooperation. We find that survey respondents are more likely to be female and socially engaged, less likely to be on financial aid, more likely to be an investigative personality type and less likely to be an enterprising personality type.

Keywords

survey non-response student engagement Holland personality type survey methodology ordered logit 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Adams, L. L. M., Gale, D. 1982Solving the quandry between questionnaire length and response rate in educational researchResearch in Higher Education17231240Google Scholar
  2. 2.
    American Association for Public Opinion Research2004Standard definitions: Final dispositions of case codes and outcome rates for surveysKSLenexaGoogle Scholar
  3. 3.
    Aviv, A. L., Zelenski, J. M., Rallo, L., Larsen, R. J. 2002Who comes when: Personality differences in early and later participation in a university subject poolPersonality and Individual Differences33487496Google Scholar
  4. 4.
    Barrick, M. R., Mount, M. K., Gupta, R. 2003Meta-analysis of the relationship between the Five-Factor Model of personality and Holland’s Occupational typesPersonal Psychology564674Google Scholar
  5. 5.
    Bradburn, N. M. (1978). Respondent burden. Paper presented at the The American Statistical Association, Proceedings of the Survey Research Methods Section.Google Scholar
  6. 6.
    Brennan, M., Hoek, J. 1992The behavior of respondents, nonrespondents, and refusers across mail surveysPublic Opinion Quarterly56530535Google Scholar
  7. 7.
    Cooper, H., Baumgardner, A. H., Strathman, A. 1991Do students with different characteristics take part in psychology experiments at different times of the semester?Journal of Personality59109127Google Scholar
  8. 8.
    Couper, M. P. 1997Survey introductions and data qualityPublic Opinion Quarterly61317338Google Scholar
  9. 9.
    Couper, M. P., Traugott, M. W., Lamias, M. J. 2001Web survey design and administrationPublic Opinion Quarterly65250253Google Scholar
  10. 10.
    Crawford, S. D., Couper, M. P., Lamias, M. J. 2001Web surveys: Perceptions of burdenSocial Science Computer Review19146162Google Scholar
  11. 11.
    Curtin, R., Presser, S., Singer, E. 2000The effects of response rate changes on the index of consumer sentimentPublic Opinion Quarterly64413428Google Scholar
  12. 12.
    De Leeuw, E., Heer, W. 2002Trends in household survey nonresponse: A longitudinal and international comparisonGroves, R. M.Dillman, D. A.Eltinge, J. L.Little, R. J. A. eds. Survey NonresponseJohn Wiley & SonsNew YorkGoogle Scholar
  13. 13.
    Dey, E. 1997Working with low survey response rates: The efficacy of weighting adjustmentsResearch in Higher Education38215227Google Scholar
  14. 14.
    Dommeyer, C. J., Moriarty, E. 2000Comparing two forms of an email survey: Embedded vs. attachedInternational Journal of Market Research423950Google Scholar
  15. 15.
    Evans, R., Donnerstein, E. 1974Some implications for psychological research of early versus late term participation by college studentsJournal of Research in Personality8102109Google Scholar
  16. 16.
    Goyder, J. 1982Further evidence on factors affecting response rates to mailed questionnairesAmerican Sociological Review47550553Google Scholar
  17. 17.
    Goyder, J. 1986Surveys on surveys: Limitations and potentialitiesPublic Opinion Quarterly502741Google Scholar
  18. 18.
    Goyder, J. 1987The Silent Minority: Nonrespondents in Sample SurveysWestview PressBoulderGoogle Scholar
  19. 19.
    Goyder, J., Warriner, K., Milller, S. 2002Evaluating socio-economic status (SES) bias in survey nonresponseJournal of Official Statistics18111Google Scholar
  20. 20.
    Greene, W. H. 1997Econometric AnalysisPrentice HallUpper Saddle River, NJGoogle Scholar
  21. 21.
    Groves, R. M., Cialdini, R. B., Couper, M. P. 1992Understanding the decision to participate in a surveyPublic Opinion Quarterly56475495Google Scholar
  22. 22.
    Groves, R. M., Couper, M. P. 1996Contact-level influences on cooperation in face-to-face surveysJournal of Official Statistics126383Google Scholar
  23. 23.
    Groves, R. M., Singer, E., Corning, A. 2000Leverage-saliency theory of survey participationPublic Opinion Quarterly64299308Google Scholar
  24. 24.
    Heberlein, T. A., Baumgartner, R. 1978Factors affecting response rates to mailed questionnaires: A quantitative analysis of the published literatureAmerican Sociological Review43447462Google Scholar
  25. 25.
    Holland, J. L. 1966The Psychology of Vocational ChoiceBlaisdellWaltham, MAGoogle Scholar
  26. 26.
    Holland, J. L. 1985Making Vocational ChoicesPrentice-HallEnglewood Cliffs, NJGoogle Scholar
  27. 27.
    Hutchinson, J., Tollefson, N., Wigington, H. 1987Response bias in college freshman’s response to mail surveysResearch in Higher Education2699106Google Scholar
  28. 28.
    Johnson, J. S., Mowrer, R. R. 2000Mail surveys among college students:Do initial responders score differently from nonresponders on the 16PF?Psychological Reports86901908Google Scholar
  29. 29.
    Kaldenberg, D. O., Koenig, H. F., Becker, B. W. 1994Mail survey response rate patterns in a population of the elderly: Does response deteriorate with age?Public Opinion Quarterly586876Google Scholar
  30. 30.
    Kandel, D., Raveis, J. L. 1983Sex differences in the characteristics of members lost to a longitudinal panel: A speculative research notePublic Opinion Quarterly47567575Google Scholar
  31. 31.
    Kojetin, B. A., Borgida, E. , and Snyder, M. (1993). Survey Topic Involvement and Nonresponse Bias. Paper presented at the Proceeding of the Section on Survey Research Method, Volume II, American Statistical AssociationGoogle Scholar
  32. 32.
    Kuh, G. D. 2003The National Survey of Student Engagement: Conceptual Framework and Overview of Psychometric PropertiesIndiana University Center for Postsecondary ResearchBloomington, INGoogle Scholar
  33. 33.
    Kwak, N., Radler, B. 2002A comparison between mail and web surveys: Response pattern, respondent profileand data qualityJournal of Official Statistics18257273Google Scholar
  34. 34.
    Lahaut, V. M. H. C. J., Jansen, H. A. M., van de Mheen, D., Garretsen, H. F. L., Verdurmen, J. E. E., van Dijk, A. 2003Estimating non-response bias in a survey on alcohol consumption: Comparison of response wavesAlcohol and Alcoholism38128134Google Scholar
  35. 35.
    Larson, L. M., Rottingham, P. J., Borgen, F. 2002Meta-analysis of big six interests and big five personality factorsJournal of Vocational Behavior61217239Google Scholar
  36. 36.
    Laurie, H., Smith, R., Scott, L. 1999Strategies for reducing nonresponse in a longitudinal panel surveyJournal of Official Statistics15269282Google Scholar
  37. 37.
    Liberty, H. J. 1993The relationship between extraversion and time of data collectionPersonality and Individual Differences14835836Google Scholar
  38. 38.
    Lin, I.-F., Schaeffer, N. C. 1995Using survey participants to estimate the impact of nonparticipationPublic Opinion Quarterly59236258Google Scholar
  39. 39.
    Long, J. S. 1997Regression Models for Categorical and Limited Dependent VariablesSage PublicationsThousand Oaks, CAGoogle Scholar
  40. 40.
    Lowe, F., McCormick, T. C. 1955Some Survey Sampling BiasesPublic Opinion Quarterly19303315Google Scholar
  41. 41.
    McCrae, R. R., Costa, P. T. J. 1996Toward a new generation of personality theories: Theoretical contexts for the five-factor modelWiggins, J. S. eds. The Five-Factor Model of Personality:Theoretical PerspectivesGuilfordNew York5187Google Scholar
  42. 42.
    McDaniel, S. W., Madden, C. S., Verille, P. 1987Do topic differences affect survey non-response?Journal of the Market Research Society295566Google Scholar
  43. 43.
    Moore, D. L., Tarnai, J. 2002Evaluating nonresponse error in mail surveysGroves, R. M.Dillman, D. A.Eltinge, J. L.Little, R. J. A. eds. Survey NonresponseJohn Wiley & SonsNew York197211Google Scholar
  44. 44.
    MountM. K., Barrick, M. A., Scullen, S. M., and Rounds, J. (2004, April 2004). Higher Order Dimensions of Personality Traits and Interests. Paper presented at the Annual meeting of the Society for Industrial and Organizational Psychology, Chicago.Google Scholar
  45. 45.
    National Center for Education Statistics (2002) National Postsecondary Student Aid Study 1999–2000 (NPSAS:2000), CATI Nonresponse Bias Analysis Report (No. 2002–03) Washington, DCGoogle Scholar
  46. 46.
    National Survey of Student Engagement (2003). The College Student Report: 2003 Overview.Google Scholar
  47. 47.
    Pealer, L. N., Weiler, R. M., Pigg, R. M.,Jr, Miller, D., Dorman, S. M. 2001The feasibility of a web-based surveillance system to collect health risk data from college studentsHealth Education & Behavior28547559Google Scholar
  48. 48.
    Porter, S. R., Umbach, P. D. (2002) College Major Choice: An Analysis of Person-Environment Fit. Paper presented at the Association for Institutional Research. TorontoCanadaGoogle Scholar
  49. 49.
    Porter, S. R., Umbach, P. D. 2004What works best? Collecting alumni data with multiple technologiesAIR Professional File90110Google Scholar
  50. 50.
    Porter, S. R., Whitcomb, M. E. 2003aThe impact of contact type on web survey response ratesPublic Opinion Quarterly67579588Google Scholar
  51. 51.
    Porter, S. R., Whitcomb, M. E. 2003bThe impact of lottery incentives on student survey response ratesResearch in Higher Education44389407Google Scholar
  52. 52.
    Porter, S. R., Whitcomb, M. E., Weitzer, W. H. 2004Multiple surveys of students and survey fatiguePorter, S. R. eds. Overcoming Survey Research Problems, New Directions for Institutional Research (Vol. 121)Jossey-BassSan FranciscoGoogle Scholar
  53. 53.
    Roman, R. J., Moskowitz, G. B., Stein, M. I., Eisenberg, R. F. 1995Individual differences in experiment participation: Structureautonomy, and the time of semesterJournal of Personality63113138Google Scholar
  54. 54.
    Sax, L. J., Gilmartin, S. K., Bryant, A. N. 2003Assessing response rates and nonresponse bias in web and paper surveysResearch in Higher Education44409432Google Scholar
  55. 55.
    Sax, L. J., Lindholm, J. A., Astin, A. W., Korn, W. S., Mahoney, K. M. 2001The American Freshman:National Norms for Fall 2001Higher Education Research InstituteUCLA, Los Angeles, CAGoogle Scholar
  56. 56.
    Singer, E., Groves, R. M., Corning, A. D. 1999Differential incentives:Beliefs about practices, perceptions of equity, and effects on survey participationPublic Opinion Quarterly63251260Google Scholar
  57. 57.
    Singer, E., van Hoewyk, J., Maher, M. P. 2000Experiments with incentives in telephone surveysPublic Opinion Quarterly64171188Google Scholar
  58. 58.
    Smart, J. C., Feldman, K. A., Ethington, C. A. 2000Academic Disciplines:Holland’s Theory and the Study of College Students and FacultyVanderbilt University PressNashvilleGoogle Scholar
  59. 59.
    Smith, T. W. 1995Trends in nonresponse ratesInternational Journal of Public Opinion Research7157171Google Scholar
  60. 60.
    Smith, K., Bers, T. 1987Improving alumni survey response rates: An experiment and cost-benefit analysisResearch in Higher Education27218226Google Scholar
  61. 61.
    Steeh, C. G. 1981Trends in nonresponse rates, 1952–1979Public Opinion Quarterly596677Google Scholar
  62. 62.
    Stevens, C. D., Ash, R. A. 2001The conscientiousness of students in subject pools: Implications for “laboratory” researchJournal of Research in Personality359197Google Scholar
  63. 63.
    Stinchcombe, A. L., Jones, C., Sheatsley, P. 1981Nonresponse bias for attitude questionsPublic Opinion Quarterly45359375Google Scholar
  64. 64.
    Tuckel, P., and O’Neill, H. (2002). The vanishing respondent in telephone surveys. Journal of Advertising Research 26–48.Google Scholar
  65. 65.
    Umbach, P. D., Porter, S. R. 2002How do academic departments impact student satisfaction? Understanding the contextual effects of departmentsResearch in Higher Education23209233Google Scholar
  66. 66.
    Van Kenhove, P., Wijnen, K., De Wulf, K. 2002The influence of topic involvement on mail-survey response behaviorPsychology & Marketing19293301Google Scholar
  67. 67.
    Voigt, L. F., Koepsell, T. D., Daling, J. R. 2003Characteristics of telephone survey respondents according to willingness to participateAmerican Journal of Epidemiology1576673Google Scholar
  68. 68.
    WhitcombM. E., and PorterS. R. (2004). Email contacts: A test of complex graphical designs. Social Science Computer Review 22(3): 370–376.Google Scholar
  69. 69.
    Zelenski, J. M., Rusting C. L., Larsen, R. J.(2000). Personality Differences and Consistency in the Time of Experiment Participation. Paper presented at the Poster presented at The Annual Meeting of the Society for Personality and Social Psychology. NashvilleTNGoogle Scholar
  70. 70.
    Zusman, B. J., Duby, P. 1987An evaluation of the use of monetary incentives in postsecondary survey researchJournal of Research and Development in Education207378Google Scholar

Copyright information

© Springer Science+Business Media, Inc. 2005

Authors and Affiliations

  1. 1.Wesleyan UniversityMiddletown
  2. 2.Office of Institutional ResearchWesleyan UniversityMiddletownUSA

Personalised recommendations