Skip to main content

Who should supervise students during self-report interviews? A controlled experiment on response behavior in online questionnaires

Journal of Experimental Criminology volume 8, pages 17–28 (2012)Cite this article

Abstract

Objectives

This study tests whether juveniles’ responses on sensitive topics such as self-reported delinquency, victimization, and substance use are comparable when teachers versus external persons supervise students while filling out online questionnaires.

Methods

Eighty classes with 1,197 students (9th grade) in eastern Switzerland were randomly assigned (at the class level) to supervision either by their teacher or by an external person (i.e., researcher). Students filled out online questionnaires about self-reported delinquency, victimization, and substance use in the classroom while being supervised either by their teacher or by an external person. Prevalence rates were compared using Chi-square tests.

Results

Only three out of 57 comparisons show significantly different outcomes. Whenever differences are found, and contrary to our expectation, sensitive experiences are more often admitted when students are supervised by their teacher. Effect sizes do not exceed 0.6, and are thus all below the limit of a small effect.

Conclusions

Using online questionnaires with teachers as supervisors may not affect validity while making surveys less expensive and intrusive.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Aquilino, W. S. (1994). Interview mode effects in surveys of drug and alcohol use: A field experiment. Public Opinion Quarterly, 58(2), 210–240.

    Article  Google Scholar 

  • Beebe, T. J., Harrison, P. A., McRae, J. A., Anderson, R. E., & Fulkerson, J. A. (1998). An evaluation of computer-assisted self-interviews in a school setting. Public Opinion Quarterly, 62(4), 623–632.

    Article  Google Scholar 

  • Bennett, S., Parpia, T., Hayes, R., & Cousens, S. (2002). Methods for the analysis of incidence rates in cluster randomized trials. International Journal of Epidemiology, 31(4), 839–846.

    Article  Google Scholar 

  • Bland, J. M. (2004). Cluster randomised trials in the medical literature: Two bibliometric surveys. BMC Medical Research Methodology, 4(1), 21–26.

    Article  Google Scholar 

  • Brener, N. D., Billy, J. O. G., & Grady, W. R. (2003). Assessment of factors affecting the validity of self-reported health-risk behavior among adolescents: Evidence from the scientific literature. Journal of Adolescent Health, 33(6), 436–457.

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Erlbaum.

    Google Scholar 

  • Gans, J. E., & Brindis, C. D. (1995). Choice of research setting in understanding adolescent health problems. Journal of Adolescent Health, 17(5), 306–313.

    Article  Google Scholar 

  • Gribble, J. N., Miller, H. G., Rogers, S. M., & Turner, C. F. (1999). Interview mode and measurement of sexual behaviors: Methodological issues. Journal of Sex Research, 36(1), 16–17.

    Article  Google Scholar 

  • Hallfors, D., Khatapoush, S., Kadushin, C., Watson, K., & Saxe, L. (2000). A comparison of paper vs computer-assisted self-interview for school alcohol, tobacco, and other drug surveys. Evaluation and Program Planning, 23(2), 149–155.

    Article  Google Scholar 

  • Hindelang, M. J., Hirschi, T., & Weis, J. G. (1979). Correlates of delinquency: The illusion of discrepancy between self-report and official measures. American Sociological Review, 44(6), 995–1014.

    Article  Google Scholar 

  • Kann, L., Brener, N. D., Warren, C. W., Collins, J. L., & Giovino, G. A. (2002). An assessment of the effect of data collection setting on the prevalence of health risk behaviors among adolescents. Journal of Adolescent Health, 31(4), 327–335.

    Article  Google Scholar 

  • Kivivuori, J. & Salmi, V. (2011). Supervision mode effects in school-based delinquency and victimization survey: Preliminary test (Research Brief No. 20/2011).

  • Lucia, S., Herrmann, L., & Killias, M. (2007). How important are interview methods and questionnaire designs in research on self-reported juvenile delinquency? An experimental comparison of Internet vs paper-and-pencil questionnaires and different definitions of the reference period. Journal of Experimental Criminology, 3, 39–64.

    Article  Google Scholar 

  • Thornberry, T. P., & Krohn, M. D. (2000). The self-report method for measuring delinquency and crime. In D. Duffee (Ed.), Measurement and analysis of crime and justice (Criminal Justice 2000, Vol. 4, pp. 33–83). Washington, DC: National Institute of Justice, Office of Justice Programs, US Department of Justice.

    Google Scholar 

  • Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60(2), 275–304.

    Article  Google Scholar 

  • Trapl, E. S. (2007). Understanding adolescent survey responses: Impact of mode and other characteristics on data outcomes and quality. Cleveland: Case Western Reserve University.

    Google Scholar 

  • Turner, C. F., Ku, L., Rogers, S. M., Lindberg, L. D., Pleck, J. H., & Sonenstein, F. L. (1998). Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280, 867–873.

    Article  Google Scholar 

  • Vereecken, C. A., & Maes, L. (2006). Comparison of a computer-administered and paper-and-pencil-administered questionnaire on health and lifestyle behaviors. Journal of Adolescent Health, 38(4), 426–432.

    Article  Google Scholar 

  • Weisband, S., & Kiesler, S. (1996). Self-disclosure on computer forms: Meta-analysis and implications. In R. Bilger, S. Guest, & M. J. Tauber (Eds.), Proceedings of the SIGCHI conference on human factors in computing systems: Common ground (pp. 3–10). Vancouver: ACM.

    Chapter  Google Scholar 

  • Wright, D. L., Aquilino, W. S., & Supple, A. J. (1998). A comparison of computer-assisted and paper-and-pencil self-administered questionnaires in a survey on smoking, alcohol, and drug use. Public Opinion Quarterly, 62(3), 331–353.

    Article  Google Scholar 

Download references

Acknowledgements

This survey was mandated by the Government of the Canton of St. Gallen, Switzerland (Department of Security & Justice and Department of Education) and carried out by the Institute of Criminology, University of Zurich, Switzerland.

Author information

Authors and Affiliations

  1. Institute of Criminology, University of Zurich, Raemistrasse 74/39, 8001, Zurich, Switzerland

    Simone Walser & Martin Killias

Authors
  1. Simone Walser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Killias
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simone Walser.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Walser, S., Killias, M. Who should supervise students during self-report interviews? A controlled experiment on response behavior in online questionnaires. J Exp Criminol 8, 17–28 (2012). https://doi.org/10.1007/s11292-011-9129-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11292-011-9129-5

Keywords

  • Internet surveys
  • Interview methods
  • Research methods
  • School-based surveys
  • Self-reported juvenile delinquency