Skip to main content
SpringerLink
Log in

Optimizing the design of web-based questionnaires – experience from a population-based study among 50,000 women

  • Published:
European Journal of Epidemiology Aims and scope Submit manuscript

Abstract

Background

Web-questionnaires are an important tool for future epidemiological research because these allow for rapid and cost-efficient assembly of self-reported information on risk factors and health outcomes. However, to achieve high response rates it is essential to accommodate factors that prevent drop out and so insure validity of future studies. We aim to study how socio-demographic variables as well as design issues such as the ordering and level of difficulty (Easy-to-hard vs. Hard-to-easy) of questions in a web-questionnaire affects the probability of drop out and non-response.

Method

In 2003 we invited 47,859 women participating in an ongoing prospective study to a follow-up using a web-based mode. Two versions of the questionnaire existed, varying in level of difficulty (Easy-to-hard vs. Hard-to-easy). We report drop out (proportion non-completers) between groups defined by level of difficulty and estimated adjusted risk differences.

Results

The drop out differs significantly depending on the order of the questions in the web-questionnaire. The socio-demographic pattern among lurkers (participants that enter, start responding to, but do not complete a web-questionnaire) differs from that among completers of web-questionnaires.

Conclusions

An additional 6% units of completers – persons initiating and completing the questionnaire – can be obtained by considering the ordering of questions. A group uniquely identified in web-surveys,␣as lurkers are potentially easier to persuade to complete an already started web-questionnaire compared to a non-responder. Lurkers thus constitute a unique opportunity of decreasing the drop out rate and therefore merit future research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

BMI:

body mass index

IT:

information technology

OC:

oral contraceptives

OS:

operative system

RA:

Rheumatoid Arthritis

URL:

Uniform Resource Locator

References

  1. Schleyer TK, Forrest JL. (2000) Methods for the design and administration of web-based surveys. J Am Med Inform Assoc 7(4):416–425

    PubMed  CAS  Google Scholar 

  2. Wyatt JC. (2000) When to use web-based surveys. J Am Med Inform Assoc 7(4):426–429

    PubMed  CAS  Google Scholar 

  3. Ekman A, Dickman PW, Klint A, Weiderpass E, Litton JE. (2006) Feasibility of using web-based questionnaires in large population-based epidemiological studies. Eur J Epidemiol 21(2):103–111

    Article  PubMed  Google Scholar 

  4. Dillman D, Tortora RL, Conradt J, Bowker D. Influence of plain vs fancy design on response rates for web surveys. In: Joint Statistical Meetings. Alexandria, Egypt, 1998

  5. Heerwegh D. (2005) Effect of personal salutations in e-mail invitations to participate in a web survey. Public Opin Q 69(4):588–598

    Article  Google Scholar 

  6. Smyth JD, Dillman DA, Christian LM, Stern MJ. How visual grouping influence answers to internet surveys. In: Annual Meeting of the American Association for Public Opinion Research. Phoenix, 2004

  7. Deutskens E, Ruyter dK, Wetzels M, Oosterveld P. (2004) Response rate and response quality of internet-based surveys: An experimental study. Mark Lett 15(1):21–36

    Article  Google Scholar 

  8. Manfreda KL, Vehovar V. Survey design features influencing response rates in web surveys. In: The International Conference on Improving Surveys, University of Copenhagen, Denmark; 2002, 25–28 August

  9. Cook C, Heath F, Thompson RL. (2000) A meta-analysis of response rates in web- or internet-based surveys. Educ Psychol Meas 60(6):821–836

    Article  Google Scholar 

  10. Eaker S, Adami HO, Granath F, Wilander E, Sparen P. (2004) A large population-based randomized controlled trial to increase attendance at screening for cervical cancer. Cancer Epidem Biomar 13(3):346–354

    Google Scholar 

  11. Kumle M, Weiderpass E, Braaten T, Persson I, Adami HO, Lund E. (2002) Use of oral contraceptives and breast cancer risk: The Norwegian-Swedish Women’s Lifestyle and Health Cohort Study. Cancer Epidemiol Biomarkers Prev 11(11):1375–1381

    PubMed  CAS  Google Scholar 

  12. Veierod MB, Weiderpass E, Thorn M, et al. (2003) A prospective study of pigmentation, sun exposure, and risk of cutaneous malignant melanoma in women. J Natl Cancer Inst 95(20):1530–1538

    PubMed  Google Scholar 

  13. The Data Inspection Board. Personal Data Act (1998:204)

  14. Bosnjak M, Tuten TL. Classifying response behaviors in web-based surveys. Journal of Computer-Mediated Communication 2001; 6

  15. Wacholder S. (1986) Binomial regression in GLIM: Estimating risk ratios and risk differences. Am J Epidemiol 123(1):174–184

    PubMed  CAS  Google Scholar 

  16. Connelly NA, Brown TL, Decker DJ. (2003) Factors affecting response rates to natural resource – focused mail surveys: Empirical evidence of declining rates over time. Soc Natur Resour 16(6):541–549

    Article  Google Scholar 

  17. Kalton G. (2000) Development in survey research in the past 25 years. Surv Methodol 26(1):3–10

    Google Scholar 

  18. Crawford SD, Couper MP, Lamias MJ. (2001) Web surveys: Perceptions of burden. Soc Sci Comput Rev 19(2):146–162

    Article  Google Scholar 

  19. Edwards P, Roberts I, Clarke M, et al. (2002) Increasing response rates to postal questionnaires: systematic review. Brit Med J 324(7347):1183

    Article  PubMed  Google Scholar 

  20. Porter SP. Raising response rates: What works? New Directions Institutional Res 2004; 2004(121): 5–21

  21. Solomon DJ. (2001) Conducting web-based surveys. Practical Assessment, Res Evaluation 7(19)

  22. Bosnjak M, Tuten TL. (2003) Prepaid and Promised Incentives in Web Surveys. An experiment: Sage Publications

Download references

Acknowledgement

The Swedish Research Council, AFA, The Swedish Cancer Society, the US army and the research funds of Karolinska Institutet funded the research. We express our gratitude for the help from Netsurvey AB with constructing the web-based questionnaires, hosting the web-servers and their expertise in corporate web-based studies.

Author information

Authors and Affiliations

  1. Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Nobels väg 12A, SE-171 77, Stockholm, Sweden

    Alexandra Ekman, Åsa Klint, Paul W. Dickman, Hans-Olov Adami & Jan-Eric Litton

Authors
  1. Alexandra Ekman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Åsa Klint
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul W. Dickman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hans-Olov Adami
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jan-Eric Litton
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexandra Ekman.

Appendix

Appendix

Questions asked on the pages of the web-questionnaire mentioned in Table 4

Appendix A.

Page 1

3 questions – regarding use of medical treatment and mammography

Page 2

5 sub-questions – regarding mammography; at what medical institution, what year, how many?

Page 3

12 questions – Regarding sleep – 5 answer alternatives

Page 16

1 question – Have you ever had a period lasting longer than a month during which you felt uneasy or worried mote of the time?

Page 19

20 questions – regarding attitudes and feelings

Page 24

4 sub-questions – regarding year, time and institution at which a Rheumatoid Arthritis diagnose was set

Page 32

12 sub-questions – regarding weight loss and gain

Page 42

1 question + additional text regarding the use of Hormone Replacement Therapy

Page 49

9 questions (grid) – the duration of time (out of 24 hours) for a set of 9 different activity levels had to be filed out

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ekman, A., Klint, Å., Dickman, P.W. et al. Optimizing the design of web-based questionnaires – experience from a population-based study among 50,000 women. Eur J Epidemiol 22, 293–300 (2007). https://doi.org/10.1007/s10654-006-9091-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10654-006-9091-0

Keywords

Search

Navigation