International Journal of Public Health

, Volume 56, Issue 4, pp 407–417 | Cite as

Agreement between web-based and paper versions of a socio-demographic questionnaire in the NutriNet-Santé study

  • Anne-Claire Vergnaud
  • Mathilde Touvier
  • Caroline Méjean
  • Emmanuelle Kesse-Guyot
  • Clothilde Pollet
  • Aurélie Malon
  • Katia Castetbon
  • Serge HercbergEmail author
Original Article



Web-based studies nowadays raise a major interest as they can improve all steps involved in observational studies. Our objective was to compare the web-based version of the NutriNet-Santé self-administered socio-demographic and economic questionnaire with the traditional paper version.


Both versions of the questionnaire were sent to 170 volunteers and filled in by 147 of them (either paper first, n = 76, or web-based first, n = 71). Agreement between versions was assessed by intraclass correlations (ICC) and kappas.


Agreement between both versions was high, with ICC and kappas ranging between 0.81–1.00 and 0.76–1.00, respectively, similarly across groups of administration order, age, gender and self-estimated web knowledge in general. The web-based version was the one preferred by 93.7% of the subjects and enabled to avoid 553 missing values (2.00% of the total entries), 24 inconsistent data (0.09%), 8 aberrant data (0.03%), 472 data entry errors (0.85%) and to save 2,800 € (US $4,072) when sent to 170 subjects.


The web-based socio-demographic and economic questionnaire provided information of similar-to-superior quality compared to the traditional paper version, with substantial logistic and cost advantages.


Internet Socio-economical status Epidemiology Web-based questionnaire 



We thank all scientists, dieticians, technicians and assistants who helped to carry out the NutriNet-Santé study and the dedicated and conscientious volunteers who participated in this test study. We especially thank Laudine Brico (data manager) for performing the statistical analysis and Gwenael Monot (computer scientist), who coordinates the computing aspects of the NutriNet-Santé study. We thank Voluntis (a healthcare software company) for developing the NutriNet-Santé web-based interface according to our guidelines. This work was supported by the French Ministry of Health (DGS), the Institut de Veille Sanitaire (InVS), the Institut National de Prévention et d’Education pour la Santé (INPES), the Fondation pour la Recherche Médicale (FRM), the Institut National de la Santé et de la Recherche Médicale (Inserm), the Institut de Recherche en Santé Publique (IRESP), the Institut National de la Rechereche Agronomique (Inra), the Conservatoire National des Arts et Métiers (CNAM) and the Paris 13 University.

Conflict of interest

None of the authors had any conflict of interest.


  1. Bertrais S, Preziosi P, Mennen L, Galan P, Hercberg S, Oppert JM (2004) Sociodemographic and geographic correlates of meeting current recommendations for physical activity in middle-aged French adults: the Supplementation en Vitamines et Mineraux Antioxydants (SUVIMAX) Study. Am J Public Health 94:1560–1566PubMedCrossRefGoogle Scholar
  2. Brigham J, Lessov-Schlaggar CN, Javitz HS, Krasnow RE, McElroy M, Swan GE (2009) Test–retest reliability of web-based retrospective self-report of tobacco exposure and risk. J Med Internet Res 11:35CrossRefGoogle Scholar
  3. Clark AM, DesMeules M, Luo W, Duncan AS, Wielgosz A (2009) Socioeconomic status and cardiovascular disease: risks and implications for care. Nat Rev Cardiol 6:712–722PubMedCrossRefGoogle Scholar
  4. Drewnowski A (2009) Obesity, diets, and social inequalities. Nutr Rev 67(Suppl 1):36–39CrossRefGoogle Scholar
  5. Ekman A, Litton JE (2007) New times, new needs; e-epidemiology. Eur J Epidemiol 22:285–292PubMedCrossRefGoogle Scholar
  6. European Interactive Advertising Association (2005)
  7. Feinstein AR, Cicchetti DV (1990) High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 43:543–549PubMedCrossRefGoogle Scholar
  8. Etude NutriNet-Santé Website (2009)
  9. Graham AL, Papandonatos GD (2008) Reliability of internet-versus telephone-administered questionnaires in a diverse sample of smokers. J Med Internet Res 10:8CrossRefGoogle Scholar
  10. Greene J, Speizer H, Wiitala W (2008) Telephone and web: mixed-mode challenge. Health Serv Res 43:230–248PubMedCrossRefGoogle Scholar
  11. Hammersley R (1994) A digest of memory phenomena for addiction research. Addiction 89:283–293PubMedCrossRefGoogle Scholar
  12. Hercberg S, Preziosi P, Briancon S, Galan P, Triol I, Malvy D, Roussel AM, Favier A (1998) A primary prevention trial using nutritional doses of antioxidant vitamins and minerals in cardiovascular diseases and cancers in a general population: the SU.VI.MAX study—design, methods, and participant characteristics. SUpplementation en VItamines et Mineraux AntioXydants. Control Clin Trials 19:336–351PubMedCrossRefGoogle Scholar
  13. Internet au quotidien: un Français sur quatre; Insee Première No. 1076 (2006)
  14. Joinson A (1999) Social desirability, anonymity, and Internet-based questionnaires. Behav Res Methods Instrum Comput 31:433–438PubMedCrossRefGoogle Scholar
  15. L’audience de l’Internet en France Médiamétrie (2010)
  16. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174PubMedCrossRefGoogle Scholar
  17. McCabe SE, Boyd CJ, Couper MP, Crawford S, D’Arcy H (2002) Mode effects for collecting alcohol and other drug use data: web and U.S. mail. J Stud Alcohol 63:755–761PubMedGoogle Scholar
  18. McLaren L (2007) Socioeconomic status and obesity. Epidemiol Rev 29:29–48PubMedCrossRefGoogle Scholar
  19. Pealer LN, Weiler RM, Pigg RM Jr, Miller D, Dorman SM (2001) The feasibility of a web-based surveillance system to collect health risk behavior data from college students. Health Educ Behav 28:547–559PubMedCrossRefGoogle Scholar
  20. Profil sociodemographique des internautes Ipsos Media (2007)
  21. Rankin KM, Rauscher GH, McCarthy B, Erdal S, Lada P, Il’yasova D, Davis F (2008) Comparing the reliability of responses to telephone-administered versus self-administered Web-based surveys in a case–control study of adult malignant brain cancer. Cancer Epidemiol Biomarkers Prev 17:2639–2646PubMedCrossRefGoogle Scholar
  22. Schatzkin A, Subar AF, Moore S, Park Y, Potischman N, Thompson FE, Leitzmann M, Hollenbeck A, Morrissey KG, Kipnis V (2009) Observational epidemiologic studies of nutrition and cancer: the next generation (with better observation). Cancer Epidemiol Biomarkers Prev 18:1026–1032PubMedCrossRefGoogle Scholar
  23. Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420–428PubMedCrossRefGoogle Scholar
  24. Stanton CA, Papandonatos G, Lloyd-Richardson EE, Niaura R (2007) Consistency of self-reported smoking over a 6-year interval from adolescence to young adulthood. Addiction 102:1831–1839PubMedCrossRefGoogle Scholar
  25. Touvier M, Mejean C, Kesse-Guyot E, Pollet C, Malon A, Castetbon K, Hercberg S (2010) Comparison between web-based and paper versions of a self-administered anthropometric questionnaire. Eur J Epidemiol 25(5):287–296PubMedCrossRefGoogle Scholar
  26. Wang YC, Lee CM, Lew-Ting CY, Hsiao CK, Chen DR, Chen WJ (2005) Survey of substance use among high school students in Taipei: web-based questionnaire versus paper-and-pencil questionnaire. J Adolesc Health 37:289–295PubMedCrossRefGoogle Scholar
  27. Ward E, Jemal A, Cokkinides V, Singh GK, Cardinez C, Ghafoor A, Thun M (2004) Cancer disparities by race/ethnicity and socioeconomic status. CA Cancer J Clin 54:78–93PubMedCrossRefGoogle Scholar

Copyright information

© Swiss School of Public Health 2011

Authors and Affiliations

  • Anne-Claire Vergnaud
    • 1
  • Mathilde Touvier
    • 1
  • Caroline Méjean
    • 1
  • Emmanuelle Kesse-Guyot
    • 1
  • Clothilde Pollet
    • 1
  • Aurélie Malon
    • 1
  • Katia Castetbon
    • 2
  • Serge Hercberg
    • 1
    • 2
    • 3
    Email author
  1. 1.Unité de Recherche en Epidémiologie Nutritionnelle (UREN), UMR U557 Inserm/U1125 Inra/Cnam/Paris 13, CRNH IdFSMBH Paris 13Bobigny CedexFrance
  2. 2.Unité de Surveillance et d’Epidémiologie Nutritionnelle (USEN), Institut de Veille SanitaireUniversité Paris 13Bobigny CedexFrance
  3. 3.Département de Santé PubliqueHôpital AvicenneBobigny CedexFrance

Personalised recommendations