Quality of Life Research

, Volume 19, Issue 4, pp 499–508 | Cite as

Does mode of administration matter? Comparison of online and face-to-face administration of a time trade-off task

  • Richard NormanEmail author
  • Madeleine T. King
  • Dushyant Clarke
  • Rosalie Viney
  • Paula Cronin
  • Deborah Street



Developments in electronic data collection methods have allowed researchers to generate larger datasets at lower costs, but relatively little is known about the comparative performance of the new methods. This paper considers the comparability of two modes of administration (face-to-face and remote electronic) for the time trade-off.


Data were collected from a convenience sample of adults (n = 135) randomised to either a face-to-face time trade-off or a remote electronic tool. Patterns of responses were considered. For each sample, standard regression analysis was undertaken to generate a valuation set, which were then contrasted.


The pattern of responses differed by mode of administration, with the electronic tool yielding larger standard deviations and higher proportions of responses at −1, 0 and 1. The impact of this on the regression was difficult to disentangle from the high variability around individual scores of states, which is a common feature of responses to time trade-off tasks. Under the scoring algorithms generated by mode of administration, the difference between scores exceeded 0.1 for 100 of the 243 EQ-5D health states.


This comparison demonstrates that variability arising from mode of administration needs to be considered in developing health state valuations. While electronic administration has considerable cost advantages, particular attention to the design of the task is required. This has wider implications, as all modes of administration may have mode-specific impacts on the distribution of valuation responses.


Mode of administration Time trade-off Preference elicitation EQ-5D 


  1. 1.
    Hawthorne, G., Richardson, J., & Day, N. A. (2001). A comparison of the assessment of quality of life (AQoL) with four other generic utility instruments. Annals of Medicine, 33(5), 358–370.CrossRefPubMedGoogle Scholar
  2. 2.
    Bansback, N., Davis, S., & Brazier, J. (2007). Using contrast sensitivity to estimate the cost-effectiveness of verteporfin in patients with predominantly classic age-related macular degeneration. Eye, 21(12), 1455–1463.CrossRefPubMedGoogle Scholar
  3. 3.
    Brazier, J., Roberts, J., & Deverill, M. (2002). The estimation of a preference-based measure of health from the SF-36. Journal of Health Economics, 21(2), 271–292.CrossRefPubMedGoogle Scholar
  4. 4.
    Dolan, P. (1997). Modelling valuations for EuroQol health states. Medical Care, 35(11), 1095–1108.CrossRefPubMedGoogle Scholar
  5. 5.
    Hawthorne, G., Richardson, J., & Osborne, R. (1999). The assessment of quality of life (AQoL) instrument: A psychometric measure of health-related quality of life. Quality of Life Research, 8(3), 209–224.CrossRefPubMedGoogle Scholar
  6. 6.
    Horsman, J., Furlong, W., Feeny, D., et al. (2003). The health utilities index (hui): Concepts, measurement properties and applications. Health and Quality of Life Outcomes, 1(54).Google Scholar
  7. 7.
    Drummond, M., O’Brien, B. J., Stoddart, G. L., et al. (2004). Methods for the economic evaluation of health care programmes. Oxford: Oxford Medical Publications.Google Scholar
  8. 8.
    Damschroder, L. J., Baron, J., Hershey, J. C., et al. (2004). The validity of person tradeoff measurements: Randomized trial of computer elicitation versus face-to-face interview. Medical Decision Making, 24(2), 170–180.CrossRefPubMedGoogle Scholar
  9. 9.
    Dolan, P. (1996). Modelling valuations for health states: The effect of duration. Health Policy, 38(3), 189–203.CrossRefPubMedGoogle Scholar
  10. 10.
    Tsuchiya, A., Ikeda, S., Ikegami, N., et al. (2002). Estimating an EQ-5D population value set: The case of Japan. Health Economics, 11(4), 341–353.CrossRefPubMedGoogle Scholar
  11. 11.
    Bowling, A. (2005). Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health (Oxford), 27(3), 281–291.CrossRefGoogle Scholar
  12. 12.
    Gwaltney, C. J., Shields, A. L., & Shiffman, S. (2008). Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: A meta-analytic review. Value in Health, 11(2), 322–333.CrossRefPubMedGoogle Scholar
  13. 13.
    Dolan, P., Gudex, C., Kind, P., et al. (1996). The time trade-off method: Results from a general population study. Health Economics, 5(2), 141–154.CrossRefPubMedGoogle Scholar
  14. 14.
    Hochberg, Y. (1988). A sharper bonferroni procedure for multiple tests of significance. Biometrika, 75(4), 800–802.CrossRefGoogle Scholar
  15. 15.
    Badia, X., Roset, M., Herdman, M., et al. (2001). A comparison of United Kingdom and Spanish general population time trade-off values for EQ-5D health states. Medical Decision Making, 21(1), 7–16.CrossRefPubMedGoogle Scholar
  16. 16.
    Chow, G. C. (1960). Tests of equality between sets of coefficients in two linear regressions. Econometrica, 28(3), 591–605.CrossRefGoogle Scholar
  17. 17.
    Pickard, A. S., Neary, M. P., & Cella, D. (2007). Estimation of minimally important differences in EQ-5D utility and VAS scores in cancer. Health and Quality of Life Outcomes, 5(70).Google Scholar
  18. 18.
    Roberts, J., & Dolan, P. (2004). To what extent do people prefer health states with higher values? A note on evidence from the EQ-5D valuation set. Health Economics, 13(7), 733–737.CrossRefPubMedGoogle Scholar
  19. 19.
    Johnson, F. R., Ozdemir, S., & Hauber, A. B. (2007). Motivating out-of-pocket treatment costs with cheap talk. iHEA 2007 6th World Congress. Copenhagen.Google Scholar
  20. 20.
    Smith, D. M., Schwarz, N., Roberts, T. R., et al. (2006). Why are you calling me? How study introductions change response patterns. Quality of Life Research, 15(4), 621–630.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  • Richard Norman
    • 1
    Email author
  • Madeleine T. King
    • 2
  • Dushyant Clarke
    • 3
    • 4
  • Rosalie Viney
    • 3
  • Paula Cronin
    • 3
  • Deborah Street
    • 5
  1. 1.Centre for Health Economics Research and Evaluation (CHERE)University of Technology SydneyBroadwayAustralia
  2. 2.Psycho-oncology Co-operative Research Group (PoCoG)University of SydneySydneyAustralia
  3. 3.CHERE, University of Technology SydneySydneyAustralia
  4. 4.University of YorkHeslingtonUK
  5. 5.Department of Mathematical Sciences, Faculty of ScienceUniversity of Technology SydneySydneyAustralia

Personalised recommendations