Skip to main content
SpringerLink
Log in

Can the Toronto Extremity Salvage Score Produce Reliable Results When Used Online?

  • Clinical Research
  • Published:
Clinical Orthopaedics and Related Research®

Abstract

Background

Web-based questionnaires have become popular, however, access to the Internet can be biased regarding age, gender, and education, among other factors. Therefore, it is unknown whether this is a reasonable avenue to administer a questionnaire to patients or whether Web-based can be a reliable alternative to paper-based.

Questions/purposes

We determined whether the Internet version of the Toronto Extremity Salvage Score is reproducible compared with the paper-based version and the compliance and completion rates.

Patients and Methods

The study population consisted of 81 adults who had had surgery for a musculoskeletal tumor of the lower extremity more than 12 months earlier. The Toronto Extremity Salvage Score was administered by paper at a baseline interview and then readministered via Internet 7 to 14 days later to those with access.

Results

Sixty of the 81 patients (74%) were able to use the Internet. Increasing age and lower education levels were correlated with a lower likelihood of using the Internet. Questionnaires were done online and on paper by 56 patients but 10 were excluded because of self-reported change in circumstances. The mean TESS was 85.7 (range, 41.1–100; SD, 17.26) for the paper-based questionnaire and 85.2 (range, 42.5–100; SD, 17.47) for the Internet-based questionnaire. The intraclass correlation coefficient was 0.97.

Conclusions

The questionnaire can be transferred successfully to the Internet and can be used reliably instead of a paper-based instrument. Recruitment to use an Internet-based questionnaire is limited only by the percentage of patients able to access and use the Internet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1A–B

Similar content being viewed by others

References

  1. Athale N, Sturley A, Skoczen S, Kavanaugh A, Lenert L. A web-compatible instrument for measuring self-reported disease activity in arthritis. J Rheumatol. 2004;31:223–228.

    PubMed  Google Scholar 

  2. Bälter KA, Bälter O, Fondell E, Lagerros YT. Web-based and mailed questionnaires: a comparison of response rates and compliance. Epidemiology. 2005;16:577–579.

    Article  PubMed  Google Scholar 

  3. Bliven BD, Kaufman SE, Spertus JA. Electronic collection of health-related quality of life data: validity, time benefits, and patient preference. Qual Life Res. 2001;10:15–22.

    Article  PubMed  CAS  Google Scholar 

  4. Buchanan T, Smith JL. Using the Internet for psychological research: personality testing on the World Wide Web. Br J Psychol. 1999;90:125–144.

    Article  PubMed  Google Scholar 

  5. Bush N, Donaldson G, Moinpour C, Haberman M, Milliken D, Markle V, Lauson J. Development, feasibility and compliance of a web-based system for very frequent QOL and symptom home self-assessment after hematopoietic stem cell transplantation. Qual Life Res. 2005;14:77–93.

    Article  PubMed  CAS  Google Scholar 

  6. Bushnell DM, Martin ML, Parasuraman B. Electronic versus paper questionnaires: a further comparison in persons with asthma. J Asthma. 2003;40:751–762.

    Article  PubMed  CAS  Google Scholar 

  7. Cals JW, Boumans D, Lardinois RJ, Gonzales R, Hopstaken RM, Butler CC, Dinant GJ. Public beliefs on antibiotics and respiratory tract infections: an internet-based questionnaire study. Br J Gen Pract. 2007;57:942–947.

    Article  PubMed  Google Scholar 

  8. Cantrell MA, Lupinacci P. Methodological issues in online data collection. J Adv Nurs. 2007;60:544–549.

    Article  PubMed  Google Scholar 

  9. Davis AM, Bell RS, Badley EM, Yoshida K, Williams JI. Evaluating functional outcome in patients with lower extremity sarcoma. Clin Orthop Relat Res. 1999;358:90–100.

    Article  PubMed  Google Scholar 

  10. Davis AM, Sennik S, Griffin AM, Wunder JS, O’Sullivan B, Catton CN, Bell RS. Predictors of functional outcomes following limb salvage surgery for lower-extremity soft tissue sarcoma. J Surg Oncol. 2000;73:206–211.

    Article  PubMed  CAS  Google Scholar 

  11. Davis AM, Wright JG, Williams JI, Bombardier C, Griffin A, Bell RS. Development of a measure of physical function for patients with bone and soft tissue sarcoma. Qual Life Res. 1996;5:508–516.

    Article  PubMed  CAS  Google Scholar 

  12. Davis RN. Web-based administration of a personality questionnaire: comparison with traditional methods. Behav Res Methods Instrum Comput. 1999;31:572–577.

    Article  PubMed  CAS  Google Scholar 

  13. Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, Kwan I. Increasing response rates to postal questionnaires: systematic review. BMJ. 2002;324:1183–1191.

    Article  PubMed  Google Scholar 

  14. Ekman A, Dickman PW, Klint A, Weiderpass E, Litton JE. Feasibility of using Web-based questionnaires in large population-based epidemiological studies. Eur J Epidemiol. 2006;21:103–111.

    Article  PubMed  Google Scholar 

  15. Ekman A, Klint A, Dickman PW, Adami HO, Litton JE. Optimizing the design of Web-based questionnaires: experience from a population-based study among 50,000 women. Eur J Epidemiol. 2007;22:293–300.

    Article  PubMed  Google Scholar 

  16. Enneking W. Modification of the system for functional evaluation in the surgical management of musculoskeletal tumours. In: Enneking W, ed. Limb Salvage in Musculoskeletal Oncology. New York, NY: Churchill Livingstone; 1987;626–639.

    Google Scholar 

  17. Enneking WF, Dunham W, Gebhardt MC, Malawar M, Pritchard DJ. A system for the functional evaluation of reconstructive procedures after surgical treatment of tumors of the musculoskeletal system. Clin Orthop Relat Res. 1993;286:241-246.

    PubMed  Google Scholar 

  18. Gerrand CH, Wunder JS, Kandel RA, O’Sullivan B, Catton CN, Bell RS, Griffin AM, Davis AM. The influence of anatomic location on functional outcome in lower-extremity soft-tissue sarcoma. Ann Surg Oncol. 2004;11:476–482.

    Article  PubMed  CAS  Google Scholar 

  19. Gilmour JA, Scott SD, Huntington N. Nurses and Internet health information: a questionnaire survey. J Adv Nurs. 2008;61:19–28.

    PubMed  Google Scholar 

  20. Hanscom B, Lurie JD, Homa K, Weinstein JN. Computerized questionnaires and the quality of survey data. Spine (Phila Pa 1976). 2002;27:1797–1801.

    Google Scholar 

  21. Harewood GC, Yacavone RF, Locke GR 3rd, Wiersema MJ. Prospective comparison of endoscopy patient satisfaction surveys: e-mail versus standard mail versus telephone. Am J Gastroenterol. 2001;96:3312–3317.

    Article  PubMed  CAS  Google Scholar 

  22. Joinson A. Social desirability, anonymity, and Internet-based questionnaires. Behav Res Methods Instrum Comput. 1999;31:433–438.

    Article  PubMed  CAS  Google Scholar 

  23. Jones JB, Snyder CF, Wu AW. Issues in the design of Internet-based systems for collecting patient-reported outcomes. Qual Life Res. 2007;16:1407–1417.

    Article  PubMed  Google Scholar 

  24. Jones R, Pitt N. Health surveys in the workplace: comparison of postal, email and World Wide Web methods. Occup Med (Lond). 1999;49:556–558.

    Article  CAS  Google Scholar 

  25. Khalily C, Behnke S, Seligson D. Treatment of closed tibia shaft fractures: a survey from the 1997 Orthopaedic Trauma Association and Osteosynthesis International-Gerhard Kuntscher Kreis meeting. J Orthop Trauma. 2000;14:577–581.

    Article  PubMed  CAS  Google Scholar 

  26. Kim HL, Gerber GS, Patel RV, Hollowell CM, Bales GT. Practice patterns in the treatment of female urinary incontinence: a postal and internet survey. Urology. 2001;57:45–48.

    Article  PubMed  CAS  Google Scholar 

  27. Kongsved SM, Basnov M, Holm-Christensen K, Hjollund NH. Response rate and completeness of questionnaires: a randomized study of internet versus paper-and-pencil versions. J Med Internet Res. 2007;9:e25.

    Article  PubMed  Google Scholar 

  28. Kumle M, Weiderpass E, Braaten T, Persson I, Adami H, Lund E. Use of oral contraceptives and breast cancer risk: The Norwegian-Swedish Women’s Lifestyle and Health Cohort Study. Cancer Epidemiol Biomarkers Prev. 2002;11:1375–1381.

    PubMed  CAS  Google Scholar 

  29. Lampert MH, Gerber LH, Glatstein E, Rosenberg SA, Danoff JV. Soft tissue sarcoma: functional outcome after wide local excision and radiation therapy. Arch Phys Med Rehabil. 1984;65:477–480.

    PubMed  CAS  Google Scholar 

  30. Leece P, Bhandari M, Sprague S, Swiontkowski MF, Schemitsch EH, Tornetta P, Devereaux PJ, Guyatt GH. Internet versus mailed questionnaires: a controlled comparison (2). J Med Internet Res. 2004;6:e39.

    Article  PubMed  Google Scholar 

  31. Matarasso A, Elkwood A, Rankin M, Elkowitz M. National plastic surgery survey: face lift techniques and complications. Plastic Reconstr Surg. 2000;106:1185–1195; discussion 1196.

    Google Scholar 

  32. Meropol NJ, Egleston BL, Buzaglo JS, Benson AB 3rd, Cegala DJ, Diefenbach MA, Fleisher L, Miller SM, Sulmasy DP, Weinfurt KP; CONNECT Study Research Group. Cancer patients’ preferences for quality and length of life. Cancer. 2008;113:3459–3466.

    Article  PubMed  Google Scholar 

  33. O’Leary TE, Diller L, Recklitis CJ. The effects of response bias on self-reported quality of life among childhood cancer survivors. Qual Life Res. 2007;16:1211–1220.

    Article  PubMed  Google Scholar 

  34. Ritter P, Lorig K, Laurent D, Matthews K. Internet versus mailed questionnaires: a randomized comparison. J Med Internet Res. 2004;6:e29.

    Article  PubMed  Google Scholar 

  35. Robinson MH, Spruce L, Eeles R, Fryatt I, Harmer CL, Thomas JM, Westbury G. Limb function following conservation treatment of adult soft tissue sarcoma. Eur J Cancer. 1991;27:1567–1574.

    Article  PubMed  CAS  Google Scholar 

  36. Saleh KJ, Radosevich DM, Kassim RA, Moussa M, Dykes D, Bottolfson H, Gioe TJ, Robinson H. Comparison of commonly used orthopaedic outcome measures using palm-top computers and paper surveys. J Orthop Res. 2002;20:1146–1151.

    Article  PubMed  Google Scholar 

  37. Schreiber D, Bell RS, Wunder JS, O’Sullivan B, Turcotte R, Masri BA, Davis AM. Evaluating function and health related quality of life in patients treated for extremity soft tissue sarcoma. Qual Life Res. 2006;15:1439–1446.

    Article  PubMed  Google Scholar 

  38. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psych Bull. 1979;86:420–428.

    Article  CAS  Google Scholar 

  39. Smith B, Smith TC, Gray GC, Ryan MA; Millennium Cohort Study Team. When epidemiology meets the Internet: Web-based surveys in the Millennium Cohort Study. Am J Epidemiol. 2007;166:1345–1354.

    Article  PubMed  Google Scholar 

  40. Sugarbaker PH, Barofsky I, Rosenberg SA, Gianola FJ. Quality of life assessment of patients in extremity sarcoma clinical trials. Surgery. 1982;91:17–23.

    PubMed  CAS  Google Scholar 

  41. Surveymonkey.com. Available at: https://www.surveymonkey.com/Default.aspx. Accessed July 29, 2010.

  42. World Health Organization. International Classification of Impairments, Disabilities and Handicaps. Geneva, Switzerland: World Health Organization; 1990.

  43. Wilson AS, Kitas GD, Carruthers DM, Reay C, Skan J, Harris S, Treharne GJ, Young SP, Bacon PA. Computerized information-gathering in specialist rheumatology clinics: an initial evaluation of an electronic version of the Short Form 36. Rheumatology (Oxford). 2002;41:268–273.

    Article  CAS  Google Scholar 

Download references

Acknowledgment

We thank Thomas Sullivan, Lecturer in the Data Management and Analysis Centre, Discipline of Public Health, University of Adelaide, for assistance with the statistical assessments.

Author information

Authors and Affiliations

  1. Department of Orthopaedics and Trauma, Royal Adelaide Hospital, North Terrace, Adelaide, 5000, Australia

    Mark Clayer MD

  2. Division of Health Care and Outcomes Research and Arthritis and Community Research and Evaluation Unit, Toronto Western Research Institute, University of Toronto, Toronto, Canada

    Aileen Davis PhD

Authors
  1. Mark Clayer MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aileen Davis PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mark Clayer MD.

Additional information

Each author certifies that he or she has no commercial associations (eg, consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article.

Each author certifies that the human protocol for this investigation was approved by an Institutional Ethics Research Committee, that all investigations were conducted in conformity with ethical principles of research, and that informed consent for participation in the study was obtained.

This work was performed at the Royal Adelaide Hospital.

About this article

Cite this article

Clayer, M., Davis, A. Can the Toronto Extremity Salvage Score Produce Reliable Results When Used Online?. Clin Orthop Relat Res 469, 1750–1756 (2011). https://doi.org/10.1007/s11999-010-1715-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11999-010-1715-8

Keywords

Search

Navigation