Advertisement

It is time for an organised, scientific approach to the application of patient-reported outcome measures in clinical studies and trials

  • Stephen P. McKenna
Editorial
  • 119 Downloads

Abstract

Too often, clinicians and researchers assume that patient-reported outcome measures (PROMs) are simply designed, can be changed to suit specific circumstances and should be freely available for use without seeking permission. If a version is not easily available, they believe they can produce a new translation without relevant experience or expertise. Copyright issues are inconvenient and commonly ignored, despite some journals requesting confirmation from authors that permission had been granted for use of copyrighted materials. When informed that authors of an article did not have such permission, the response from an editor was that nothing could be done as he had to rely on the word of the authors.

In fact, high quality PROMs are carefully designed, developed for new cultures by means of proven standardised methodologies, carefully re-validated in the new culture and then administered by a responsible organisation to ensure that they are used appropriately in studies. If such care has not been taken, questions should be raised about the quality of the measure and the information it generates.

Several problems have arisen with the Ankylosing Spondylitis Quality of Life Questionnaire (ASQoL). This is probably because it is recognised as the most appropriate measure of quality of life for people with Ankylosing Spondylitis and has been widely used in international clinical trials and studies. Fifty-seven official language adaptations of the questionnaire have been produced. Unfortunately, as a result of unauthorised adaptation and use, questions are raised about the validity of several publications that have used the ASQoL.

Keywords

Adaptations Ankylosing Spondylitis Quality of Life Questionnaire (ASQoL) Copyright Patient-reported outcomes 

Patient-reported outcome measure (PROM) development

All PROMs should be based on a coherent construct theory that dictates the nature of the items that should be selected for the measure. Such a theory is not simply one that states that a disease causes certain symptoms and that these should be measured. The model underlying the ASQoL is that the impact of AS on an individual’s life is the extent to which it prevents that person from meeting his or her fundamental human needs. Items for the measure were generated entirely from patients, as they are the experts at living with the disease. The original UK ASQoL [1] took items directly from the interview transcripts. This led to the inclusion in the questionnaire of several colloquial English phrases. Item reduction was governed by a strict mathematical model (Rasch measurement theory) that ensures that the final measure is unidimensional and that each item represents a distinct but different value of QoL on an interval scale [2]. Consequently, respondents can be given accurate scores that indicate their precise level of QoL.

Adapting PROMs for new cultures

All need-based measures of QoL are carefully adapted to ensure that the new language version works well in the target culture [3]. Back translation is avoided as it has been shown to produce poor adaptations [4]. In the case of the ASQoL and all need-based measures, a dual-panel methodology is employed [5]. The first (bilingual) panel provides potential translations for items and instructions. The second panel (consisting of people of low educational attainment) is an essential part of the process as it ensures that simple language is used in the final translation [6]. In a naturally blinded study, this methodology has been shown to produce translations that are more acceptable to patients [7].

When translating patient-based questionnaires such as the ASQoL, it is not enough to consider the English wording of items. Their actual meaning requires an understanding of the way in which they were expressed by the original interviewees. Additional information is provided by determining their psychometric value through the application of statistical modelling. None of this information is published for the ASQoL, preventing meaningful translations being produced, without the involvement of the instrument developers. Indeed, it could be considered that the UK version is only one of many different language versions of the questionnaire. The ASQoL contains several English colloquial phrases that are difficult to adapt without changing the wording of the item.

A Spanish adaptation of the ASQoL was reported by Ariza-Ariza [8]. This was produced without any reference to the authors of the original questionnaire or seeking their permission or help. The published article did not report on the translation method employed, so it is not possible to check how this was done. This unauthorised version differs considerably from the official Spanish ASQoL. In effect, there are now two very different ASQoL versions being used in Spain. What is particularly unhelpful is that they have the same name. Clinicians and researchers in Spain are unlikely to be aware of this problem. However, results generated from the two questionnaires will differ, meaning that the results from different studies cannot be compared. Furthermore, users of the unauthorised version will not have access to experts in use of the ASQoL or its manual, which contains background information on the questionnaire and guidelines on administration, scoring, analysis and interpretation of results.

The fact that an unauthorised adaptation for Spain exists does not imply that it can be used by others, as the authors had no authority to produce an adaptation and cannot give permission for use by others of any version of the ASQoL. An unfortunate consequence is that researchers and clinicians using the unauthorised version are also breaching copyright.

The problem in Spain was compounded by the Sociedad Española de Reumatología Fundación FER publishing the unauthorised version of the Spanish ASQoL on their website. It is no surprise that Spanish researchers have been led to believe that this is an official version of the questionnaire, freely available for use. Organisations’ misguided desire to publish existing questionnaires for the convenience of clinicians has led instrument developers to stop publishing questionnaires in journals. This prevents interested readers from judging the potential value of the PROMs as outcome measures for their proposed studies. The Society has been requested to remove the questionnaire from their website but did not respond to this request.

Where different versions of a measure exist leading to the publication of conflicting results, the quality of the scale is drawn into question. An extreme example is that five different versions of the Nottingham Health Profile have been used in the Netherlands.

Both unauthorised use and the development of unofficial adaptations of a questionnaire represent breaches of copyright. Where such actions occur, advantage is taken of other people’s work for personal benefit. Frequently, the original development work is unreferenced. The primary purpose of copyright law is not to protect the interests of the authors, but rather to promote the progress of science. Copyright also gives moral rights to be identified as the author of works, and to object to their ‘distortion and mutilation’ [9].

As test developers, we take the responsibility to maintain quality very seriously. We provide a service to individuals wishing to use a questionnaire, provide accurate versions of the available language versions, advise on how to use and score the measure and maintain a file of current and completed studies that use the outcome measure. Whenever possible, we work with researchers around the world to develop high quality adaptations, which we generally subsidise or cover the cost of in full. Use of the measures is free from licence fee for clinicians and researchers conducting non-commercial studies. Income accruing from commercial studies is used to cover the costs of administration, scientific development of the instrument, production of new language versions and publication of scientific articles. A minimal administration fee for non-commercial studies is charged, as experience shows that anything given free to users is not considered to be of value.

Misuse of the ASQoL

A review of published articles using the ASQoL cited on PubMed for 2018 alone identified ten studies describing unauthorised use of the measure. Eight of these studies employed unauthorised adaptations of the questionnaire. Consequently, the quality of the results presented in these publications must be highly questionable. It is interesting to enquire why reviewers did not question the use of these unauthorised versions, as the development papers could not have been properly referenced. Several journals require authors to confirm that they have permission to use the questionnaires.

A second PubMed review, of all articles using the ASQoL since its development, identified 125 articles, of which most were unauthorised. This highlights how difficult it is to maintain the quality of the questionnaire as a scientific instrument without the support of editors and reviewers. These articles include unauthorised adaptations of the ASQoL for Spain [8], China [10], India [11] and Tunisia [12]. In addition, a second French adaptation was produced [13] even though the authors referenced the official French adaptation without providing a justification for repeating work unnecessarily!

Of the published ASQoL articles identified, 29 employed unauthorised language versions. These were for the following countries; Egypt [14], France [13], Iran [15, 16]), India [17, 18, 19], Tunisia [12, 20, 21, 22], China [10, 23, 24, 25, 26, 27, 28] and Spain [8, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38]. This last study is included as it used data from the Spanish Register of Biological Therapy in Spondyloarthritides. This register is related to the organisation that published the unauthorised version of the Spanish ASQoL. An unfortunate consequence of this is that the validity of ASQoL data generated by the registry must be in doubt. While the greatest misuse of the ASQoL has arisen in Spain, this is primarily the consequence of the original production of an unauthorised Spanish adaptation.

Conclusions and recommendations

The careful work involved in developing high quality PROMs is frequently wasted due to misuse of the questionnaires. This can take the form of failing to discuss use of the measure with the developers, using versions that are incorrect or that have been amended by others, developing unauthorised adaptations and changing the scoring systems used without justification. All such modifications can lead to significant changes in the measure, making it impossible to judge the meaning of scores or make comparisons of outcomes across studies. Most published applications of the ASQoL were unauthorised, with the consequence that confidence in the measure is diminished. If researchers do not believe that the original measure is of sufficient quality, they should develop their own and show that it is superior. This problem does not only apply to the ASQoL, but to many of the need-based instruments. No doubt it is a problem for other instrument developers also.

We believe that this is an important issue that should be addressed if confidence is to be regained in the use of such PROMs. The following recommendations are made:

Instrument authors
  • Take responsibility for monitoring use of PROMs by practitioners and researchers, rather than allowing laissez faire use of measures. Clearly mark copyright on the questionnaire and the name of the copyright holders and insist that this is not removed.

  • Do not allow publication of the full measure in articles. Do not allow publication of the full measure by patient or specialist groups.

  • State clearly in publications that written permission is required to use the questionnaire or to produce adaptations or new language versions. PROMs may be administrated by other organisations, but it should be noted that these third parties should not be trusted to produce adaptations without full involvement of the test developer.

  • Attempt to monitor misuse of the PROM. However, this can be difficult for a successful, widely used measure.

Researchers/clinicians/registries
  • Conduct an adequate literature search prior to selecting a PROM suitable for their purposes. This should identify the developers of the different PROMs who will be able to provide any necessary information and grant permission for use, if appropriate, and advice on administration and scoring. If contact cannot be made with the developers or administrators, the measure should not be used.

  • Do not obtain copies of questionnaires from anyone other than the developers. All publications should provide address details for readers interested in using or adapting the measure. Do not remove copyright symbols from copies of the PROM.

  • Take care when making multiple copies of PROMs so that items or pages are not omitted.

Journal editors
  • Insist that authors provide clear written (and signed) evidence of permission to use the PROM in the specific study. This must be provided by the original developer of the PROM, not an individual who has produced a local adaptation, as this may have been unauthorised.

Reviewers
  • Ensure that the PROM(s) used in studies are properly described and are appropriate for their stated purpose. Ensure that the PROMs used are appropriately referenced; both the original instrument development and the adaptation and validation of the relevant language adaptation(s). Ask authors to confirm that the adaptations were authorised.

Such a concerted effort should help minimise the damage caused by unauthorised use of PROMs.

Notes

Compliance with ethical standards

Conflict of interest disclosure statement

The author has been a project leader on the development of a large body of patient-reported outcome measures that have each been officially adapted for use in up to 70 languages.

References

  1. 1.
    Doward LC, Spoorenberg A, Cook SA, Whalley D, Helliwell PS, Kay LJ, McKenna S, Tennant A, van der Heijde D, Chamberlain MA (2003) Development of the ASQoL: a quality of life instrument specific to ankylosing spondylitis. Ann Rheum Dis 62:20–26CrossRefPubMedPubMedCentralGoogle Scholar
  2. 2.
    Tennant A, McKenna SP, Hagell P (2004) Application of Rasch analysis in the development and application of quality of life instruments. Value Health 7(Supp 1):S22–S26CrossRefPubMedGoogle Scholar
  3. 3.
    McKenna SP, Doward LC (2005) The translation and cultural adaptation of patient-reported outcome measures. Value Health 8:89–91CrossRefPubMedGoogle Scholar
  4. 4.
    Swaine-Verdier A, Doward LC, Hagell P, Thorsen H, McKenna SP (2004) Adapting quality of life instruments. Value Health 7(Suppl 1):S27–S30CrossRefPubMedGoogle Scholar
  5. 5.
    Hunt SM, Alonso J, Bucquet D, Niero M, Wiklund I, McKenna SP (1991) Cross-cultural adaptation of health measures. Health Policy 19:33–44CrossRefPubMedGoogle Scholar
  6. 6.
    Doward LC, McKenna SP, Meads DM, Twiss J, Revicki D, Wong RL, Luo MP (2007) Translation and validation of non-English versions of the ankylosing spondylitis quality of life (ASQOL) questionnaire. Health Qual Life Outcomes 5:7CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Hagell P, Hedin PJ, Meads DM, Nyberg L, McKenna SP (2010) Effects of method of translation of patient-reported health outcome questionnaires: a randomized study of the translation of the rheumatoid arthritis quality of life (RAQoL) instrument for Sweden. Value Health 13:424–430CrossRefPubMedGoogle Scholar
  8. 8.
    Ariza-Ariza R, Hernández-Cruz B, López-Antequera G, Toyos FJ, Navarro-Sarabia F (2006) Cross-cultural adaptation and validation of a Spanish version of a specific instrument to measure health-related quality of life in patients with ankylosing spondylitis. Reumatol Clin 2:64–69CrossRefPubMedGoogle Scholar
  9. 9.
  10. 10.
    Zhao LK, Liao ZT, Li CH, Li TW, Wu J, Lin Q, Huang F, Yu DT, Gu JR (2007) Evaluation of quality of life using ASQoL questionnaire in patients with ankylosing spondylitis in a Chinese population. Rheumatol Int 27:605–611CrossRefPubMedGoogle Scholar
  11. 11.
    Zochling J (2011) Measures of symptoms and disease status in ankylosing spondylitis: ankylosing spondylitis disease activity score (ASDAS), ankylosing spondylitis quality of life scale (ASQoL), bath ankylosing spondylitis disease activity index (BASDAI), bath ankylosing spondylitis functional index (BASFI), bath ankylosing spondylitis global score (BAS-G), bath ankylosing spondylitis metrology index (BASMI), Dougados functional index (DFI), and health assessment questionnaire for the spondylarthropathies (HAQ-S). Arthritis Care Res (Hoboken) 63(Suppl 11):S47–S58CrossRefGoogle Scholar
  12. 12.
    Hamdi W, Haouel M, Ghannouchi MM, Mansour A, Kchir MM (2012) Validation of the ankylosing spondylitis quality of life questionnaire in Tunisian language. Tunis Med 90:564–570PubMedGoogle Scholar
  13. 13.
    Pham T, van der Heijde DM, Pouchot J, Guillemin F (2010) Development and validation of the French ASQoL questionnaire. Clin Exp Rheumatol 28:379–385PubMedGoogle Scholar
  14. 14.
    El M, Y EGM, El Aroussy N, Ahmed I, Youssef S, Palmer D (2011) Patient reported outcomes in ankylosing spondylitis: development and validation of a new questionnaire for functional impairment and quality of life assessment. Clin Exp Rheumatol 29:801–810Google Scholar
  15. 15.
    Fallahi S, Jamshidi AR, Bidad K, Qorbani M, Mahmoudi M (2014) Evaluating the reliability of Persian version of ankylosing spondylitis quality of life (ASQoL) questionnaire and related clinical and demographic parameters in patients with ankylosing spondylitis. Rheumatol Int 34:803–909CrossRefPubMedGoogle Scholar
  16. 16.
    Jang AR, Jang KS (2017) Structural equation modeling on health-related quality of life of patients with ankylosing spondylitis. Iran J Public Health 46:1338–1346PubMedPubMedCentralGoogle Scholar
  17. 17.
    Rajoli SR, Kanna RM, Aiyer SN, Shetty AP, Rajasekaran S (2017) Circumferential fusion through all-posterior approach in Andersson lesion. Asian Spine J 11:444–453CrossRefPubMedPubMedCentralGoogle Scholar
  18. 18.
    Bansal N, Duggal L, Jain N (2017) Validity of simplified ankylosing spondylitis disease activity scores (SASDAS) in Indian ankylosing spondylitis patients. J Clin Diagn Res 11:OC06–OC09PubMedPubMedCentralGoogle Scholar
  19. 19.
    Gupta P, Parshad R, Balakrishna P, Saraya A, Makharia GK, Sachdeva S, Sharma R (2018) Angle of his accentuation is a viable alternative to dor fundoplication as an adjunct to laparoscopic Heller cardiomyotomy: results of a randomized clinical study. Dig Dis Sci 24. [Epub ahead of print]  https://doi.org/10.1007/s10620-018-5130-4
  20. 20.
    Kchir MM, Hamdi W, Kochbati S, Azzouz D, Daoud L, Saadellaoui K, Ghannouchi MM, Kaffel D, Ben Hamida A, Zouari B (2009) Validation of the Tunisian versions of bath ankylosing spondylitis functional index (BASFI) and disease activity index (BASDAI). Tunis Med 87:527–530PubMedGoogle Scholar
  21. 21.
    Hamdi W, Chelli-Bouaziz M, Ahmed MS, Ghannouchi MM, Kaffel D, Ladeb MF, Kchir MM (2011) Correlations among clinical, radiographic, and sonographic scores for enthesitis in ankylosing spondylitis. Joint Bone Spine 78:270–274CrossRefPubMedGoogle Scholar
  22. 22.
    Younes M, Jalled A, Aydi Z, Younes K, Jguirim M, Zrour S, Ben Salah Z, Bejia I, Touzi M, Bergaoui N (2011) Quality of life in ankylosing spondylitis. Tunis Med 89:374–378PubMedGoogle Scholar
  23. 23.
    Guan M, Wang J, Zhu Z, Xiao J, Zhao L, Li Z, Shi Z (2014) Comparison in clinical features and life impact between juvenile-onset and adult-onset ankylosing spondylitis. Turk J Med Sci 44:601–605CrossRefPubMedGoogle Scholar
  24. 24.
    He MY, Fan FY (2015) Adjunctive treatment of axial undifferentiated spondyloarthritis by Qiangji recipe: a clinical study. Zhongguo Zhong Xi Yi Jie He Za Zhi 35:37–40PubMedGoogle Scholar
  25. 25.
    Zhang G, Li J, Liu D, Wang T, Wang Y, Xu W (2018) Translation and validation of the Chinese social role participation questionnaire in patients with ankylosing spondylitis. Clin Rheumatol 37:655–660CrossRefPubMedGoogle Scholar
  26. 26.
    Sun YY, Cui HJ, Dong JN, Sun P, Meng QX (2018) Randomized, Controlled Trial: Efficacy of Ultrasound and Exercise in Patients with Ankylosing Spondylitis. Altern Ther Health Med pii: AT5705. [Epub ahead of print] Google Scholar
  27. 27.
    Li G, Lv CA, Tian L, Jin LJ, Zhao W (2018) A retrospective study of alendronate for the treatment of ankylosing spondylitis. Medicine (Baltimore) 97:e10738.  https://doi.org/10.1097/MD.0000000000010738 CrossRefGoogle Scholar
  28. 28.
    Chen FC, Jin ZL, Wang DF (2018) A retrospective study of transcutaneous electrical nerve stimulation for chronic pain following ankylosing spondylitis. Medicine (Baltimore) 97:e11265.  https://doi.org/10.1097/MD.0000000000011265 CrossRefGoogle Scholar
  29. 29.
    Almodóvar R, Zarco P, Collantes E, González C, Mulero J, Fernández-Sueiro JL, Gratacós J, Torres-Alonso JC, Juanola X, Batlle E, Ariza R, Muñoz E (2009) Relationship between spinal mobility and disease activity, function, quality of life and radiology. A cross-sectional Spanish registry of spondyloarthropathies (REGISPONSER). Clin Exp Rheumatol 27:439–445PubMedGoogle Scholar
  30. 30.
    Almodóvar R, Carmona L, Zarco P, Collantes E, González C, Mulero J, Sueiro JL, Gratacós J, Torre-Alonso JC, Juanola X, Batlle E, Ariza R, Font P (2010) Fibromyalgia in patients with ankylosing spondylitis: prevalence and utility of the measures of activity, function and radiological damage. Clin Exp Rheumatol 28(6 Suppl 63):S33–S39PubMedGoogle Scholar
  31. 31.
    Abalos-Medina GM, Ruiz-Villaverde G, Sánchez-Cano D, Ruiz-Villaverde R, Ocaña-Peinado F, Villaverde-Gutiérrez C (2010) Functional level and quality of life in ankylosing spondylitis, pilot study after 16 weeks TNF blocker treatment. Rev Esp Geriatr Gerontol 45:331–334CrossRefPubMedGoogle Scholar
  32. 32.
    Bautista-Caro MB, Arroyo-Villa I, Castillo-Gallego C, de Miguel E, Peiteado D, Puig-Kröger A, MartínMola E, Miranda-Carús ME (2013) Decreased Th17 and Th1 cells in the peripheral blood of patients with early non-radiographic axial spondyloarthritis: a marker of disease activity in HLA-B27+ patients. Rheumatology (Oxford) 52:352–362CrossRefGoogle Scholar
  33. 33.
    Ariza-Ariza R, Hernández-Cruz B, Navarro-Compán V, Leyva Pardo C, Juanola X, Navarro-Sarabia F (2013) A comparison of telephone and paper self-completed questionnaires of main patient-related outcome measures in patients with ankylosing spondylitis and psoriatic arthritis. Rheumatol Int 33:2731–2736CrossRefPubMedGoogle Scholar
  34. 34.
    Rodríguez-Lozano C, Juanola X, Cruz-Martínez J, Peña-Arrébola A, Mulero J, Gratacós J et al (2013) Outcome of an education and home-based exercise programme for patients with ankylosing spondylitis: a nationwide randomized study. Clin Exp Rheumatol 31:739–748PubMedGoogle Scholar
  35. 35.
    Aguiar R, Ambrósio C, Cunha I, Barcelos A (2014) Sexuality in spondyloarthritis--the impact of the disease. Acta Reumatol Port 39:152–157PubMedGoogle Scholar
  36. 36.
    Villaverde-García V, Cobo-Ibáñez T, Candelas-Rodríguez G, Seoane-Mato D, Campo-Fontecha PDD, Guerra M et al (2017) The effect of smoking on clinical and structural damage in patients with axial spondyloarthritis: a systematic literature review. Semin Arthritis Rheum 46:569–583CrossRefPubMedGoogle Scholar
  37. 37.
    López-Medina C, Garrido-Castro JL, Castro-Jiménez J, González-Navas C, Calvo-Gutiérrez J, CastroVillegas MC et al (2018) Evaluation of quality of life in patients with axial spondyloarthritis and its association with disease activity, functionality, mobility, and structural damage. Clin Rheumatol 37:1581–1588CrossRefPubMedGoogle Scholar
  38. 38.
    Moreno M, Gratacós J, Navarro-Compán V, de Miguel E, Font P, Clavaguera T et al (2018) Should overtreatment of axial spondyloarthritis with biologics remain a concern after the issue of the new ASAS criteria? Data from REGISPONSERBIO (Spanish Register of Biological Therapy in Spondyloarthritides). Clin Exp Rheumatol [Epub ahead of print] Google Scholar

Copyright information

© International League of Associations for Rheumatology (ILAR) 2018

Authors and Affiliations

  1. 1.Galen ResearchManchesterUK
  2. 2.School of Health SciencesUniversity of ManchesterManchesterUK

Personalised recommendations