It is time for an organised, scientific approach to the application of patient-reported outcome measures in clinical studies and trials
- 119 Downloads
Too often, clinicians and researchers assume that patient-reported outcome measures (PROMs) are simply designed, can be changed to suit specific circumstances and should be freely available for use without seeking permission. If a version is not easily available, they believe they can produce a new translation without relevant experience or expertise. Copyright issues are inconvenient and commonly ignored, despite some journals requesting confirmation from authors that permission had been granted for use of copyrighted materials. When informed that authors of an article did not have such permission, the response from an editor was that nothing could be done as he had to rely on the word of the authors.
In fact, high quality PROMs are carefully designed, developed for new cultures by means of proven standardised methodologies, carefully re-validated in the new culture and then administered by a responsible organisation to ensure that they are used appropriately in studies. If such care has not been taken, questions should be raised about the quality of the measure and the information it generates.
Several problems have arisen with the Ankylosing Spondylitis Quality of Life Questionnaire (ASQoL). This is probably because it is recognised as the most appropriate measure of quality of life for people with Ankylosing Spondylitis and has been widely used in international clinical trials and studies. Fifty-seven official language adaptations of the questionnaire have been produced. Unfortunately, as a result of unauthorised adaptation and use, questions are raised about the validity of several publications that have used the ASQoL.
KeywordsAdaptations Ankylosing Spondylitis Quality of Life Questionnaire (ASQoL) Copyright Patient-reported outcomes
Patient-reported outcome measure (PROM) development
All PROMs should be based on a coherent construct theory that dictates the nature of the items that should be selected for the measure. Such a theory is not simply one that states that a disease causes certain symptoms and that these should be measured. The model underlying the ASQoL is that the impact of AS on an individual’s life is the extent to which it prevents that person from meeting his or her fundamental human needs. Items for the measure were generated entirely from patients, as they are the experts at living with the disease. The original UK ASQoL  took items directly from the interview transcripts. This led to the inclusion in the questionnaire of several colloquial English phrases. Item reduction was governed by a strict mathematical model (Rasch measurement theory) that ensures that the final measure is unidimensional and that each item represents a distinct but different value of QoL on an interval scale . Consequently, respondents can be given accurate scores that indicate their precise level of QoL.
Adapting PROMs for new cultures
All need-based measures of QoL are carefully adapted to ensure that the new language version works well in the target culture . Back translation is avoided as it has been shown to produce poor adaptations . In the case of the ASQoL and all need-based measures, a dual-panel methodology is employed . The first (bilingual) panel provides potential translations for items and instructions. The second panel (consisting of people of low educational attainment) is an essential part of the process as it ensures that simple language is used in the final translation . In a naturally blinded study, this methodology has been shown to produce translations that are more acceptable to patients .
When translating patient-based questionnaires such as the ASQoL, it is not enough to consider the English wording of items. Their actual meaning requires an understanding of the way in which they were expressed by the original interviewees. Additional information is provided by determining their psychometric value through the application of statistical modelling. None of this information is published for the ASQoL, preventing meaningful translations being produced, without the involvement of the instrument developers. Indeed, it could be considered that the UK version is only one of many different language versions of the questionnaire. The ASQoL contains several English colloquial phrases that are difficult to adapt without changing the wording of the item.
A Spanish adaptation of the ASQoL was reported by Ariza-Ariza . This was produced without any reference to the authors of the original questionnaire or seeking their permission or help. The published article did not report on the translation method employed, so it is not possible to check how this was done. This unauthorised version differs considerably from the official Spanish ASQoL. In effect, there are now two very different ASQoL versions being used in Spain. What is particularly unhelpful is that they have the same name. Clinicians and researchers in Spain are unlikely to be aware of this problem. However, results generated from the two questionnaires will differ, meaning that the results from different studies cannot be compared. Furthermore, users of the unauthorised version will not have access to experts in use of the ASQoL or its manual, which contains background information on the questionnaire and guidelines on administration, scoring, analysis and interpretation of results.
The fact that an unauthorised adaptation for Spain exists does not imply that it can be used by others, as the authors had no authority to produce an adaptation and cannot give permission for use by others of any version of the ASQoL. An unfortunate consequence is that researchers and clinicians using the unauthorised version are also breaching copyright.
The problem in Spain was compounded by the Sociedad Española de Reumatología Fundación FER publishing the unauthorised version of the Spanish ASQoL on their website. It is no surprise that Spanish researchers have been led to believe that this is an official version of the questionnaire, freely available for use. Organisations’ misguided desire to publish existing questionnaires for the convenience of clinicians has led instrument developers to stop publishing questionnaires in journals. This prevents interested readers from judging the potential value of the PROMs as outcome measures for their proposed studies. The Society has been requested to remove the questionnaire from their website but did not respond to this request.
Where different versions of a measure exist leading to the publication of conflicting results, the quality of the scale is drawn into question. An extreme example is that five different versions of the Nottingham Health Profile have been used in the Netherlands.
Both unauthorised use and the development of unofficial adaptations of a questionnaire represent breaches of copyright. Where such actions occur, advantage is taken of other people’s work for personal benefit. Frequently, the original development work is unreferenced. The primary purpose of copyright law is not to protect the interests of the authors, but rather to promote the progress of science. Copyright also gives moral rights to be identified as the author of works, and to object to their ‘distortion and mutilation’ .
As test developers, we take the responsibility to maintain quality very seriously. We provide a service to individuals wishing to use a questionnaire, provide accurate versions of the available language versions, advise on how to use and score the measure and maintain a file of current and completed studies that use the outcome measure. Whenever possible, we work with researchers around the world to develop high quality adaptations, which we generally subsidise or cover the cost of in full. Use of the measures is free from licence fee for clinicians and researchers conducting non-commercial studies. Income accruing from commercial studies is used to cover the costs of administration, scientific development of the instrument, production of new language versions and publication of scientific articles. A minimal administration fee for non-commercial studies is charged, as experience shows that anything given free to users is not considered to be of value.
Misuse of the ASQoL
A review of published articles using the ASQoL cited on PubMed for 2018 alone identified ten studies describing unauthorised use of the measure. Eight of these studies employed unauthorised adaptations of the questionnaire. Consequently, the quality of the results presented in these publications must be highly questionable. It is interesting to enquire why reviewers did not question the use of these unauthorised versions, as the development papers could not have been properly referenced. Several journals require authors to confirm that they have permission to use the questionnaires.
A second PubMed review, of all articles using the ASQoL since its development, identified 125 articles, of which most were unauthorised. This highlights how difficult it is to maintain the quality of the questionnaire as a scientific instrument without the support of editors and reviewers. These articles include unauthorised adaptations of the ASQoL for Spain , China , India  and Tunisia . In addition, a second French adaptation was produced  even though the authors referenced the official French adaptation without providing a justification for repeating work unnecessarily!
Of the published ASQoL articles identified, 29 employed unauthorised language versions. These were for the following countries; Egypt , France , Iran [15, 16]), India [17, 18, 19], Tunisia [12, 20, 21, 22], China [10, 23, 24, 25, 26, 27, 28] and Spain [8, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38]. This last study is included as it used data from the Spanish Register of Biological Therapy in Spondyloarthritides. This register is related to the organisation that published the unauthorised version of the Spanish ASQoL. An unfortunate consequence of this is that the validity of ASQoL data generated by the registry must be in doubt. While the greatest misuse of the ASQoL has arisen in Spain, this is primarily the consequence of the original production of an unauthorised Spanish adaptation.
Conclusions and recommendations
The careful work involved in developing high quality PROMs is frequently wasted due to misuse of the questionnaires. This can take the form of failing to discuss use of the measure with the developers, using versions that are incorrect or that have been amended by others, developing unauthorised adaptations and changing the scoring systems used without justification. All such modifications can lead to significant changes in the measure, making it impossible to judge the meaning of scores or make comparisons of outcomes across studies. Most published applications of the ASQoL were unauthorised, with the consequence that confidence in the measure is diminished. If researchers do not believe that the original measure is of sufficient quality, they should develop their own and show that it is superior. This problem does not only apply to the ASQoL, but to many of the need-based instruments. No doubt it is a problem for other instrument developers also.
We believe that this is an important issue that should be addressed if confidence is to be regained in the use of such PROMs. The following recommendations are made:
Take responsibility for monitoring use of PROMs by practitioners and researchers, rather than allowing laissez faire use of measures. Clearly mark copyright on the questionnaire and the name of the copyright holders and insist that this is not removed.
Do not allow publication of the full measure in articles. Do not allow publication of the full measure by patient or specialist groups.
State clearly in publications that written permission is required to use the questionnaire or to produce adaptations or new language versions. PROMs may be administrated by other organisations, but it should be noted that these third parties should not be trusted to produce adaptations without full involvement of the test developer.
Attempt to monitor misuse of the PROM. However, this can be difficult for a successful, widely used measure.
Conduct an adequate literature search prior to selecting a PROM suitable for their purposes. This should identify the developers of the different PROMs who will be able to provide any necessary information and grant permission for use, if appropriate, and advice on administration and scoring. If contact cannot be made with the developers or administrators, the measure should not be used.
Do not obtain copies of questionnaires from anyone other than the developers. All publications should provide address details for readers interested in using or adapting the measure. Do not remove copyright symbols from copies of the PROM.
Take care when making multiple copies of PROMs so that items or pages are not omitted.
Insist that authors provide clear written (and signed) evidence of permission to use the PROM in the specific study. This must be provided by the original developer of the PROM, not an individual who has produced a local adaptation, as this may have been unauthorised.
Ensure that the PROM(s) used in studies are properly described and are appropriate for their stated purpose. Ensure that the PROMs used are appropriately referenced; both the original instrument development and the adaptation and validation of the relevant language adaptation(s). Ask authors to confirm that the adaptations were authorised.
Such a concerted effort should help minimise the damage caused by unauthorised use of PROMs.
Compliance with ethical standards
Conflict of interest disclosure statement
The author has been a project leader on the development of a large body of patient-reported outcome measures that have each been officially adapted for use in up to 70 languages.
- 7.Hagell P, Hedin PJ, Meads DM, Nyberg L, McKenna SP (2010) Effects of method of translation of patient-reported health outcome questionnaires: a randomized study of the translation of the rheumatoid arthritis quality of life (RAQoL) instrument for Sweden. Value Health 13:424–430CrossRefPubMedGoogle Scholar
- 8.Ariza-Ariza R, Hernández-Cruz B, López-Antequera G, Toyos FJ, Navarro-Sarabia F (2006) Cross-cultural adaptation and validation of a Spanish version of a specific instrument to measure health-related quality of life in patients with ankylosing spondylitis. Reumatol Clin 2:64–69CrossRefPubMedGoogle Scholar
- 9.Irish Patents Office (2018). https://www.patentsoffice.ie/en/Copyright/Benefits-of-CopyrightProtection/
- 11.Zochling J (2011) Measures of symptoms and disease status in ankylosing spondylitis: ankylosing spondylitis disease activity score (ASDAS), ankylosing spondylitis quality of life scale (ASQoL), bath ankylosing spondylitis disease activity index (BASDAI), bath ankylosing spondylitis functional index (BASFI), bath ankylosing spondylitis global score (BAS-G), bath ankylosing spondylitis metrology index (BASMI), Dougados functional index (DFI), and health assessment questionnaire for the spondylarthropathies (HAQ-S). Arthritis Care Res (Hoboken) 63(Suppl 11):S47–S58CrossRefGoogle Scholar
- 14.El M, Y EGM, El Aroussy N, Ahmed I, Youssef S, Palmer D (2011) Patient reported outcomes in ankylosing spondylitis: development and validation of a new questionnaire for functional impairment and quality of life assessment. Clin Exp Rheumatol 29:801–810Google Scholar
- 15.Fallahi S, Jamshidi AR, Bidad K, Qorbani M, Mahmoudi M (2014) Evaluating the reliability of Persian version of ankylosing spondylitis quality of life (ASQoL) questionnaire and related clinical and demographic parameters in patients with ankylosing spondylitis. Rheumatol Int 34:803–909CrossRefPubMedGoogle Scholar
- 19.Gupta P, Parshad R, Balakrishna P, Saraya A, Makharia GK, Sachdeva S, Sharma R (2018) Angle of his accentuation is a viable alternative to dor fundoplication as an adjunct to laparoscopic Heller cardiomyotomy: results of a randomized clinical study. Dig Dis Sci 24. [Epub ahead of print] https://doi.org/10.1007/s10620-018-5130-4
- 26.Sun YY, Cui HJ, Dong JN, Sun P, Meng QX (2018) Randomized, Controlled Trial: Efficacy of Ultrasound and Exercise in Patients with Ankylosing Spondylitis. Altern Ther Health Med pii: AT5705. [Epub ahead of print] Google Scholar
- 29.Almodóvar R, Zarco P, Collantes E, González C, Mulero J, Fernández-Sueiro JL, Gratacós J, Torres-Alonso JC, Juanola X, Batlle E, Ariza R, Muñoz E (2009) Relationship between spinal mobility and disease activity, function, quality of life and radiology. A cross-sectional Spanish registry of spondyloarthropathies (REGISPONSER). Clin Exp Rheumatol 27:439–445PubMedGoogle Scholar
- 30.Almodóvar R, Carmona L, Zarco P, Collantes E, González C, Mulero J, Sueiro JL, Gratacós J, Torre-Alonso JC, Juanola X, Batlle E, Ariza R, Font P (2010) Fibromyalgia in patients with ankylosing spondylitis: prevalence and utility of the measures of activity, function and radiological damage. Clin Exp Rheumatol 28(6 Suppl 63):S33–S39PubMedGoogle Scholar
- 32.Bautista-Caro MB, Arroyo-Villa I, Castillo-Gallego C, de Miguel E, Peiteado D, Puig-Kröger A, MartínMola E, Miranda-Carús ME (2013) Decreased Th17 and Th1 cells in the peripheral blood of patients with early non-radiographic axial spondyloarthritis: a marker of disease activity in HLA-B27+ patients. Rheumatology (Oxford) 52:352–362CrossRefGoogle Scholar
- 33.Ariza-Ariza R, Hernández-Cruz B, Navarro-Compán V, Leyva Pardo C, Juanola X, Navarro-Sarabia F (2013) A comparison of telephone and paper self-completed questionnaires of main patient-related outcome measures in patients with ankylosing spondylitis and psoriatic arthritis. Rheumatol Int 33:2731–2736CrossRefPubMedGoogle Scholar
- 36.Villaverde-García V, Cobo-Ibáñez T, Candelas-Rodríguez G, Seoane-Mato D, Campo-Fontecha PDD, Guerra M et al (2017) The effect of smoking on clinical and structural damage in patients with axial spondyloarthritis: a systematic literature review. Semin Arthritis Rheum 46:569–583CrossRefPubMedGoogle Scholar
- 37.López-Medina C, Garrido-Castro JL, Castro-Jiménez J, González-Navas C, Calvo-Gutiérrez J, CastroVillegas MC et al (2018) Evaluation of quality of life in patients with axial spondyloarthritis and its association with disease activity, functionality, mobility, and structural damage. Clin Rheumatol 37:1581–1588CrossRefPubMedGoogle Scholar
- 38.Moreno M, Gratacós J, Navarro-Compán V, de Miguel E, Font P, Clavaguera T et al (2018) Should overtreatment of axial spondyloarthritis with biologics remain a concern after the issue of the new ASAS criteria? Data from REGISPONSERBIO (Spanish Register of Biological Therapy in Spondyloarthritides). Clin Exp Rheumatol [Epub ahead of print] Google Scholar