Patient-reported outcome measure (PROM) development

All PROMs should be based on a coherent construct theory that dictates the nature of the items that should be selected for the measure. Such a theory is not simply one that states that a disease causes certain symptoms and that these should be measured. The model underlying the ASQoL is that the impact of AS on an individual’s life is the extent to which it prevents that person from meeting his or her fundamental human needs. Items for the measure were generated entirely from patients, as they are the experts at living with the disease. The original UK ASQoL [1] took items directly from the interview transcripts. This led to the inclusion in the questionnaire of several colloquial English phrases. Item reduction was governed by a strict mathematical model (Rasch measurement theory) that ensures that the final measure is unidimensional and that each item represents a distinct but different value of QoL on an interval scale [2]. Consequently, respondents can be given accurate scores that indicate their precise level of QoL.

Adapting PROMs for new cultures

All need-based measures of QoL are carefully adapted to ensure that the new language version works well in the target culture [3]. Back translation is avoided as it has been shown to produce poor adaptations [4]. In the case of the ASQoL and all need-based measures, a dual-panel methodology is employed [5]. The first (bilingual) panel provides potential translations for items and instructions. The second panel (consisting of people of low educational attainment) is an essential part of the process as it ensures that simple language is used in the final translation [6]. In a naturally blinded study, this methodology has been shown to produce translations that are more acceptable to patients [7].

When translating patient-based questionnaires such as the ASQoL, it is not enough to consider the English wording of items. Their actual meaning requires an understanding of the way in which they were expressed by the original interviewees. Additional information is provided by determining their psychometric value through the application of statistical modelling. None of this information is published for the ASQoL, preventing meaningful translations being produced, without the involvement of the instrument developers. Indeed, it could be considered that the UK version is only one of many different language versions of the questionnaire. The ASQoL contains several English colloquial phrases that are difficult to adapt without changing the wording of the item.

A Spanish adaptation of the ASQoL was reported by Ariza-Ariza [8]. This was produced without any reference to the authors of the original questionnaire or seeking their permission or help. The published article did not report on the translation method employed, so it is not possible to check how this was done. This unauthorised version differs considerably from the official Spanish ASQoL. In effect, there are now two very different ASQoL versions being used in Spain. What is particularly unhelpful is that they have the same name. Clinicians and researchers in Spain are unlikely to be aware of this problem. However, results generated from the two questionnaires will differ, meaning that the results from different studies cannot be compared. Furthermore, users of the unauthorised version will not have access to experts in use of the ASQoL or its manual, which contains background information on the questionnaire and guidelines on administration, scoring, analysis and interpretation of results.

The fact that an unauthorised adaptation for Spain exists does not imply that it can be used by others, as the authors had no authority to produce an adaptation and cannot give permission for use by others of any version of the ASQoL. An unfortunate consequence is that researchers and clinicians using the unauthorised version are also breaching copyright.

The problem in Spain was compounded by the Sociedad Española de Reumatología Fundación FER publishing the unauthorised version of the Spanish ASQoL on their website. It is no surprise that Spanish researchers have been led to believe that this is an official version of the questionnaire, freely available for use. Organisations’ misguided desire to publish existing questionnaires for the convenience of clinicians has led instrument developers to stop publishing questionnaires in journals. This prevents interested readers from judging the potential value of the PROMs as outcome measures for their proposed studies. The Society has been requested to remove the questionnaire from their website but did not respond to this request.

Where different versions of a measure exist leading to the publication of conflicting results, the quality of the scale is drawn into question. An extreme example is that five different versions of the Nottingham Health Profile have been used in the Netherlands.

Both unauthorised use and the development of unofficial adaptations of a questionnaire represent breaches of copyright. Where such actions occur, advantage is taken of other people’s work for personal benefit. Frequently, the original development work is unreferenced. The primary purpose of copyright law is not to protect the interests of the authors, but rather to promote the progress of science. Copyright also gives moral rights to be identified as the author of works, and to object to their ‘distortion and mutilation’ [9].

As test developers, we take the responsibility to maintain quality very seriously. We provide a service to individuals wishing to use a questionnaire, provide accurate versions of the available language versions, advise on how to use and score the measure and maintain a file of current and completed studies that use the outcome measure. Whenever possible, we work with researchers around the world to develop high quality adaptations, which we generally subsidise or cover the cost of in full. Use of the measures is free from licence fee for clinicians and researchers conducting non-commercial studies. Income accruing from commercial studies is used to cover the costs of administration, scientific development of the instrument, production of new language versions and publication of scientific articles. A minimal administration fee for non-commercial studies is charged, as experience shows that anything given free to users is not considered to be of value.

Misuse of the ASQoL

A review of published articles using the ASQoL cited on PubMed for 2018 alone identified ten studies describing unauthorised use of the measure. Eight of these studies employed unauthorised adaptations of the questionnaire. Consequently, the quality of the results presented in these publications must be highly questionable. It is interesting to enquire why reviewers did not question the use of these unauthorised versions, as the development papers could not have been properly referenced. Several journals require authors to confirm that they have permission to use the questionnaires.

A second PubMed review, of all articles using the ASQoL since its development, identified 125 articles, of which most were unauthorised. This highlights how difficult it is to maintain the quality of the questionnaire as a scientific instrument without the support of editors and reviewers. These articles include unauthorised adaptations of the ASQoL for Spain [8], China [10], India [11] and Tunisia [12]. In addition, a second French adaptation was produced [13] even though the authors referenced the official French adaptation without providing a justification for repeating work unnecessarily!

Of the published ASQoL articles identified, 29 employed unauthorised language versions. These were for the following countries; Egypt [14], France [13], Iran [15, 16]), India [17,18,19], Tunisia [12, 20,21,22], China [10, 23,24,25,26,27,28] and Spain [8, 29,30,31,32,33,34,35,36,37,38]. This last study is included as it used data from the Spanish Register of Biological Therapy in Spondyloarthritides. This register is related to the organisation that published the unauthorised version of the Spanish ASQoL. An unfortunate consequence of this is that the validity of ASQoL data generated by the registry must be in doubt. While the greatest misuse of the ASQoL has arisen in Spain, this is primarily the consequence of the original production of an unauthorised Spanish adaptation.

Conclusions and recommendations

The careful work involved in developing high quality PROMs is frequently wasted due to misuse of the questionnaires. This can take the form of failing to discuss use of the measure with the developers, using versions that are incorrect or that have been amended by others, developing unauthorised adaptations and changing the scoring systems used without justification. All such modifications can lead to significant changes in the measure, making it impossible to judge the meaning of scores or make comparisons of outcomes across studies. Most published applications of the ASQoL were unauthorised, with the consequence that confidence in the measure is diminished. If researchers do not believe that the original measure is of sufficient quality, they should develop their own and show that it is superior. This problem does not only apply to the ASQoL, but to many of the need-based instruments. No doubt it is a problem for other instrument developers also.

We believe that this is an important issue that should be addressed if confidence is to be regained in the use of such PROMs. The following recommendations are made:

Instrument authors

  • Take responsibility for monitoring use of PROMs by practitioners and researchers, rather than allowing laissez faire use of measures. Clearly mark copyright on the questionnaire and the name of the copyright holders and insist that this is not removed.

  • Do not allow publication of the full measure in articles. Do not allow publication of the full measure by patient or specialist groups.

  • State clearly in publications that written permission is required to use the questionnaire or to produce adaptations or new language versions. PROMs may be administrated by other organisations, but it should be noted that these third parties should not be trusted to produce adaptations without full involvement of the test developer.

  • Attempt to monitor misuse of the PROM. However, this can be difficult for a successful, widely used measure.

Researchers/clinicians/registries

  • Conduct an adequate literature search prior to selecting a PROM suitable for their purposes. This should identify the developers of the different PROMs who will be able to provide any necessary information and grant permission for use, if appropriate, and advice on administration and scoring. If contact cannot be made with the developers or administrators, the measure should not be used.

  • Do not obtain copies of questionnaires from anyone other than the developers. All publications should provide address details for readers interested in using or adapting the measure. Do not remove copyright symbols from copies of the PROM.

  • Take care when making multiple copies of PROMs so that items or pages are not omitted.

Journal editors

  • Insist that authors provide clear written (and signed) evidence of permission to use the PROM in the specific study. This must be provided by the original developer of the PROM, not an individual who has produced a local adaptation, as this may have been unauthorised.

Reviewers

  • Ensure that the PROM(s) used in studies are properly described and are appropriate for their stated purpose. Ensure that the PROMs used are appropriately referenced; both the original instrument development and the adaptation and validation of the relevant language adaptation(s). Ask authors to confirm that the adaptations were authorised.

Such a concerted effort should help minimise the damage caused by unauthorised use of PROMs.