Patient-Reported Outcome (PRO) Consortium translation process: consensus development of updated best practices

The PRO Consortium was formed in 2008 by the Critical Path Institute (C-Path) in cooperation with the U.S. Food and Drug Administration’s (FDA) Center for Drug Evaluation and Research and the pharmaceutical industry [1, 2]. Its mission is to establish and maintain a collaborative framework with appropriate stakeholders for the qualification [3] of patient-reported outcome (PRO) measures and other clinical outcome assessment (COA) tools that will be publicly available for use in clinical trials where COA-based endpoints are used to support product labeling claims.

The PRO Consortium’s structure consists of a Coordinating Committee, subcommittees that address consortium-wide topics, and therapeutic area working groups. The Coordinating Committee, including one C-Path representative and representatives from each member firm, oversees consortium operations, recommends research priorities, and approves research projects and policies, among other functions. Among all the subcommittees, the Process Subcommittee is charged with developing policies and procedures that address common concerns and issues that have implications for all PRO Consortium working groups, identifying areas for further research to advance measurement science, and identifying needs for guidance on procedural issues.

Currently, there are 10 working groups in the PRO Consortium focusing on diseases or conditions with an unmet measurement need. The goal of these working groups is to generate and/or compile the necessary evidence to enable new or existing COA measures to be qualified by FDA for use in assessing primary or secondary clinical trial endpoints.

The PRO Consortium has developed a number of novel PRO measures through its therapeutic area working groups. Once qualified by FDA, these measures could, over time, become ‘gold standard’ assessment tools for a given concept of interest and context of use. It is therefore imperative that the integrity of these measures be maintained when they are implemented in clinical trials and other research studies. Clinical trial sponsors (e.g., pharmaceutical firms) seeking to use these newly developed PRO measures frequently require translation and cultural adaptation (TCA) of the tools for use in global clinical development programs, which together with cognitive interviewing of the resulting translations is known as linguistic validation (LV) [4]. The PRO Consortium defines LV as the process of assessing and confirming the conceptual equivalence [4, 5] and content validity of translations of PRO measures. The Process Subcommittee therefore determined that a detailed LV process was required to ensure that all translations of Consortium-developed PRO measures are performed using a standardized approach with the rigor required to meet regulatory and pharmaceutical industry expectations, as well as having a clearly defined instrument translation process that the translation industry can support.

Numerous translation companies perform LV. However, during the development of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) principles of good practice for the TCA of PRO measures [6], members of the ISPOR TCA Task Force found that translation companies often employed differing methods for similar translation/adaptation tasks and used different terminology when referring to the same aspects of the translation process, all of which could increase variability between languages and potentially undermine the validity of research data collected with these measures, as well as the aggregation of global data sets. Based on their findings, a high-level set of consensus-based good practices was developed [6], hereafter referred to as ISPOR recommendations. Published in 2005, the ISPOR recommendations present a 10-step translation and cognitive interview process; this document is frequently referenced as a recognized LV methodology within the pharmaceutical industry and by FDA. Since its publication, literature reviews have been published which compare LV approaches found in published guidelines [5, 7]. Epstein and colleagues [7] found that there was no consensus in the literature reviewed and recommended that any validated method could be used, while Acquadro and colleagues [5] found that most guidelines recommend a multistep process, but varied regarding the actual steps necessary. In both cases, the authors recommended that empirical research on LV methodology is needed rather than expert-based guidelines.

We acknowledge that no ‘gold standard’ for LV of PRO measures exists [7] due to a lack of empirical evidence to support one approach over another. The Process Subcommittee decided to use the ISPOR recommendations as a starting place for the consensus process rather than to conduct additional literature reviews given the significant variability in methods already noted. The Process Subcommittee also sought an efficient method for finalizing the PRO Consortium LV process to facilitate the completion of required translations, which did not allow for the conduct of a comparative study.

While the ISPOR recommendations [6] remain relevant, the Process Subcommittee determined that the recommendations do not provide the level of detail required by end users to ensure consistency in the LV process when different translation companies are commissioned to translate PRO measures. Additionally, it is important to ensure that best practice guidelines are updated periodically to reflect any methodological advances and changes in regulatory requirements since their development. The ISPOR recommendations had been in use for a decade when the PRO Consortium’s consensus process began, and they were based on literature and experience dating back 10 years or more prior to publication. Recent feedback from representatives from clinical trial sponsors and translation companies confirms that current LV methodologies continue to differ across companies. The ISPOR recommendations were intentionally not prescriptive with regard to how the steps were performed, and as a result, member firms interested in using PRO Consortium measures in their trials requested further guidance regarding the appropriate LV process for these measures. As such, this consensus process was also driven by the need to meet increasing clinical trial requests. With the goal of ensuring consistency in the implementation of PRO Consortium-developed measures in global clinical trials and in the methods and evidence used to support their LV, the Process Subcommittee determined it would be necessary to develop a detailed LV process that reflects current best practice based on the ISPOR recommendations with adequate detail to ensure that a standardized LV methodology is used for all PRO Consortium measures.

The Process Subcommittee’s initial goal was to define a detailed LV process that reflects the rigor that regulatory agencies expect without being excessively burdensome for translation companies to perform. The FDA’s guidance for industry titled Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims (hereafter called FDA’s PRO Guidance) [8] refers to the importance of comparability of content validity and other measurement properties across all language versions of a PRO measure. Although FDA’s PRO guidance does not specify LV requirements for individual situations, it states that FDA will review methodologies and documentation surrounding steps that were taken to prepare translations for populations that will be included in clinical trials. In addition, the European Medicines Agency (EMA) in its reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products [9] states that it will evaluate the strength of evidence of validation including cultural adaptation/translation as applicable to the study. Awareness of these regulatory recommendations remained at the forefront of the PRO Consortium’s LV process development. Given the time that has passed since the release of the ISPOR recommendations [6] and EMA reflection paper [9] in 2005 and the FDA’s PRO Guidance in 2009 [8], the Process Subcommittee felt it would be beneficial to review and evaluate the ISPOR recommendations, possibly supplement them, and provide updates based on feedback solicited from translation companies and pharmaceutical industry experts. The ultimate goal was to create a translation process where content validity and other measurement properties are comparable to the original language version.

There is not always a clear path forward when determining translation needs based on regulatory guidance. Based on feedback from translation agencies and clinical trial sponsors, the Process Subcommittee considered it would be helpful to the pharmaceutical industry, the translation industry, and regulatory agencies to provide a uniform translation process as a reference. This would allow alignment on the topic and a basic understanding of good LV practices.

This paper is meant to complement the ISPOR recommendations and focuses on improvement in qualitative methods for translation and LV by providing additional detail, steps, and information to clinical trial sponsors and to translation companies. This paper summarizes the consensus process used to review and expand upon the ISPOR recommendations and outlines the resulting steps in the PRO Consortium-defined LV process. The final translation process documents are currently in use as guideline documents for member firms that are funding new language translations of PRO Consortium measures for use in their multinational trials.

The PRO Consortium followed a structured, iterative feedback process to reach consensus on its LV methods. The process was similar to a modified Delphi approach [10, 11] in that information was gathered from individuals representing companies experienced in LV as well as researchers with this expertise, and their information was consolidated and returned to the group members for further comment and feedback in two iterative rounds. The process differed in that each individual’s feedback from each round was not shared with group members in successive rounds, but rather adjudicated by the PRO Consortium team (consisting of the Executive Director, two Project Managers, an LV expert, and an external measure development consultant) to prepare revised documents for the following round, and there was no final vote taken to reach consensus. Group members were asked whether they had strong objections to accepting the proposed process in the final feedback round. The LV process was finalized at the point at which no further objections from either the translation company representatives or the pharmaceutical industry representatives were raised.

In 2015, the PRO Consortium initiated the development of an instrument translation process by requesting information from 13 translation companies, members of the Process Subcommittee, and other LV experts to determine whether the ISPOR recommendations were being followed and to identify additional best practices within the translation industry that exceeded the ISPOR recommendations. This information-gathering process involved reviewing documentation (e.g., guidelines, websites) from the 13 translation companies and conducting follow-up discussions via telephone or email to clarify the exact LV methods used by each company. A matrix document was created which listed the steps from the ISPOR recommendations, and then each company’s version of the steps to facilitate comparison of the execution of each step, which resulted in a total of 17 steps. Individual translation company procedures were compared with each other and against the ISPOR recommendations to identify the most rigorous approach for each suggested step in the process based on PRO Consortium team determination. A document was created that listed 17 steps in three columns: the ISPOR steps in column 1, the most rigorous steps identified in our review in column 2, and then the proposed PRO Consortium version of each step in column 3. Rather than looking for the most commonly performed way of executing each step, this approach was intended to identify the best practice approach for each step. The document was then reviewed by an LV expert to ensure that regulatory needs would be met by the proposed PRO Consortium steps and to identify steps that were either company-specific and not required as part of best practice or not necessary for regulatory purposes. This review led to the removal of four steps which were either deemed too burdensome and not required as part of best practice (i.e., risk mitigation assessment and plan, risk audit and summary report) or not the responsibility of the translation company (i.e., establishing a document repository, translatability assessment), and consolidation of two steps into one due to redundancy (forward translation reconciliation and establish reconciled forward translation).

In early 2016, the PRO Consortium used the 12 steps identified during the LV best practices information-gathering stage to draft a proposed Instrument Translation Process in the form of a numbered table that described each recommended step in moderate detail. The proposed process was distributed to the original 13 translation companies along with three others identified in 2016 which perform LV of PRO measures and were known to PRO Consortium member firms. In an effort to reach consensus, the translation companies were asked to review the proposed process document and provide constructive feedback including, if applicable, an indication of agreement with the process described. Recommendations on ways to improve the delineation of the process or identification of areas of concern were also requested. This round of review took place between March and May 2016, with written responses received from 13 of the 16 (81%) companies invited to participate. Concerns were raised regarding most steps and/or descriptions in the process. These concerns were reviewed by the LV expert and then adjudicated by the Process Subcommittee.

Based on feedback received from translation companies during the initial round of review, a number of significant changes were made to the process, additional process documents were prepared in response to needs identified in the feedback round, and these documents were then circulated for review and comment by translation company representatives and Process Subcommittee representatives in July 2016.

A consensus development teleconference was held in August 2016, attended by nine translation company representatives and four Process Subcommittee members, during which further feedback on the revised process documents was discussed by attendees. Additional revisions were made following the teleconference to address the remaining issues and final versions of the process documents were reviewed by the Process Subcommittee and formally approved by the PRO Consortium’s Coordinating Committee. The documents are now available for use by PRO Consortium member firms as well as the broader scientific community.

Changes to the PRO consortium instrument translation process based on input from multiple stakeholders

A number of changes were made to the PRO Consortium’s instrument translation process based on the feedback received from translation companies and member firms. As shown in Fig. 1, the single process document originally distributed was inadequate to delineate the process envisioned by the PRO Consortium, and led to the creation of several separate documents to explain the PRO Consortium Instrument Translation Process: an overview, glossary, flowcharts, and separate process documents to address universal and country-specific approaches (See Table 1. Definitions of Key Terms for definitions). The country-specific process was separated into two documents, one for new translations and another for adaptations of an existing translation.

Open image in new window

Table 1

Definitions of Key Terms

Key Term	Definition
Back-translation	Process of translating a document that has already been translated into another language back to the original language - preferably by an independent translator.
Country-specific translation	Translation approach focused on respecting the diversity of languages and sense of cultural identity of the target populations through their languages. Involves preparing separate translations for countries sharing the same language (e.g., separate Spanish translations for Spain, Mexico, Chile, and U.S.).
Forward translation	Translation from the source language to the target language.
International Harmonization	Harmonization of all translations with each other and the source version. Objective is to detect and deal with any discrepancies between different language versions that threaten conceptual equivalence and comparability across languages. Provides an additional quality control step and further ensures that data from global trials can be safely aggregated.
Linguistic validation	The process of assessing and confirming the conceptual equivalence [4, 5] and content validity of translations of patient-reported outcome (PRO) measures. Usually, linguistic validation refers to a process whereby translated text is actively tested with patients in the target population and target language group through cognitive interviews.
Preparation	Initial planning and actions carried out before the translation process begins, such as identifying translation consultants and in-country affiliates and creating translation files if needed.
Forward translation reconciliation	Process of comparing and merging more than one forward translation into a single forward translation resulting in a reconciled forward translation.
Universal translation	Translation approach focused on commonalities rather than differences to develop one version to be used in regions or countries speaking the same language (e.g., preparing a single Spanish translation that will be used by all Spanish-speaking countries).

Other significant changes to the PRO Consortium Translation Process included the following:

Universal vs country-specific approaches

Although most translation companies perform both types of translations, six supported the PRO Consortium’s preference for universal translations, while seven raised questions or expressed concerns with the approach. Change: The ‘Preparation Step’ was revised to state that C-Path, in conjunction with the clinical trial sponsor, will determine if the translations to be performed will follow the universal or country-specific approach.

Use of certified translators

Five translation companies disagreed with this requirement, noting that many of the best translators are not certified and that certification is not available in some countries. Change: Reference to ‘certified’ was replaced with ‘qualified’ to emphasize the need for translators with appropriate qualifications to perform the translations as opposed to having people without proper training translate PRO measures.

Forward translations

Forward translations are handled differently by many translation companies and a minimum number had not been defined. Change: The ‘Forward Translation Step’ was revised to reflect a minimum requirement of two forward translations.

Forward translation reconciliation

The reconciliation process is handled differently by translation companies and the approach initially proposed (independent third party reconciles the two forward translations) was considered too complex by many of them. This finding is consistent with Koller and colleagues [12], who provide recommendations for decision-making during reconciliation but did not find consensus on the people involved in the process. Change: In the ‘Forward Translation Reconciliation Step,’ multiple options are provided as examples for accomplishing this process, noting that the decision-making must be well-documented.

In-country consultants

Several translation companies expressed concerns and some confusion regarding the involvement of in-country consultants while others strongly supported their involvement. Change: Clarified roles on process documents and in Glossary.

Harmonization

There was general confusion regarding what this involved. Change: This step was relabeled the ‘International Harmonization Step’ and requested only a description of the planned methodology and rationale.

In-country affiliates

Several translation companies questioned whether in-country affiliates should be involved in conjunction with the proofreading step prior to cognitive interviewing, while others felt that their involvement early in the process was very important. Change: This step was made mandatory to provide affiliate representatives with a recognized role in the process and to prevent unsolicited feedback after translations were finalized that could threaten their validity, and a recommendation was added for the clinical trial sponsor to identify affiliates early (during ‘Preparation’), and that, if none are available, alternatives would be provided by the translation company.

ePRO translations

Many translation companies questioned how electronic PRO (ePRO) implementation should be integrated into the process. Change: Addressed in ‘Final Review and Documentation (Proofreading) Step’ as recommended best practice.

Clinician review

This was suggested by one translation company and one member firm as a new step. Decision: The sponsor can include this step if desired but it is not required in the PRO Consortium translation process.

Detailed descriptions of the Steps mentioned above are available in Table 3.

Despite the fact that recommendations for a rigorous approach to translation of PRO measures have been in place for decades [5, 7, 18, 19, 20, 21, 22], translations for clinical trials have not followed a consistent methodology and were sometimes performed by local language speakers who were not qualified translators. However, the member firm representatives within the PRO Consortium recognize that this approach can put valuable PRO data at risk of bias or increased variability that can attenuate an efficacy signal. Therefore, the PRO Consortium member firms support, and will benefit from, having a well-defined and clear process for LV of the measures emerging from the Consortium’s working groups, which are intended to serve as primary or key secondary endpoint measures in support of product label claims. The PRO Consortium translation process will play an important role in maintaining the quality of the data generated through these measures by ensuring that they are translated by qualified linguists following an established, consistent, and rigorous process that reflects best practice. The PRO Consortium’s translation process meets the minimum standards recommended by a task force from the International Society of Quality of Life Research (ISOQOL) in that it documents the methods used and includes qualitative evidence (i.e., cognitive interviews) to evaluate the translation [23]. Regulatory agencies are showing increased concern regarding the cross-cultural suitability of PRO measures that generate data used to support clinical trial endpoints [3, 8, 9]. The FDA’s PRO guidance [8] includes an expectation that translation of such measures follows a recognized process and that the measurement properties are comparable across languages, and the EMA’s reflection paper [9] states that evidence supporting cultural adaptation/translation is expected. The PRO Consortium’s translation process is intended to maintain the integrity of its FDA-qualified measures by ensuring that these regulatory standards will be met. It provides a standardized, consensus-driven approach that enhances the rigor of LV for PRO measures and advances measurement science in a multinational clinical trial environment.