Background

One of the key factors in the evaluation of an intervention in controlled clinical trials is the clinical relevance of the selected study endpoints or outcome measures, together with an understanding of what comprises a minimal clinically important difference (MCID) in these endpoints. Establishing the MCID for study endpoints allows the clinical relevance of efficacy data from published trial results to be determined. This is of particular relevance in studies that investigate treatment efficacy in chronic, progressive diseases such as the lysosomal storage disorders.

Late-onset Pompe disease (also known as glycogen storage disease type II or acid α-glucosidase deficiency) is a rare lysosomal storage disorder caused by a genetic deficiency in the enzyme acid α-glucosidase. Overall incidence ranges from 1 in 33,000 persons to 1 in 300,000 persons, depending on geographic region and ethnicity[1–3]. Late-onset Pompe disease may develop in children and adults of any age, and presents with a wide spectrum of clinical phenotypes[4–7]. Patients with late-onset Pompe disease usually present with progressive muscle weakness (often in a limb-girdle pattern) and loss of muscular function, leading to problems with activities of daily living (ADL), reduced mobility, and eventually wheelchair use. The disease also affects the respiratory muscles resulting in the need for ventilatory support in a high proportion of patients[6, 8, 9]. Patients with untreated late-onset Pompe disease have higher mortality rates compared with the general age- and gender-matched healthy population[8], while those treated with enzyme replacement therapy (ERT) experience a 59% reduction in mortality[10].

Long-term studies and clinical trials with ERT in late-onset Pompe disease commonly include exercise capacity (normally measured using the 6-minute walking test [6MWT]) and pulmonary function (forced vital capacity [FVC]) as outcome measures. However, the clinical relevance and impact on the patients’ ADL of any observed changes in the 6MWT distance (6MWD) and FVC (measured as percentage change in predicted FVC [% predicted FVC]) in studies of late-onset Pompe disease are currently unclear. Some information exists on the clinical relevance of these outcome measures for other chronic diseases, including respiratory diseases such as chronic obstructive pulmonary disease (COPD) and idiopathic pulmonary fibrosis (IPF), but the relevance and MCID of the 6MWT and FVC in late-onset Pompe disease have not been established and there is a need to look beyond the 6MWT and FVC as clinical endpoints.

In this article we determine the clinical relevance of 6MWD and % predicted FVC (main outcome measures), which are currently used to assess late-onset Pompe patients, to compare these with the parameters used in long-term studies in other neuromuscular disorders (NMDs) such as Duchenne/Becker muscular dystrophy (DMD/BMD), and to consider the potential clinical relevance of alternative clinical endpoints in late-onset Pompe disease.

Methods

Three parallel literature searches were conducted using the PubMed database. The first search used the terms “6MWT or 6MWD”, “quality of life”, and “activities of daily living”. The second search used the terms “forced vital capacity”, “quality of life”, and “activities of daily living”. These two searches were not restricted to late-onset Pompe disease in order to identify studies in other diseases that could help interpret the clinical relevance of 6MWT and FVC as outcome measures. The final literature search was performed to identify endpoints commonly used in clinical trials in NMDs, and to investigate new endpoints and scoring systems currently under investigation in NMDs and other chronic diseases. The search was limited to articles from human studies, published in English, with full abstracts. The retrieved articles were scanned to identify papers which utilized a threshold of the 6MWD and FVC to indicate a change either in the patients’ ability to function or in the patients’ perception of their health. The reference lists of all relevant papers were also reviewed to identify any additional publications missed in the original literature search.

Outcome measures used in late-onset Pompe disease

Studies of late-onset Pompe patients commonly report absolute or relative changes in the 6MWD and the change in FVC as outcome measures. Other measures used in long-term clinical trials involving late-onset Pompe patients include the Walton Gardner Medwin (WGM) score, maximum inspiratory pressure (MIP), maximum expiratory pressure (MEP), timed muscle function tests (e.g. modified Gowers’ maneuver), and quantitative muscle testing[11–15].

The 6MWT was originally developed as an integrated assessment of cardiac, pulmonary, circulatory, and muscular capacity in patients with moderate or severe lung disease, and provides a measure of the functional exercise level required to undertake daily physical activities[16]. The 6MWT allows patients to rest when needed and, therefore, provides a measure of submaximal exercise capacity, designed to reflect the physical effort used in ADL. In a study of healthy adults aged 40–80 years (median age 58 years), the mean baseline 6MWD was 571 m with significantly shorter distances observed in those aged ≥60 years[17]. In comparison, in patients with untreated late-onset Pompe disease a decline in walking ability is observed at a much younger age than in healthy subjects and, therefore, the normal age-related decline may occur from a lower base level of walking ability in most patients. Studies of late-onset Pompe patients have reported baseline 6MWDs from 246–340 m (Angelini et al.[12] mean age at study entry 43 years [range 7–72]; Regnery et al.[13] mean age at ERT start 50.7 years [range 23–69]; van der Ploeg et al.[18] mean age 45.3 years [range 15.9–70.0]; Ravaglia et al.[19] mean age 54.2 years; and Wokke et al.[7] median age 42.6 years [range 24.3–68.5]). When evaluating the effect of a treatment in a chronic disease such as late-onset Pompe disease it is important to take into account the natural decline in function that occurs with age, as well as the deleterious impact of the disease. In this context, stabilization of muscle function may reflect a positive impact of treatment on disease progression in late-onset Pompe disease. In older late-onset Pompe patients, however, stabilization may not occur during ERT because of the natural age-related decline in walking ability, although the rate of decline may be attenuated; long-term studies are required to evaluate this.

Although it is a valuable and widely used functional measure, the 6MWT is associated with considerable inter- and intra-investigator variability[17], and does not elucidate the mechanisms behind the compromised physical function that it measures[20]. In addition, the distance walked can be affected by factors such as patient motivation, age, sex, height, and weight as well as skeletal problems[16], which can affect gait and thereby influence the distance walked. Moreover, a high proportion of late-onset Pompe patients become wheelchair-dependent over time and so the 6MWT is no longer relevant. This is an important consideration for long-term clinical studies.

FVC provides a simple measure of pulmonary function and is a widely used outcome measure in studies of patients with NMDs. However, the presence of factors such as severe scoliosis or other lung disease can impact on the reliability of results[21].

Clinical relevance of the 6MWT and FVC

6MWT: MCID

Nine studies[22–30] were identified in chronic diseases other than late-onset Pompe disease that had aimed to relate changes in the 6MWT to changes in patient perception (Table 1)[12–14, 18, 19, 22–35]. Redelmeier and colleagues found that in a cohort of 112 patients with COPD, the 6MWD would need to differ by 54 m (95% CI 37–71 m), a relative change of 15%, for patients to stop perceiving their walking ability as “about the same” and to begin to rate themselves as “a little bit better”[22]. Studies in diseases such as IPF[23], coronary artery disease following acute coronary syndrome[24], and pulmonary arterial hypertension[25] identified a 5–11% relative change from baseline as the minimal clinically important change or MCID for the 6MWT (Table 1).

Table 1 Clinical relevance of 6MWD changes in patients with chronic respiratory disease and late-onset Pompe disease
Full size table

Studies of the natural history of untreated late-onset Pompe disease[5, 7, 36] have not reported changes in walking distance as observational data. However, a randomized study of ERT in late-onset Pompe disease included data for a placebo group (30 patients) followed up for 18 months[18]. Here the mean decrease in walking distance was 3 m in untreated patients over 18 months (mean baseline 6MWD 317.9 m) (Figure 1)[12–14, 17, 18, 31, 32, 35, 37]. This is equivalent to a relative change of 0.6% annually in walking ability. Applying the 6MWD MCID to this untreated population, and assuming that the MCID for the 6MWD deterioration is the same as for improvement, it would take ≥9 years for patients to experience a clinically significant decrease in walking distance.

Figure 1
figure 1

6MWD in healthy adults and late-onset Pompe patients (untreated and those receiving treatment) [[37]]. 6MWD in healthy adults (black dot) [17], untreated late-onset Pompe patients (red line, data extracted from van der Ploeg et al. [18]), and across several studies of Pompe patients receiving alglucosidase alfa treatment [12–14, 18, 31, 32, 35]. Adapted with permission from [37] © 2013, Springer Science + Business Media. 6MWD, 6-minute walking test distance.

Full size image

Ten clinical studies of late-onset Pompe patients treated with alglucosidase alfa (Myozyme®; Genzyme, Cambridge, MA, USA) reported the 6MWT as a functional outcome measure (Table 1)[12–14, 18, 19, 31–35]. When the absolute and relative MCID for the 6MWT identified in other chronic diseases was applied to the findings of clinical trials in late-onset Pompe disease, the majority of studies (9 of 10) reported absolute changes from baseline in 6MWT that lay within[18, 19, 31, 34, 35] or above[12, 14, 32, 33] the absolute MCID levels reported for other diseases (24–54 m). All 10 clinical studies reported a relative change from baseline that was above or within the MCID range (5–11%) established for other diseases[12–14, 18, 19, 31–35]. These findings indicate that in late-onset Pompe disease ERT is associated with an improved functional capacity which, if observed in patients with respiratory disease, would be expected to manifest as either a disease stabilization or noticeable physical improvement for patients.

FVC: MCID

When FVC is used as a measure of respiratory function, predicted FVC values >80% are considered to be within normal range. In patients with chronic lung diseases, change in FVC over time is a valid outcome measure. Guidelines for the assessment of patients with systemic scleroderma cite that an improvement or reduction of 10% from baseline values is required to ensure that the variation in lung capacity can be ascribed to a change in disease severity rather than measurement error[38]. However, review of the studies identified here which used FVC as an outcome measure revealed that the definition of a relevant change from baseline in FVC is variable (Table 2)[12, 13, 18, 30, 31, 34, 39–45]. In 2 studies in patients with IPF, absolute changes of between 5 and 10% (equivalent to relative changes of 7–14%) at 6 months were considered “unchanged” or “marginal”; and “more than minimal” or “significant” changes in FVC were associated with absolute changes of >10–12% (equivalent to relative changes of >14–18%)[40, 41]. Such changes were also found to impact on quality of life (QoL; Figure 2)[41]. In a larger study of 1156 patients with IPF[39], the MCID in FVC was defined as 2–6% (equivalent to a 3–9% relative change) and changes from baseline in % predicted FVC reflected changes in global health status (Figure 3)[39].

Table 2 Clinical relevance of FVC changes in patients with chronic respiratory disease and late-onset Pompe disease
Full size table
Figure 2
figure 2

Changes in SF-36 scores stratified by changes in FVC percentage in patients with IPF [41]. Reproduced from[41] © 2010, with permission from Elsevier. BP, bodily pain; FVC, forced vital capacity; GH, general health; MCS, mental component summary; MH, mental health; PCS, physical component summary score; PF, physical functioning; RE, role emotional domain; RP, role physical; SF, social functioning; VT, vitality.

Full size image
Figure 3
figure 3

Changes in global health status according to changes in % predicted FVC in patients with IPF [[39]]. FVC, forced vital capacity. IPF, idiopathic pulmonary fibrosis.

Full size image

Evidence from studies in patients with untreated late-onset Pompe disease indicate that the expected change in % predicted FVC in the upright position and in the supine position range from −1.0% to −4.6%[5, 7, 36, 46] and 1.3% to −5.5%[5, 7, 36], respectively, annually. At the rate of disease progression observed in untreated patients with late-onset Pompe disease, and assuming that the MCID for deterioration of the FVC is the same as for improvement in FVC reported in IPF patients, untreated late-onset Pompe patients would have clinically significant worsening of lung function within 2 years.

Nine clinical studies of late-onset Pompe patients were identified that used % predicted FVC as an outcome measure[12, 13, 18, 31, 34, 42–45]. When the MCID ranges established for FVC in IPF were applied to these studies, both the absolute and relative change in FVC were above the MCID range in 2 studies[44, 45], within the MCID in 3 studies[12, 13, 43], and below the MCID in 4 studies (Table 2)[18, 31, 34, 42]. In the majority of studies of alglucosidase alfa treatment, patients who had a change in FVC within or above the MCID range established for IPF also reported a noticeable improvement. However, there were some studies in which the change observed was below the MCID, but the patients still reported feeling either “somewhat better” or “much better” in their overall health[18, 31, 34]. This correlates with findings reported in a study in IPF in which patients whose mean % predicted FVC declined by 2.1% nevertheless reported that their global health status was “somewhat better” (Figure 3)[39].

Limitations of applying MCID from other chronic diseases to late-onset Pompe disease

It is important to note that applying the MCID for the 6MWT and FVC established in studies of chronic respiratory diseases to late-onset Pompe studies has several important limitations. The IPF and late-onset Pompe studies with measure disease course in patients with a different set of pathologies and symptoms. For example, in pulmonary fibrosis, FVC reflects lung volumes and airway obstruction, not respiratory effort. Also, in respiratory diseases, QoL relates to breathlessness more than to muscle function. In this analysis of the clinical relevance of 6MWD in late-onset Pompe disease, patients perceived an improvement in their walking ability in 7 of the 10 studies identified[12, 14, 19, 31–33, 35], despite the fact that in the 3 remaining studies[13, 18, 34], the absolute changes in 6MWD were below/within, and the relative changes in 6MWD were within/above, the MCID range extrapolated from studies of other diseases. This may exemplify the problems of applying data from different diseases (e.g. COPD, IPF) to late-onset Pompe patients and highlights the need to explore alternative clinical endpoints for long-term clinical studies in these patients.

Outcome measures currently used in patients with DMD/BMD

The clinical presentation and symptoms of late-onset Pompe disease are similar to many of the NMDs, particularly DMD/BMD. Therefore, it is instructive to look at the measures of strength and function that have been used to assess progressive muscle weakness in studies of patients with other NMDs. A wide range of outcome measures have been studied in DMD, including the 6MWT and other timed functional tests (e.g. timed stair climb, time to get up from the floor)[27, 47–51]. Although timed functional activities are reliable outcome measures in DMD[49], the clinical relevance of changes measured using these methods and the correlation with day-to-day activities and QoL in DMD remains to be established. In many cases the standard tests may benefit from modification to reflect disease-specific factors; for example the 6MWT has been modified to suit the needs of patients with DMD[28].

Validated disease-specific outcome measures have been developed to assess the clinical relevance of changes in physical function during treatment of DMD. The multicomponent North Star Ambulatory Assessment (NSAA) assesses functional ability in boys with DMD[52–54], and includes the 6MWT and several other timed muscle function tests[54]. NSAA has proved to be a reliable and valid measure of walking ability in boys with DMD, as assessed by Rasch analysis[53], and it has been shown to be responsive to treatment-related change over time, supporting its use as an endpoint in clinical trials[55]. Another composite assessment recently developed for use in the study of DMD and other NMDs is the Motor Function Measure (MFM), which evaluates 3 different dimensions of motor performance. The MFM has good responsiveness in patients with DMD and good correlation with patient and physician perceptions of clinical changes[56].

Recent studies in DMD patients have shown that the use of a combination of outcome measures (e.g. NSAA, 6MWT, timed muscle function tests) are an effective approach that could provide information on different aspects of motor function which may not be detected by a single measure[54]. Using a similar approach, the Cooperative International Neuromuscular Research Group (CINRG) has developed a standardized composite muscle strength testing system — the CINRG Quantitative Measurement System (CQMS) — which combines anthropometrics and muscle, pulmonary, functional, and timed tests[57]. The CQMS is currently being evaluated in patients with DMD (ClinicalTrials.gov identifier, NCT01125709).

Progressive neuromuscular impairment has a significant impact on health-related QoL and limits ADL in patients with muscular dystrophy. This can be assessed using questionnaires such as the generic Dutch instruments TACQOL (for children aged 6–15 years) and TAAQOL (for adults)[58], and the ACTIVLIM questionnaire for ADL[59]. Interestingly, motor impairments in patients with NMDs do not correlate well with measures of activity limitation[59].

Outcome measures used in neuromuscular and other chronic diseases

Measurement of walking speed is a quick and inexpensive test, which has been shown to be predictive of survival in older adults[60, 61]. Walking speed reflects a range of factors such as energy, movement control and support, as well as the strength of cardiovascular, pulmonary, and musculoskeletal systems[60]; it may also be of value in assessing disease progression or treatment effect in NMDs. Several performance tests are used to assess walking speed in the clinical environment. In patients with amyotrophic lateral sclerosis (ALS) the 10 m walking (gait) speed is a validated and reliable measure of disease progression[62], and the timed ‘get up and go’ test has been validated in patients with multiple sclerosis (MS) and used to measure functional mobility[63].

Ideally, walking speed should be measured in an uncontrolled setting that more closely reflects real-life activity. This is now being made possible by advances in mobile accelerometry. For example, actibelt®, a 3D accelerometer which can be placed inside a belt buckle, has been developed to calculate walking speed[64]. Recent data from patients with MS suggest that although the actibelt® may be useful for measuring walking speed in patients with mild functional disabilities it was less precise than the 6MWT, and accuracy worsened in patients with more severe walking impairments[65]. Temporal and spatial parameters of gait can be measured using the GAITRite® Walkway System, an electronic pressure-sensitive walkway. GAITRite® scores have been shown to be sensitive to disease-related gait changes in patients with mucopolysaccharidosis[66] and Wolfram Syndrome[67]. In patients with MS, changes in temporo-spatial gait parameters measured using GAITRite® technology correlate with patient-reported changes in walking impairment as measured using the MS walking scale-12[68]. However, a possible shortcoming is that the use of measures of walking speed and gait parameters is limited to patients with mild and moderate disability.

Disease-specific tools developed for measuring QoL/ADL in MS and ALS patients[69, 70] might also be applicable to late-onset Pompe patients.

Future directions: improving clinical endpoints for clinical trials in late-onset Pompe disease

Although not yet evaluated in late-onset Pompe disease, the sniff nasal inspiratory pressure (SNIP) test allows earlier detection of changes in respiratory muscle strength than FVC, and has been shown to be reliable and easy for young DMD patients to perform[71]. SNIP might also be a useful measure to assess late-onset Pompe patients who have bulbar muscle involvement[72]. MIP and MEP are direct measures of respiratory function that have also been used as outcome measures in clinical studies of late-onset Pompe disease treatment[18, 72]. MIP and MEP reflect neuromuscular (diaphragmatic) ventilation dysfunction and may, therefore, better discriminate progressive muscular weakness from interstitial lung function alterations such as those seen in fibrotic lung disease. A recently published retrospective analysis of data from adults with late-onset Pompe disease found that although upright vital capacity measurement provides a strong measure that can be used for long-term follow-up of respiratory function, MEP and maximal SNIP also correlated with lung and respiratory function as measured using non-invasive transdiaphragmatic methods[73]. However, the clinical relevance of MIP and MEP as endpoints in clinical studies still requires validation. The hours of use of home mechanical ventilation (HMV) per day and the number of hospitalizations necessary due to pulmonary exacerbations were more sensitive indicators than FVC of variations in breathing autonomy in response to long-term ERT, according to a study in late-onset Pompe patients with severe pulmonary impairment[74]. Therefore, these indicators might be considered as alternative respiratory outcomes in late-onset Pompe patients with high ventilator dependency[74].

The Gait, Stairs, Chair, Gower (GSCG) score has recently been validated as an alternative outcome measure for motor function in late-onset Pompe patients receiving ERT[75]. The GSCG score is a composite test that evaluates the four main motor performances quantitatively and qualitatively and should be used in combination with both the WGM and the 6MWT to identify individual response to ERT in late-onset Pompe patients[75].

Inter- and intra-investigator variances of measurements in non-technically guided investigations should also be considered when designing new clinical trials. In this context, technically-assisted measurements may overcome the intra- and inter-investigator differences in outcome measures associated with endpoints such the 6MWT, and improve the reliability and validity of functional measurements.

Regulatory authorities are increasingly calling for outcome measures to be validated, reliable, and time-responsive. Assessments that are designed and tested using modern psychometric methods, such as Rasch analysis, meet these criteria. Recently a patient-based interval scale, the Rasch-designed Pompe-specific Activity (R-PAct) scale, has been developed and validated[76]. The R-PAct scale detects limitations in physical activities and social participation throughout the entire disease spectrum in late-onset Pompe patients aged >16 years. As a measure of disease-specific patient-relevant outcomes, the R-PAct scale may have considerable potential as an endpoint in future clinical trials (e.g. phase IV studies of ERT in late-onset Pompe disease).

Conclusions

To date, clinical trials in late-onset Pompe disease have commonly used endpoints such as the 6MWT and FVC to assess muscular and respiratory function during disease progression or treatment. Although there is evidence of the clinical relevance of these endpoints in other chronic progressive illnesses, it is important to establish that these parameters are reliable, valid, responsive, and clinically relevant for late-onset Pompe patients. Our analysis identified the range of MCID (absolute and relative) for 6MWT and FVC in chronic respiratory disease, and extrapolated these findings to the natural history of late-onset Pompe disease and to the results of clinical studies of ERT in late-onset Pompe disease. Our data suggest that in untreated late-onset Pompe patients a MCID deterioration in FVC would occur in approximately 2 years and a deterioration in the 6MWD within 9 years. In a majority of studies in which late-onset Pompe patients were treated with alglucosidase alfa, the changes from baseline in the 6MWT were above or within the MCID established in respiratory diseases, and a clinical improvement was perceived by patients in 7 of the 10 studies. Additionally, in two-thirds of the studies in which late-onset Pompe patients were treated with alglucosidase alfa, the changes from baseline in % predicted FVC were above or within the MCID established in respiratory diseases, and the difference was perceived as either an improvement or stabilization by patients. These data suggest that the changes observed during alglucosidase alfa treatment could represent significant reversal of the disease, rather than just stabilization, particularly when we consider the relatively short duration (up to 3 years) of the late-onset Pompe studies included in our analysis. However, longer-term studies are required to confirm this hypothesis. Furthermore, as the relevance of 6MWT or FVC MCIDs to late-onset Pompe patients is unclear, future studies evaluating the MCID of these two measures in these patients are also required, as well as studies evaluating the clinical value of potential alternative outcome measures. Attention should be given to identifying and validating alternative endpoints (e.g. SNIP, MIP, MEP, R-PAct) for use in future clinical trials, as well as late-onset Pompe disease-specific questionnaires addressing QoL and ADL. Such measures may be more reflective of the challenges faced by late-onset Pompe patients in their day-to-day life and have greater long-term clinical relevance.