Background

During the past few decades quality of life assessment has become one central outcome in treatment of patients with chronic lower limb ischaemia. Different generic quality of life instruments such as the Nottingham Health Profile (NHP) and the Short Form 36 Health Survey (SF-36) [1, 2] have previously been tested, revealing conflicting results in these patients according to psychometric attributes in short-term evaluations. The strengths and weakness of the NHP and the SF-36 scales are not extensively examined and further research is needed to establish which is the more appropriate and responsive quality of life instrument for patients with chronic lower limb ischaemia in the long term. The main goal of vascular surgical treatment is the relief of symptoms and improvement in patients quality of life. A majority of the patients are elderly and have generally widespread arterial disease with numbers of symptoms due to the chronic lower limb ischaemia, which may affect the patients' quality of life [3–5]. Intermittent claudication (IC) means leg pain constantly produced by walking or muscular activity and is relieved by rest, while critical leg ischaemia (CLI) means pain even at rest and problems with non-healing ulcers or gangrene [6]. It is important to identify dimensions which are influenced by the severity and nature of the disease when selecting a suitable quality of life instrument [7].

The World Health Organization QOL group [8] has identified and recommended five broad dimensions – physical and psychological health, social relationship perceptions, function and well-being – which should be included in a generic quality of life instrument. Generic instruments cover a broad range of dimensions and allow comparisons between different groups of patients. Disease-specific instruments, on the other hand, are specially designed for a particular disease, patient group or areas of function [9]. The functional scale, Walking Impairment Questionnaire (WIQ) [10] and quality of life instruments such as Intermittent Claudication Questionnaire (ICQ) [11] and Claudication Scale (CLAU-S) [12] are examples of disease-specific instruments which have been developed in recent years for patients with IC. However, at present there is no accepted disease-specific questionnaire for quality of life assessment in patients with CLI. Nevertheless, the TransAtlantic Inter-Society Consensus (TASC) [6] recommended that quality of life instruments should be used in all clinical trials and preferably include both generic and disease-specific quality of life measures.

Outcome measures need to satisfy different criteria to be useful as a suitable health outcome instrument in clinical practice. Construct validity is one of the most important characteristics and is a lengthy and ongoing process [13]. An essential consideration is the instrument's ability to discriminate between different levels of the disease; another consideration is its reliability, which means the degree to which the instrument is free from random error and all items measure the same underlying attribute [14]. Further, the requirement for a useful outcome measure is the responsiveness in detecting small but important clinical changes of quality of life in patients following vascular interventions [13]. Finally the ideal quality of life instrument must also be acceptable to patients, simple and easy to use and preferably short. Comparisons among quality of life instruments and their psychometric characteristics and performance are needed to provide recommendations about their usefulness as outcome measures for these particular groups of patients.

The aim of this study was to compare two generic quality of life questionnaires, the Nottingham Health Profile (NHP) and the Short Form 36 Health Survey (SF-36) regarding the internal consistency reliability, validity, responsiveness and suitability as outcome measures in patients with lower limb ischaemia in a longitudinal perspective.

Methods

Patients

Ninety consecutive patients from a Swedish vascular unit in southern Sweden were invited to participate in this study. The assessment took place before and 12 months after revascularization. Out of 90 patients, 24 (27%) dropped out during the follow-up period, of whom 14 suffered from CLI. Six patients (7%) died, 15 (17%) did not wish to participate and 3 (3%) had other concurrent diseases. The inclusion criteria were patients admitted for active treatment of documented lower limb ischaemia, having no communication problems and having no other disease restricting their walking capacity [1]. The severity of ischaemia was graded according to suggested standards for grading lower limb ischaemia [15]. Sixty-two (68.8%) patients were treated with a surgical bypass, 24 (26.6%) had a percutaneous angioplasty (PTA) and 4 (4.6%) had a surgical thromboendatherectomy (TEA) (Table 1). Routine medical history, risk factors and clinical examinations, which included ankle blood pressure (ABP) and ankle-brachial pressure index (ABPI), were obtained before and one year after revascularization in accordance with the Swedish Vascular Registry (Swedvasc) [16]. The questionnaire also contains questions about sex, age, housing and civil status. Demographic characteristics and clinical data were obtained from the patients' medical records.

Table 1 Demographic characteristics of the patient groups before revascularization (n = 90).
Full size table

Nottingham Health Profile

The Nottingham Health Profile (NHP) was developed to be used in epidemiological studies of health and disease [17]. It consists of two parts. Part I contains 38 yes/no items in 6 dimensions: pain, physical mobility, emotional reactions, energy, social isolation and sleep. Part II contains 7 general yes/no questions concerning daily living problems. The two parts may be used independently and part II is not analysed in this study. Part I is scored using weighted values which give a range of possible scores from zero (no problems at all) to 100 (presence of all problems within a dimension). Swedish weights have been developed and used in this study [18]. The Swedish version has proved to be valid and reliable, for example, in patients with arthrosis of the hip joint [19] and in patients suffering from grave ventricular arrhythmias [20]. The NHP scale has also proved capable of measuring changes in perceived health following different treatments such as radical surgery for colorectal cancer [21] and after vascular interventions in lower limb ischaemia patients [4, 22, 23].

Short Form 36 Health Survey

The Short Form 36 Health Survey (SF-36) was developed by Ware et al [24] and designed to provide assessments involving generic health concepts that are not specific to age, disease or treatment group. It includes 36 items covering eight health concepts: bodily pain, physical functioning, role limitations due to physical problems, mental health, vitality, social functioning, role limitations due to emotional problems and general health. The response format is yes or no or in a three-to-six response scale. For each health concept questions scores are coded, summed and transformed on a scale from zero (worst health) to 100 (best health). In this study, the standard Swedish version was used [25]. The SF-36 has shown acceptable validity and reliability in population studies [26, 27] and in various groups of patients, for example after stroke [28] and in patients with rheumatoid arthritis [29]. The SF-36 scale has also shown responsiveness to changes in health status over time in patients with critical ischaemia [30–32] and in patients with intermittent claudication following a revascularization [33–35].

Procedure

The patients were asked by the head nurse to fill out the NHP and the SF-36 questionnaire during their admission before treatment. At the one-year follow-up, the questionnaire was sent home to the patients with a covering letter and a prepaid envelope. The Ethics Committee of Lund University approved the study (LU 470-98, Gbg M 098-98).

Statistical analysis

Differences in characteristics between patients with IC and with CLI before revascularization were analysed using Chi-squared test and Mann-Whitney U-test. The prevalence of the lowest ("floor" effect) and highest ("ceiling" effect) possible quality of life score in NHP and SF-36 was also calculated.

Construct validity was evaluated for aspects of convergent and discriminant validity by the Multitrait-Multimethod matrix (MTMM) [13] based on five comparable domains, including pain, physical mobility, emotional reactions, energy and social isolation for the NHP and bodily pain, physical functioning, mental health, vitality and social functioning for the SF-36 (Table 2). Further, the Mann-Whitney U-test was used to examine the relative ability of the two instruments to discriminate among the degrees of severity of the peripheral vascular disease. Spearman's rank correlation coefficient was used to express the correlation between quality of life scores, ABPI, type of intervention and age. The internal consistency based on correlations between items for each scale was assessed with Cronbach's alpha [36]. The recommended reliability standard for group-level comparisons is a reliability coefficient of 0.70, while comparisons between individuals demands a reliability coefficient of 0.90 [25].

Table 2 Comparable domains between the Nottingham Health Profile (NHP) and the Short Form 36 (SF-36) and number of items.
Full size table

The Wilcoxon Signed Ranks test was used to detect the responsiveness of within-subjects changes over time, before and one year after revascularization, in patients with IC and CLI. Data analysis was performed for overall comparisons using the statistical package SPSS 11.0 and a P value of <.05 was taken as statistically significant.

Results

Forty-eight (53.3%) patients had IC of whom 26 (54%) were men. The remaining 42 (46.7%) suffered from CLI and 22 (52%) of them were men. There was a significant difference in age between the two groups with a mean age of 67 and 71 respectively (Table 1). One year postoperatively, sixty-six (73%) patients (38 with IC and 28 with CLI) remained in the study and secondary reconstructions were made on 7 (10%) patients during the follow-up. There were no significant differences at baseline in sex, age, ABPI and quality of life scores between the drop-out patients and the patients who completed the study. Further, there were no significant differences between the drop-outs and the remaining patients regarding the method of treatment or severity of ischaemia.

Analyses between the comparable domains showed that the NHP scores were more skewed than the SF-36 scores, especially in emotional reactions, energy and social isolation (Figure 1). The "floor effect", the proportion of individuals having the lowest possible scores (SF-36 = 0, NHP = 100), was larger for the NHP scale in energy one year (19.7%) after revascularization than for the SF-36. The "ceiling effect", the proportion of individuals having the best possible scores (SF-36 = 100, NHP = 0), was also larger for the NHP scale in emotional reactions (50.0%), energy (42.4%) and social isolation (71.2%) one year after revascularization (Table 3).

Figure 1
figure 1

Frequency distribution of scores on the NHP (left side) and comparable dimensions on the SF-36 (right side) one year after revascularization. NHP scores had 100 subtracted for consistency with SF-36

Full size image
Table 3 Reliability and "floor" and "ceiling" effects in comparable NHP and SF-36 scales before and one year after revascularization.
Full size table

Validity

The average convergent validity coefficients exceeded 0.5 one year postoperatively except for physical mobility and physical functioning (r = -0.46) and for social isolation and social functioning (r = -0.32), indicating a considerable convergence of the SF-36 and NHP (Table 4). One year postoperatively significant correlations between ABPI and physical functioning (r = 0.29) (SF-36), physical mobility (r = 0.42) and pain (r = 0.42) (NHP) were found. The severity of the ischaemia had a significant influence in the NHP-measured domain of pain (P < .003) and physical mobility (P < .03), indicating lower quality of life scores in patients with critical ischaemia. In the ability to discriminate between levels of ischaemia in the other comparable quality of life domains, no significant differences were found (Table 5).

Table 4 Multitrait-Multimethod matrix of correlation coefficient for the comparable scales of the SF-36 and NHP in patients with varying degrees of lower limb ischaemia one year after revascularization.
Full size table
Table 5 Differences in comparable domains in the NHP and the SF-36 between claudicants and patients with critical ischaemia before revascularization.
Full size table

Internal consistency

Physical functioning (α = 0.82), mental health (α = 0.76) and vitality (α = 0.70) for the SF-36 and pain (α = 0.71), emotional reactions (α = 0.76) and energy (α = 0.71) for the NHP scale were reliable, with coefficients >0.70 before revascularization. For the SF-36, all of the comparable domains except for social functioning (α = 0.64) exceeded the Cronbach's alpha value of 0.8 at the one-year follow-up. For the NHP the internal consistency coefficient was less than 0.8 but still exceeded 0.70 (Table 3).

Responsiveness

The NHP scale and SF-36 were not equally good at detecting within-patient changes over time. In patients with IC the SF-36 scale showed significant improvements in bodily pain (P < .01) and in physical functioning (P < .001) and for the patients with CLI there were significant improvements in bodily pain (P < .004) at the one-year follow-up (Figure 2). The NHP scale showed no significant improvements in patients with IC, while in patients with CLI, significant improvements in pain (P < .001) and physical mobility (P < .03) were found (Figure 3).

Figure 2
figure 2

Changes in median score for the SF-36 in claudicants and critical ischaemia patients before and one year after revascularization. Tested by Wilcoxon Signed Ranks test. A higher score indicates fewer perceived problems. BP = Bodily pain, PF = Physical functioning, MH = Mental health, VT = Vitality and SF = Social functioning.

Full size image
Figure 3
figure 3

Changes in median score for the NHP in claudicants and critical ischaemia patients before and one year after revascularization. Tested by Wilcoxon Signed Ranks test. A higher score indicates more perceived problems. P = Pain, PM = Physical mobility, EM = Emotional reactions, E = Energy and SO = Social isolation.

Full size image

Discussion

The result showed that the SF-36 was less skewed and more homogeneous with lower "floor" and "ceiling" effects than the NHP. A considerable convergence in three of the five comparable domains one year postoperatively indicates an average convergent validity. The SF-36 showed a higher internal consistency except for social functioning one-year postoperatively and was more responsive in detecting changes over time in the IC group. The NHP was more sensitive in discriminating among levels of ischaemia regarding pain and more able to detect changes in the CLI group.

The attrition or loss of subjects (27%) in this study could have affected the outcome. Further analysis showed that there were no significant differences in quality of life, sex, age, method of treatment and severity of disease between the attrition group and those who completed the study. The fact that the NHP and SF-36 differ in their nature and content may limit the study design. Therefore the analyses in this study focused only on the comparable domains of the two instruments, including the basic domains of physical, mental and social health identified by the WHOQOL group [7]. A suitable quality of life instrument for patients with chronic lower limb ischaemia should not only be valid, reliable and responsive but also simple for elderly people to understand and complete. In this study there was no difference in response rate between the two instruments and both seemed to be user-friendly and took about 5–10 minutes to complete. The findings strengthen earlier results suggesting that both scales are practical and acceptable to use in elderly patients [37, 38].

A generic quality of life instrument, designed for a variety of populations and measuring a comprehensive set of health concepts, is likely to have problems with "ceiling" and "floor" effects [24]. In this study the NHP showed higher "ceiling" effects in all dimensions than the SF-36. There were minor "floor" effects in both the NHP and SF-36, indicating the lowest possible quality of life. This is in accordance to Klevsgård et al, [1] who also showed higher "ceiling" effects in social isolation, emotional reactions and energy for the NHP than the SF-36. Other studies have also reported fewer "ceiling" and "floor" effects in the SF-36 than in the NHP in patients with chronic obstructive pulmonary disease [39] and after a myocardial infarction [37]. The advantage of the SF-36 may be due to its use of a Likert-type response format with a number of possible different scores and its ability to detect positive as well negative states of health, whereas the NHP items are dichotomous and state more extreme ends of ill health [39]. This could mean that a patient with acceptable initial scores fails to improve even if the improvement is obvious. Furthermore, a false negative response will be more likely when a patient perceives to having perfect functioning on a measure that only assesses severe dysfunction. The result confirms the importance of finding a quality of life instrument that does have a spectrum of dimensions which match the patients with chronic lower limb ischaemia related to the presence of numerous and often severe comorbid conditions.

In this study the internal consistency was not as high as desirable for any of the instruments before revascularization, but both instruments exceeded the minimum internal consistency value of 0.7, except for social functioning in the SF-36 one year postoperatively. The SF-36, however, had considerably higher α-values in all other dimensions. Several studies have previously reported that the SF-36 has higher Cronbach's α values than the NHP, but the domains in which the highest and lowest values were estimated differ [37–40]. The findings suggest that it is not only the magnitude of the correlation among items, but also the number of items in the scale that affects the internal consistency. For example, the domains of pain and social functioning in the NHP contain 8 and 5 items respectively, while bodily pain and social functioning in the SF-36 contain only 2 items. This is further strengthened by the fact that both the scales were not sensitive enough to identify significant within-patients changes in social isolation and social functioning. Another explanation could be that the patients were a more homogeneous group before treatment, with similar problems which affected the quality of life, but one year postoperatively they have become more heterogeneous and represent different states of recovery [13]. Both instruments meet the reliability standards for group-level application in most respects, although none of them achieved the degree of reliability that be would be desirable in individual-based assessment.

The result in this study showed significant convergent correlation coefficients between scores of the comparable dimensions except for physical activity and social activity, indicating a considerable convergence of the NHP and SF-36 scale. Prieto et al [39] and Meyer-Rosberg et al [40] demonstrated similar results with an average convergent validity. Thus the NHP and SF-36 are relatively equal in the validity and corroborate that the subscales probably reflect similar impacts of chronic lower limb ischaemia. However, social isolation in the NHP showed a higher correlation with mental health in the SF-36 and might measure more psychological aspects of social life, whilst social functioning in the SF-36 tends to assess social activities according to the higher correlation with energy in NHP. Similarly the physical functioning in the SF-36 showed a higher correlation with energy and may reflect physical activities of daily living rather than physical mobility. This suggests that the SF-36 and NHP measure different aspects of physical and social activities.

Validity in terms of the instruments' relative ability to discriminate among different levels of the ischaemia could only demonstrate that patients with CLI had significantly more problems with pain and physical mobility before treatment than patients with IC measured by the NHP. Klevsgård et al [1] showed similar results, that the NHP was more sensitive in discriminating deterioration in pain and physical mobility than the SF-36. In contrast, Brown et al [37] demonstrated that the SF-36 was more sensitive than the NHP for identifying people still troubled with angina or breathlessness after a myocardial infarction. Despite the lack of significant differences between patients with IC and patients with CLI, the NHP scale tends to be more sensitive in explaining the quality of life in this group of patients with regard to the dimension of pain and physical mobility. The important issue thus is to consider how well the measurement method explains health-related phenomena which are significant for the particular targeted disease or group of patients.

The SF-36 was the more responsive instrument in detecting changes in quality of life over time in patients with IC, including bodily pain and physical functioning one year postoperatively. However, in patients with CLI, the NHP was the more responsive instrument, with significant changes in pain and physical mobility, while the SF-36 showed a significant change only in bodily pain. Falcoz et al [38] also demonstrated that the SF-36 was more responsive than the NHP in detecting changes five weeks after cardiac surgery. In contrast, Klevsgård et al [1] showed that the NHP was more responsive in patients with chronic lower limb ischaemia one month after revascularization. The result of the present study supports the TransAtlantic Inter-Society Consensus (TASC) [12] recommendation that SF-36 should be used as a generic health outcome measure in patients with chronic lower limb ischaemia. It seems to be more sensitive for detecting changes in quality of life than the NHP in patients with IC. In the group of CLI patients who have more problems with mobility and pain, however, it is harder to evaluate whether the one questionnaire is superior to the other, the NHP could be a preferable instrument in this group of patients.

Conclusion

The findings indicate that both the SF-36 and the NHP have acceptable degrees of reliability for group-level comparisons, convergent and construct validity one year after revascularization. Nevertheless, the SF-36 seems generally to have more superior psychometric properties and was more suitable than the NHP for evaluating quality of life in patients with intermittent claudication. The NHP, however, discriminated better among severity of ischaemia and was more responsive in detecting changes over time in patients with critical leg ischaemia.