Background

Care transition, or the movement of patients from one healthcare setting to another, is a concern of many healthcare systems because it affects the overall quality of patients’ care. It has been well documented that inadequate discharge planning and/or patients’ poor self-management following discharge from hospitals increased avoidable rehospitalisation and the use of emergency room [1–5]. It is not surprising that leading healthcare organisations for quality and safety such as the Joint Commission and The National Quality Forum (USA) are emphasising high-quality care transition [6–12].

An important element in the measurement of quality of transition is patients’ experience [13, 14], as patients and caregivers are often the only common link across various care providing settings. In 2005, the American Institute of Medicine recommended research and development of measures to address patients’ experience during care transitions [15].

The Care Transition Measure (CTM-15) is, to the best of our knowledge, the only available measure of quality of care during transition, from the patients’ perspective [16–19]. It was designed to measure the overall care transition experience and not merely the hospital discharge phase [20]. Studies have shown that the CTM-15 is a valuable tool for health system performance evaluation by measuring the quality of care delivered across settings [17, 20]. Recently, a shorter version of the CTM-15 comprising 3 of the original CTM items (CTM-3) [16, 21] was endorsed by the National Quality Forum [22]. The CTM-3 closely approximates the CTM-15 in measuring the quality of care transition and has an advantage of minimising cost and response burden for assessing quality of care [16]. Both the CTM-15 and CTM-3, however, have been validated in patient populations in the US and Israel only [23].

The purpose of our study was to test the psychometric properties of the CTM-15 and CTM-3 in Singapore, a multi-ethnic urban state in South-east Asia. The quality of Singapore’s healthcare services was regarded high by both the World Health Organization [24] and Singaporeans [25]. Although it is not in the form of national health insurance, Singapore’s healthcare system has successfully achieved the goal of providing good and affordable basic medical care to all Singaporeans. Nevertheless, as a fast aging country with high public hospitals bed occupancy, Singapore is also facing the challenge of growing concerns over quality of care transition [26, 27]. Therefore, there is a need to have a measure of care transition to assess the quality of care of patients in transition between care settings in Singapore. As the majority of the Singaporean population speaks either English or Chinese, we assessed both the English and Chinese versions of this instrument. We also compared the effects of three-point and five-point Likert response scales (currently used in the CTM) on the psychometric properties of the measures. This analysis was for maximising the usefulness of the instrument as fewer response choices pose less administrative and cognitive burden to respondents.

Methods

Study setting and sampling

Patients who were being discharged from two tertiary public hospitals in Singapore: National University Hospital (NUH) and Tan Tock Seng Hospital (TTSH) between January to June 2009 were recruited prospectively as subjects for this survey study. Inclusion criteria were inpatients aged 50 years and above, hospital care by disciplines of general medicine, general surgery, orthopedics, or geriatric medicine, and home residence upon discharge. At the time of this study, both hospitals had in place a care transition program that was implemented in collaboration with the Agency for Integrated Care (AIC) in June (NUH) and September (TTSH) 2008. AIC was a state funded national agency which coordinated referrals and care for people who require long term care. Dedicated care coordinators employed and trained by AIC and based at the respective hospitals coordinated discharge planning and care transition for patients from the hospitals to community settings. In order to increase the representativeness of the study sample, we recruited equal number of subjects from the two participating hospitals and those who were and were enrolled by the care coordination program. The National Healthcare Group Ethics Board, which had research ethics purview of both hospitals, approved the study.

Data collection

Before discharge, patients were informed about the survey and consent was taken. Patients were contacted by a research assistant 3 weeks after their discharge from the hospital for an interview over the phone between February to July 2009 using the CTM-15 questionnaire. Interviews were conducted in English or Chinese based on respondents’ preference. In case a patient was not contactable, too weak to be interviewed, or had language barrier, a caregiver who started to look after the patient before discharge, if available, was interviewed as a proxy of the patient. The survey also sought information about patients’ demographics, length of stay before discharge, visit to emergency department, non-elective rehospitalisation for the condition of index hospitalisation, and experience with care after discharge.

Instrument

The CTM-15 is a 15-item measure of perceived quality of care transition. The measure is based on a tested conceptual framework containing items relating to patients’ critical understanding, importance of preferences, management preparation and existence of a written and understandable care plan [20]. All questions use a five-point Likert response scale comprising ‘strongly disagree’, ‘disagree’, ‘neutral’, ‘agree’ and ‘strongly agree’. A single total score ranging from 0 to 100 can be calculated from the CTM-15, with higher scores indicating better care transition. In this study the English (US) version and its Chinese translation were used. The Chinese version was developed by a single forward translation of the English CTM conducted by one of the Chinese-speaking investigators of this study.

The CTM-3 comprises three of the CTM-15 items: (i) patients’ understanding of their medications; (ii) the extent to which patient preferences are taken into account in deciding what the patient’s health-care needs will be on discharge; and (iii) patients’ understanding of their responsibilities in managing their health [28]. Hence, the CTM-3 score can be calculated from the administration of the CTM-15. In this study, the CTM-3 score was transformed to the same range (0 to 100) of the CTM-15 score for the purposes of easy comparison.

A total of five statements were used to assess patients’ negative experience with care after discharge including (i) I felt like I really need to talk to a health professional but did not know whom to call; (ii) I was treated more like a number than a person; (iii) I went to a doctor or nurse who seemed to not understand my healthcare needs; (iv) I had difficulties getting the care I thought I needed; and (v) I found that people did not really care about me as a person. Those questions were used in a previous study for assessing the CTM-15 [17]. In the present study, answers to the five questions (yes = 0/no = 1) were used to calculate a total score, with higher scores indicating better experience.

Statistical analysis

Descriptive analysis was performed to describe the study sample by calculating frequency for categorical variables and mean (standard deviation) for continuous variables. CTM item and total scores were described using mean (standard deviation) for easy comparison, although those were not interval data.

The internal consistency reliability of the CTM was assessed by Cronbach’s alpha. An alpha value of ≥ 0.7 was considered satisfactory [29]. Construct validity was assessed by testing a priori hypotheses. We hypothesised that patients who visited emergency department (ED) or were rehospitalised for the index condition after discharge would have lower CTM scores than those who did or were not (known-groups validity). The mean CTM score for patients with and without ED visits and patients with and without rehospitalisation were tested using two-sample t-tests. We also hypothesised that the CTM score would be at least moderately correlated with the experience score (convergent validity). For testing the criterion validity of the CTM-3, we hypothesised that the CTM-15 and CTM-3 would be highly correlated. Correlation was assessed by Pearson’s correlation coefficient. A correlation coefficient < 0.35, 0.35 to 0.49, and 0.5 to 1.0 were considered weak, moderate, and strong, respectively [29].

The English and Chinese language versions of the CTM-15 and CTM-3 were assessed separately. We also examined the validity and reliability of the CTM-15 and CTM-3 based on the three-point Likert response scale. We did this analysis because a three-point response scale such as disagree/neutral/agree would be easier to both administer and answer. Previous studies suggested that the three-point response scale is preferable to the five-point response scale for use in health-status questionnaires for respondents who are illiterate [30] or with intellectual disability [31]. For testing a hypothetical three-point response scale, we assigned ‘strongly (dis)agree’ and ‘(dis)agree’ the same coding as if respondents had answered on a three-point response scale (i.e. disagree/neutral/agree).

In order to examine the latent constructs measured by the CTM-15 items, we performed exploratory factor analysis. Factors with an eigenvalue > 1.0 were consider important and retained in a Varimax rotation analysis. Items with a factor loading > 0.5 were used to name the resultant factors.

All analyses were performed with SPSS for Windows (version 13) and all statistical tests were 2-tailed with a significance level of 0.05.

Results

A total of 600 patients or their caregivers were interviewed. The overall participation rate was 58%. The main reason for refusal was lack of interest in the study. After excluding 21 respondents who were interviewed in Malay, 579 patients or their caregivers who were interviewed in English (n = 414) or Chinese (n = 165) with no missing data on the CTM-15 questions were included in data analysis. Table 1 summarises the basic demographic information of the participants of the study. It can be seen that the vast majority of respondents was proxies. English-speaking and Chinese-speaking respondents had similar characteristics except that higher proportion of respondents were patients themselves among those who were interviewed in Chinese (15.2%) than in English (3.9%) (p <0.01). Distribution of CTM item scores are displayed in Table 2. For both English and Chinese respondents, CTM-3 scores were lower than CTM-15 scores, and CTM scores based on the 5-point response scale were lower than those based on the 3-point response scale.

Table 1 Characteristics of respondents
Full size table
Table 2 Means and standard deviations of CTM item and total scores measured on 5-point and 3 point response scales
Full size table

CTM-15

The Cronbach’s alpha values of the CTM-15 scores bases on the five-point and three-point response scale were 0.87, 0.81 for the English version and 0.82, 0.85 for the Chinese version, respectively.

Among English-speaking respondents, patients who did not report ED visits for the index condition had higher mean CTM-15 scores than those who reported ED visits; similarly, patients who reported no rehospitalisation for the index condition had higher CTM-15 scores than those who reported rehospitalisation (Table 3). However, such differences in CTM-15 scores were not observed in the Chinese-speaking subgroup (Table 3). For example, the mean CTM-15 scores based on the 5-point scale were 60.9 and 63.2 (p > 0.05, two-sample test) for patients with and without ED visits, respectively. There was moderate correlation between the CTM-15 and the experience scale, regardless of survey language and response scale (correlation coefficient: 0.39 to 0.46, p < 0.001 for all, Table 3).

Table 3 Association of CTM score with ED visits, rehospitalisation, and experience with care
Full size table

The English and Chinese versions of the CTM-15 items measured on the five-point scale demonstrated a similar 4-factor structure. Based on item loadings, the four factors were general care plan, medication, agreement on care plan, and specific care instructions (Table 4). Factor analysis of the 15 CTM items based on the three-point scale showed similar factor structures (data not shown).

Table 4 Factor loadings of the CTM-15 items measured on the 5-point scale after Varimax rotation
Full size table

CTM-3

The Cronbach’s alpha value of the CTM-3 based on the five-point response scale and three-point response scale was 0.58 and 0.42 for the English version, and 0.63 and 0.57 for Chinese version, respectively. The correlation between the CTM-3 and CTM-15 was 0.89 and 0.87 for English and Chinese versions, respectively, when the five-point response scale was used; the correlation was 0.83 and 0.76 for the English and Chinese version, respectively, when they were scored on the three-point response scale.

Similar to the results for CTM-15, the CTM-3 scores discriminated well between patients with and without ED visits or rehospitalisation among English-speaking respondents; however, the CTM-3 scores were not different between patients reporting and not reporting ED visits or rehospitalisation in the Chinese-speaking group (Table 3). The correlation between the CTM-3 and experience scale was moderate for both language groups and response scales (correlation coefficient: 0.36 to 0.40, p < 0.001 for all, Table 3).

Discussion

We found that the psychometric measurement properties of the CTM were generally satisfactory, although variations existed between different versions of this instrument. Generally, the long form is more reliable than the short form and English version is more valid than the Chinese version. The English version of the CTM-15 is the most valid and reliable measure of care transition. It exhibited adequate internal consistency and was able to discriminate between patients with and without ED visits or rehospitalisation and converged with reported experience with care post-discharge. These results were consistent with those in previous studies [16, 17, 23]. Also, three of the four factors underlying the CTM items in our study were similar to those observed in the US and Israel populations [16, 23].

Most of participants in our study were not patients but their proxies. This was because most patients designated a family member to answer the interview questions for them. Reasons for using a proxy included caregivers knowing better than patients about care-related matters, patients too weak to be interviewed, and patients having language barriers. Generally, using proxies to assess quality of care is a limitation because proxies and patients themselves may have very different perception of the care during transition. Nevertheless, it is necessary to use proxies when patients are not well enough to be the assessors. Moreover, it makes good sense to ask for the opinions of family members when it comes to assessment of quality of care from care users’ perspective in Singapore. Singapore is a small city country dominated by Asian culture and Singaporeans are very involved in the medical and long term care of their family members. Family members, particularly spouses and children not only provide informal care, but influence or even make medical and long term care decisions for the patient and also contribute to the healthcare costs. Singapore’s unique healthcare system allows contribution of healthcare costs from family members using individuals’ compulsory savings, which strengthens family bonds. Hence, it is not unexpected that the majority of our respondents were family members (Table 1). For this reason, a quality of care measure would be relevant and more valuable in Singapore with proxies’ assessment. The most important contribution of our study is therefore the provision of evidence for the usefulness of CTM when it is administered to proxies. To the best of our knowledge, this is the first study that explored the extended application of this instrument to assessor other than patients themselves.

The Chinese CTM-15 and CTM-3 were similar to their English counterparts in psychometric properties but exhibited poor discriminatory power. There are several possible reasons for this result. First, the poor discriminatory power could be partly be attributed to sampling error. This is supported by the observation that less than thirty patients reported ED visits or rehospitalisation in the Chinese-speaking subgroup (Table 1). It is possible that those patients visited ED or were rehospitalised for reasons not related to poor care transition. Therefore, the Chinese CTM should be reevaluated in a larger sample of patients in the future. Second, it could also be attributed to respondents’ poor understanding of the CTM items. Although we did not collect information on education profile of the patients, Chinese-speaking patients generally had fewer years of formal education than English Speaking patients in Singapore [32, 33] as English has been the primary language in education. Since the statements used in the CTM questionnaire are relatively long, a larger proportion of Chinese-speaking respondents might not comprehend the statements fully, which in turn affected the discriminatory power of the CTM. Another possible reason for poor understanding is suboptimal wording of the Chinese CTM which was the product of a single forward translation of the English CTM. Certain Chinese CTM items might not be accurate or clear. Nevertheless, this may not be a serious issue; factor analysis of English and Chinese CTM items resulted in similar latent constructs, suggesting no dramatic difference in response patterns between the two language groups. Third, Chinese-speaking respondents may have idiosyncratic response styles due to culture reasons. A previous study found that a visual analog scale for self-assessing overall health was not discriminative between Singaporean diabetics in differing health conditions [34]. Exact reason for this phenomenon is unknown. Hence, future studies using qualitative research methods such as in-depth interview or focus group discussion are needed to find out how respondents understand and respond to the CTM items, which will hopefully help us find ways to improve the performance of the instrument among Chinese-speaking persons.

It is interesting that the CTM scores based on the three-point response scale performed similarly well as those based on the originally five-point response scale. That means if the interviewer had offered only three response options (i.e. disagree, neutral, and agree) in the interviews, the resultant CTM scores would not be less reliable or discriminative. This result is not surprising as three-point response scales were found to be superior to five-point scales in some populations [35]. This is a useful finding as it suggests that the CTM can be administered in a more economical and easy-to-answer way.

It was not unexpected that the CTM-3 demonstrated worse internal consistency than the CTM-15. Internal consistency increases with increasing number of items. Although the reliability of the CTM-3 was suboptimal, it was highly correlated with the CTM-15 scores and exhibited good construct validity in English-speaking respondents. Therefore, the CTM-3 can be used as a substitute for the CTM-15 when administration burden is a concern or in large-scale studies where large sample sizes can compensate its suboptimal reliability.

There are some limitations in the study. First, our findings have limited generalisability. Factors that limit generalisability include relatively low response rate, all patients aged 50 years or older, half of the study sample enrolled in a care coordination program, most respondents being proxies, and some proxies possibly not experiencing the entire care transition process. However, our study for the first time demonstrated that the CTM was valid and reliable in proxies. Second, that we administered the CTM at the time when ED visits and rehospitalisation had occurred may have introduced response bias to the study. That is, those who had experienced ED visits or rehospitalisation might ascribe those undesirable events to care transition, which in turn intensified their memory of negative experience during discharge including those experienced during the discharge process. Arguably, it may be more appropriate to administer the CTM soon right after the hospital discharge (with the discretion of the problem we discussed next) and obtain information about ED visits and rehospitalisation from administrative records or inquire at a later time so that the respondents’ answers to the CTM questions are more clearly disassociated from ED visits or readmission events. Third, although genuine, the reports from some older patients may not be reliable. A previous study suggested delirium, including periods of altered perception, is highly prevalent among the elderly recently discharged to a post-acute care facility from hospital [30]. However, this should not be a big concern as the vast majority of our respondents were their caregivers.

Conclusion

The care transition measure is a valid and reliable measure for quality of care transition in Singapore. Moreover, the care transition measure can be administered to proxies using a simpler response scale. The discriminatory power of the Chinese version of this instrument needs to be further tested in future studies.