Influence of sociodemographic and socioeconomic features on treatment outcome in RCTs versus daily psychiatric practice
Sociodemographic and socioeconomic characteristics of participants in antidepressant and psychotherapy efficacy trials (AETs and PETs) for major depressive disorder (MDD) may limit the generalizability of the results. We compared trial participants with daily practice patients. We subsequently assessed the influence of socio-demographic and socioeconomic status on treatment outcome in daily practice.
Data on daily practice patients were derived through routine outcome monitoring (ROM). We included 626 patients with MDD according to the MINIplus. Distributions of age, gender, race, marital status and employment status were compared with participants in 63 selected AETs and PETs. Influence of these features on treatment outcome was explored through multivariate regression analysis.
Trial participants were older, more often male (diff. 4 %, p = 0.05), white (diff. 4 %, p < 0.001) and not married (diff. 7 %, p = 0.003). Although significant, most differences were relatively small. However, the difference in employment status was striking: 34 % of the ROM patients were currently working versus 68 % of the trial participants (diff. 34 %, p < 0.001). Being employed contributed to a positive treatment outcome: OR 1.8 for response [50 % reduction of Montgomery Asberg Rating Scale for Depression (MADRS)], OR 1.9 for remission (MADRS ≤10).
Employment status should be taken into account while interpreting results from randomized controlled trials and as predictor of treatment success in daily practice.
KeywordsMajor depressive disorderRandomized controlled trialSociodemographic statusSocioeconomic statusPatient selection
International guidelines on treatment of major depressive disorder (MDD) are based on evidence retrieved from scientific research. Meta-analyses and large randomized controlled trials (RCTs) are considered to give the most reliable evidence on treatment outcome for most therapies. RCTs use very strict procedures in design, patient selection, randomization, and methodology to enhance internal validity, i.e. the ability to adequately determine the efficacy of an intervention versus a control condition. While doing so, RCTs diminish their external validity, i.e. the generalizability to routine daily practice. Therefore, many clinicians and researchers have stated that the practical value of RCTs as source of evidence for treatment efficacy in daily practice is limited [1–7].
In previous research , many differences which might influence treatment outcome between trial methodology and daily practice have been described. One of the important domains that influence external validity is the selection of participants. The methodology of recruitment, the use of eligibility criteria, the (unintentional) use of criteria beyond eligibility (e.g., by recruitment only of patients with medical insurance, or in a certain area), the use of run-in periods and enrichment strategies, all lead to the exclusion of possible non-responders, and may lead to selection bias in the research population. For MDD, most research has focused on the influence of eligibility criteria on external validity. Several researchers [8–11] have shown that only a minority of daily practice patients would be eligible for RCTs. The influence of these eligibility criteria on treatment outcome is not yet clear: Wisniewski and colleagues  found that eligible patients had better treatment outcome than non-eligible patients, on the other hand, in our own research , we found no differences in treatment outcome between eligible and non-eligible patients. However, the use of eligibility criteria is only one aspect of patient selection. Even if eligibility criteria are met, recruited RCT participants differ from non-recruited participants in terms of age, sex, race, educational status, social class and place of residence . In previous research, we found a great variety in the reporting of sociodemographic and socioeconomic status (SES) features in major depression trials. SES factors are known to influence treatment outcome in MDD [12–22]. Differences in SES of participants might be one of the explanations for the differences between treatment outcome in RCTs and in pragmatic trials. However, although much effort is put in approaching daily practice as much as possible in pragmatic trials like STAR*D, they still have several properties of an RCT and selection bias by recruitment is not completely ruled out .
In the current project, we first compared the SES of participants in psychotherapy efficacy trials (PETs) and antidepressants efficacy trials (AETs) with the SES of a daily practice population of patients suffering from MDD. Subsequently, we explored the influence of the SES on treatment outcome in daily practice to assess whether possible SES differences between RCT participants and daily practice patients contribute to differences in treatment outcome between RCTs and routine practice. Comprehensive data on patients’ characteristics, treatment modalities and treatment outcome were available through an extensive routine outcome monitoring system (ROM) and extensive chart review. ROM provides outcome data on treatments without the properties of clinical trial but with strong methodological features. Therefore, ROM is a qualified instrument to obtain insight in “real life daily practice” .
Aims of the study
We investigate the differences in sociodemographic and socioeconomic features between “real life” patients suffering from MDD and trial participants. Furthermore, we explore the influence of sociodemographic and socioeconomic features on treatment outcome. This study gives insight on the extent of generalizability of results from RCTs to daily practice as well as the influence of sociodemographic and socioeconomic status on treatment outcome in daily practice.
Selection of RCTs and definitions of SES features in RCTs
Age age at baseline assessment calculated in years.
Gender percentage females in the population
Race percentage white patients in the population.
Marital status percentage of married/cohabitating patients.
Employment status percentage of patients who have a paid job
Patient selection and definitions of SES features in the ROM population
In the Netherlands, health insurance for all citizens is regulated by the government. Therefore, (mental) health care is easily accessible and not restricted by the financial means of individual patients. The Dutch health care system is organized in a stepped care manner and described in treatment-guidelines. First, patients visit their general practitioner (GP). In case of depression, the treatment guidelines for GPs recommend that patients with mild depression are treated by their GP with lifestyle advices. If the depression is moderate (clinical judgment), the GP can decide to prescribe antidepressants. Reasons to refer patients to a regional mental health provider (RMHP) are preference of patients for psychotherapy (not provided by GPs); severe depression (clinical judgment); presence of comorbid disorders, complex situation of the patient due to physical health problems, social problems, etc.; duration of the depression; and lack of result of antidepressant therapy. After baseline assessment and a clinical interview at our RMHP, patients suffering from major depression are offered to choose between psychotherapy and antidepressant therapy (if the severity is not too high, judged by the interviewing clinician in combination with results from ROM). When patients are already taking antidepressants from their GP, if needed the dose is optimized, and subsequently patients are offered to switch to another antidepressant or psychotherapy is added to the antidepressant therapy.
In 2002, the RMHP Rivierduinen (service area with 1.1 million inhabitants), in collaboration with the University Medical Hospital Leiden, implemented a comprehensive system of ROM and evidence-based, stepped care protocols. In ROM, all patients referred to the RMHP for treatment of a mood, anxiety or somatoform disorder have an extensive baseline assessment. Treatment progress is then assessed at intervals of 3–4 months and before starting a new treatment step. The baseline assessment comprises a standardized diagnostic interview (Mini International Neuropsychiatric Interview Plus , the collection of socio-demographic and socioeconomic data, the administration of observer-rated scales and self report questionnaires, and general measures of health and quality of life. All patients with sufficient mastery of the Dutch language who are able to complete computerized and written questionnaires are eligible for ROM. After baseline assessment and a clinical interview, patients suffering from major depression are offered to choose between psychotherapy and antidepressant therapy. When ROM data are used for research purposes, these are provided to researchers in an anonymous form, as dictated by the Psychiatric Academic Registration Leiden (PAREL) regulation. This procedure has been approved by the Medical Ethical Committee of the University Medical Hospital Leiden. The design of ROM has been reported previously .
Patients had a DSM-IV diagnosis of a current MDD as established by the Mini International Neuropsychiatric Interview.
Patients had at least one follow-up assessment in ROM.
Patients were not suspected of bipolarity/psychotic features by their clinician.
Patients were not admitted to an inpatient clinic during follow-up.
Patients did not have a MADRS-score ≤10 at baseline assessment
The time-span between baseline and follow-up assessment was not too short (less than 4 weeks) or too long (more than 1 year) to provide reliable information.
We conducted an extensive chart review to obtain information on the type of offered treatment. We examined possible selection bias due to our inclusion criteria by comparing the baseline severity of the depression and the SES features of the selected patients to the characteristics of patients who had no follow-up assessments in ROM. We compared the selected patients with all patients who suffered from major depression according to the MINIplus (including patients who dropped out of treatment after baseline assessment and patients of whom it was not clear if they dropped out or not). Subsequently, we compared the patients with follow-up to patients who did receive treatment, but had no follow-up assessments in ROM. Information on how many patients did receive treatment was obtained through an anonymized database from the medical administration of RMHP Rivierduinen.
Patients in our population received different types of treatment for their depression. Most of the treatments (84 %) were in line with international treatment guidelines: antidepressants (19 % with additional social supportive therapy/counseling), individual psychotherapy (mostly cognitive behavioral therapy or interpersonal therapy) or combination therapy of pharmacotherapy and cognitive behavioral therapy or interpersonal therapy. Sixteen percent of the patients received treatment other than antidepressants, individual psychotherapy or combination treatment. In line with RCTs, we defined treatment response as a 50 % reduction of symptoms on the Montgomery Asberg Rating Scale for Depression (MADRS, ) and remission as MADRS ≤10.
In ROM, extensive information on socio-demographic and socioeconomic status is gathered. Age and gender are recorded. Ethnicity is primarily assessed by the recording of patients’ and their parents’ countries of birth. Consistent with most RCTs, we considered patients who were born (or whose parents were born) in Suriname, Antilles, Turkey and Morocco as non-white. Two-third of the patients from other non-Dutch origins was non-white. We considered patients who were single, living-apart-together, divorced or widowed as “not married/cohabitating”.
Like other member states of the European Union, The Netherlands have a social security system which provides sickness benefit to people who are temporarily not able to work and disability benefit to those who are not able to work anymore. In the USA, the provision of sickness or disability benefit depends upon the insurance policies. Other countries provide no sickness or disability benefit or use their own system. Clearly, there is an important difference in financial status between receiving a sickness or disability benefit or not. It is unknown whether in RCTs, patients who receive sickness benefit are considered as having a paid job or not. Therefore, to allow comparison with RCTs, we defined employment status in ROM with two separate variables: employment status I and II.
For employment status I, patients who reported having a paid job at baseline assessment are considered as having “paid work”. In this definition, “paid work” means that patients are working at the time of baseline assessment. Patients on sickness or disability benefit, unemployed, student or housewife were considered as having “no paid work”. For employment status II, patients who receive sickness or disability benefit were classified as having “paid work” while unemployed patients, students and housewives were still considered as having “no paid work”. In this definition, having “paid work” means that patients receive an income out of a job, but that a substantial part of these patients (53 % of the patients classified as having paid work) is not working but on sickness benefit.
We first compared the included sample from our ROM population with the lost-to-follow-up in ROM patients using Chi-square tests and independent sample t tests. Subsequently, the SES features of RCT participants were compared to those of our ROM population by comparison of proportions of two independent groups . The selected RCTs did not report enough information on the characteristic “Age” to allow a comparison between RCT population and our population.
Finally, we examined the influence of SES features on outcome in routine clinical practice. In our sample, there were missing values for the following variables: type of offered treatment (n = 28), and socioeconomic characteristics (n = 82). Comparison of complete cases and cases with missing data showed differences on several variables such as age, gender and treatment outcome. In such instances, complete case analysis may yield biased estimates . Therefore, the multivariate imputation by chained equations (MICE ) method was used to estimate missing values for type of offered treatment, and socioeconomic characteristics. The influence of the sociodemographic (age, gender, race) and socioeconomic (marital status, employment status) on treatment outcome was computed by logistic regression after MICE. Univariate regression models (unadjusted analysis) were used to explore the influence of the individual SES features on treatment outcome. A multivariate regression model (adjusted analysis) was used to explore the joint influence of all SES features on treatment outcome. In this model, type of offered treatment and educational level was entered as possible confounders. In our previous research  on the same patient population, we showed that typical clinical features of MDD patients in daily practice, namely comorbid Axis I disorders, substance abuse, and suicidality, which are often used as exclusion criteria in RCTs, were not associated with treatment outcome in clinical practice. A low baseline severity, however, was associated with a less favorable treatment outcome. Therefore, we entered baseline severity of the depression as another possible confounder in our multivariate regression model.
Odds ratio (OR) and their confidence-intervals were computed using the robust standard error. Statistical analyses were performed with SPSS 16.0 and STATA10.0.
Daily practice (ROM) population
Baseline characteristics of ROM patients
n = 626
Mean age in years (standard deviation)
Presence of comorbid other Axis I disorder
69.9 % (majority anxiety disorder)
Mean MADRS at baseline
Type of offered treatmentb + duration of treatment in weeks (95 % CI)c
AD + IP
AD + SST
Duration of treatment in weeks (95 % CI)c
AD + IP
AD + SST
Parental birth country
Paid work Ia
Paid work IIa
The selected patient group was similar to the non-selected patients in baseline severity of the depression and most SES features. The selected patients were slightly older (39.2 vs. 37.3 years) and more often married/cohabitating (52.6 vs. 46.9 %) than the patients who were not selected (see Table 1, supplementary material). Of the patients who were lost-to-follow-up and, therefore, not selected, 63 % received treatment in our outpatient clinics for mood-, anxiety, and somatoform disorders after baseline assessment (37 % was referred to other specialized departments or did drop out of treatment). We also compared the selected patients with the patients who did receive treatment in our outpatient clinics but had no follow-up in ROM and found them to be very similar (see Table 2, supplementary material). Therefore, we consider our selected patient sample as fairly representative for patients who receive outpatient-treatment for depressive disorders.
RCT participants versus daily practice (ROM) population
SES features of RCT participants versus ROM patients
RCT (%, n)
AET (%, n)
PET (%, n)
ROMb (%, n)
Difference in proportion RCT versus ROM (95 % CI; p value)
Difference in proportion AET versus ROM (95 % CI, p value)
Difference in proportion PET versus ROM (95 % CI, p value)
Age in yearsa
39 (SD 11.4)
Gender (% female)
62 % (5,880)
61 % (5,269)
73 % (611)
66 % (415)
−4 % (−8 to 0 %; 0.05)
−5 % (−9 to −11 %; 0.016)
7 % (2–11 %; 0.008)
Race (% white)
89 % (5,820)
89 % (5,400)
85 % (420)
84 % (457)
4 % (2–8 %; <0.001)
4 % (2–8 %; <0.001)
−1 % (−3 to −6 %; NS)
Marital status (% married)
45 % (972)
46 % (728)
42 % (244)
52 % (284)
−7 % (−12 to −3 %; 0.003)
−10 % (−10 to −1 %; 0.017)
−18 % (−24 to −12 %; <0.001)
Employment Ic (% paid work)
68 % (1,107)
68 % (905)
66 % (202)
34 % (187)
34 % (29–38 %; p < 0.001)
34 % (30–39 %; <0.001)
32 % (25–38 %; <0.001)
Employment II (% paid work or sickness benefit)
68 % (1,107)
68 % (905)
66 % (202)
74 % (402)
−6 % (−10 to −1 %; 0.013)
−5 % (−10 to −1 %; 0.027)
−8 % (−14 to −1 %; 0.019)
Influence of SES features on treatment outcome in the daily practice (ROM) population
Influence of individual SES features on treatment outcome: unadjusted analysis
Remission (MADRS ≤10)
Response (50 % reduction MADRS)
OR 0.99 (95 % CI 0.97–1.00; RSE 0.008; p = 0.13)
OR 0.99 (95 % CI 1.0–1.0; RSE 0.01; p = 0.10)
OR 0.97 (95 % CI 0.65–1.44; RSE 0.20; p = 0.87)
OR 1.09 (95 % CI 0.8–1.6; RSE 0.21; p = 0.66)
OR 0.48 (95 % CI 0.27–1.1; RSE 0.41; p = 0.08)
OR 0.51 (95 % CI 0.2–1.1; RSE 0.41; p = 0.08)
OR 1.32 (95 % CI 0.9–2.0; RSE 0.02; p = 0.17)
OR 1.32 (95 % CI 0.9–2.0; RSE 0.20; p = 0.17)
OR 2.30 (95 % CI 1.6–3.4; RSE 0.20; p < 0.001)
OR 1.82 (95 % CI 1.3–2.6; RSE 0.34; p = 0.002)
OR 1.12 (95 % CI 0.7–1.1; RSE 0.23; p = 0.61)
OR 1.18 (95 % CI 0.8–1.8; RSE 0.21; p = 0.42)
Joint influence of SES features on treatment outcome: adjusted analysis
Remission (MADRS ≤10)
Response (50 % reduction of MADRS)
OR 0.99 (95 % CI 0.97–1.00; RSE 0.01; p = 0.21)
OR 0.99 (95 % CI 0.97–1.00; RSE 0.01; p = 0.15)
OR 1.07 (95 % CI 0.69–1.67; RSE 0.23; p = 0.75)
OR 1.10 (95 % CI 0.73–1.59; RSE 0.20; p = 0.65)
OR 0.67 (95 % CI 0.32–1.40; RSE 0.38; p = 0.29)
OR 0.63 (95 % CI 0.33–1.19; RSE 0.32; p = 0.15)
OR 1.33 (95 % CI 0.85–2.07; RSE 0.23; p = 0.21)
OR 1.10 (95 % CI 0.73–1.67; RSE 0.21; p = 0.64)
OR 1.89 (95 % CI 1.24–2.81; RSE 0.21; p = 0.003
OR 1.76 (95 % CI 1.19–2.60; RSE 0.20; p = 0.004)
In the current project, we compared sociodemographic and socioeconomic features of participants in RCTs for depression with those of patients in daily practice. Participants in RCTs for major depression differed from daily practice patients with respect to age, male–female ratio, ethnicity and marital status, but those differences were relatively small (less than 7 % difference on all features). One striking difference between RCT participants and daily practice patients is their employment status. Only 34 % of the daily practice patients had a paid job (patients who were students, housewives, unemployed or on sickness or disability benefit were considered as having “no paid work”) at time of assessment, while 68 % of the RCT participants had a paid job at time of the trial participation. In routine clinical practice, having a paid job contributed positively to treatment outcome, both on remission (OR 1.85) and response (OR 1.76) on the MADRS. Age, gender, race and marital status did not contribute to treatment outcome in routine clinical practice.
Previous research found unequivocal results on the associations between sociodemographic and socioeconomic features and treatment outcome: in general, increased age seemed to be associated with worse outcome in pharmacotherapy [13–19], but not in psychotherapy [20–22], and one study found that men had better treatment outcome in psychotherapy than women . Patients who were married benefitted more from both pharmacotherapy and psychotherapy [18–21, 32–35].
The difference in employment status between RCT participants and daily practice patients was striking. In a recent study, it was found that the quality of mental health is associated with lesser years of unemployment . In previous research, in non-routine practice settings, on the relation between employment and treatment success, it was found that employment status improved outcome for antidepressants, but not for psychotherapy [18–21, 32–35]. In the univariate analyses, we found that “currently working” (employment status I) is associated with better treatment outcome, while “receiving income out of a job” (employment status II) seemed to not be associated with treatment success. It is, however, likely that other clinical factors, such as baseline severity, could have acted as a confounder in these univariate analyses, since patients with a more severe depression might drop out from work earlier. Therefore, in our multivariate analyses, we adjusted for baseline severity of the depression. Still, having paid work (defined as currently working) almost doubled the probability of response or remission. From clinical experience, one might expect that the social status and the daily structure, routine and distraction provided by paid work will be factors that contribute to the positive response on depression treatment. Alternatively, it may be that patients who remain working despite the opportunity of sickness benefit have personality traits that increase the likelihood of positive treatment outcome, like for example optimism. Future research is needed to explore these different possibilities.
Of course, besides differences in SES features, there are many other differences between RCT participants and daily practice patients, caused by the use of stringent exclusion criteria in RCTs. For instance, the presence of comorbid Axis I disorders is used as an exclusion criterion in more than 75 % of the antidepressant efficacy trials  and in 25–50 % of the psychotherapy efficacy trials . In previous research, we found that only 17–24 % of our patients would be eligible for participation in antidepressant efficacy trials . Most of our patients would have been excluded because of the presence of comorbid Axis I disorders and not meeting a baseline severity threshold. However, in our previous research , we also found that in daily practice the exclusion of patients with comorbid Axis I disorders does not influence the treatment result for depression. One can, therefore, argue that it is unlikely that co morbidity acted as a major confounder in our analysis of the influence of SES features on treatment outcome for depression. It is, however, very well possible that having a comorbid disorder could lead to earlier withdrawing from work. Since comorbid disorders occur frequently in depressed patients, more specific research on the association between drop out from work and co morbidity is recommended.
We consider the large sample of well-characterized, routinely monitored patients from routine clinical practice to be the major strength of our study. To our knowledge, no previous research has reported on the influence of sociodemographic and socioeconomic features on treatment outcome in major depression in routine daily clinical practice.
There are also some limitations to consider. In our model of sociodemographic and socioeconomic features of RCT participants, we converted all information on these features reported in the individual RCTs into dichotomous or trichotomous variables. This conversion leads to loss of information, but was necessary for comparability with published data . Another limiting factor might be that in the model AETs outnumbered PETs, both in number of included trials and number of included patients per trial. Thus, the features of the RCT participants as a group were dominated by the features of AET participants . There was a considerable loss to follow-up in our sample. Nevertheless, we demonstrated that our patient selection was fairly representative for the daily practice patients who received treatment. It was not possible to analyze in our study which aspect of “having a paid job” contributed positively to a favorable treatment outcome. Furthermore, no information on the duration and the number of episodes of depression was available in our ROM data. Chronic depression is known to have a less favorable prognosis and it is possible that especially patients who suffer from chronic depression have to resign from work, which may have confounded our results. Finally, since sociodemographic and socioeconomic features are closely related to the culture in the country of origin in which research takes place, cultural aspects might have somewhat limited the generalizability of our results. It is unknown that to what extent the Dutch health care system (and that of several other, mostly European, countries) may limit the generalizability of our results to other countries that do not have an extensive social security system. Furthermore, we have to take into account that we compared a western psychiatric population with an RCT population derived from western countries (Europe and USA). Since our aim was to compare RCT participants (who are most often from western countries) to our daily practice patients, the fact that all RCT participants were from western countries did not limit our research. However, our results are probably not generalizable to countries outside the western world. Future research on the generalizability of results from RCTs to psychiatric patients in other parts of the world is highly recommended.
In conclusion, we found that RCT participants and daily practice patients only differed slightly on most sociodemographic and socioeconomic features, with the exception of having a paid job. Having a paid job contributed significantly to treatment success in daily practice and should be taken into account both while interpreting results from RCTs as well as in depression treatment in daily practice. Further research is recommended to explore which specific aspects of employment status contribute to better treatment outcome.
Mrs. R. van der Lem (first author) has received a research grant from ZonMW (Grant Number 100-002-026), an independent research fund from the Dutch Government.
Conflict of interest
The authors have no financial or personal conflict of interest.