Introduction

The introduction of clinical databases and computerized medical records has made it possible to combine data in order to choose the best treatment strategy for a specific person [7]. Such evaluation systems are potentially valuable in mental health services, particularly for patients who have been diagnosed with severe mental illness (SMI). SMI patients have a combination of psychiatric, somatic and social needs and thus require complex multidisciplinary care. Many patients diagnosed with schizophrenia or psychosis (around 75% of all SMI patients; see below) have residual symptoms, comorbid drug use and poor somatic health as well as needs in the areas of self-care, accommodation, daytime activities and social contact. These patients therefore require tailor-made rehabilitation strategies in order to bring about an enduring impact on outcome.

The cumulative needs for care monitor (CNCM) was developed in order to standardize and improve needs-based diagnosis in a defined geographical area in an European country (the Netherlands; area of South Limburg; population: 660,000). In the near future, the monitor will be extended to cover a larger geographical area. The system was developed to collect clinically relevant information on various patient outcomes and to provide systematic feedback on individual patient outcomes for perusal by professional carers. Data are stored in an anonymized database (the cumulative needs for care register; CNCR), enabling analyses at group level in order to address scientific hypotheses in the area of health services research and to generate management information on groups of patients with certain characteristics. It offers a unique opportunity to modify the mental health infrastructure on the basis of local evidence. The present paper describes the assessment methods and the research logistics of the CNCM.

Purpose of the CNCM

The CNCM is a tool to be used by clinicians in standard care. Therefore, instruments were selected on clinical relevance. The aim is to follow SMI patients over time and to provide standardized individual feedback to clinicians to support clinical decision-making. Each clinician who files an assessment form receives a feedback report within two working days. This is a personalized analysis of the course and development over time and in relation to a chosen reference group. Clinicians can use the feedback reports to design customized treatment plans. Clinicians instructed to regularly fill in the CNCM form (see procedure) are psychiatrists, psychologists, case managers (assertive community treatment, see below), nurses, social workers, the staff of sheltered housing projects and other mental health care staff (hereafter: ‘professional carers’). Although the instruments included in the CNCM are also suitable for routine outcome assessment (ROA), the CNCM is more than ROA, as it follows individual patients over time in various settings and provides systematic feedback to clinicians.

Development of the CNCM

The CNCM assessment form is a locally developed update of clinically relevant and internationally accepted reference assessment instruments. It was intensely piloted between 1998 and 2001 in part of the CNCM region. The interview form assesses, for example, the need for care, the severity of the symptoms and the quality of life (see below).

In 2002, mental health care services in the CNCM area adopted a variant of assertive community treatment (ACT), the aim of which was to deliver services that were tailored to the patients’ specific needs and strengths. The CNCM was used to monitor these. Teams working with the Dutch variant provide intensive or traditional ACT care as well as extensive care at lower levels of contact frequency and intensity. This ensures continuity of care and allows for a more natural match with changing treatment needs over the course of the illness [41, 42]. Caseloads comprise around 20–30 patients per ACT team member.

The monitor was extended and improved over the years, and new items were added when relevant topics were missed in the standard scales. For example, the prescription of medication and the receipt of psychosocial treatment were included. In 2004, the geographically defined area was extended. The scope of the CNCM was broadened from an individual patient evaluation tool to a regional health services research instrument. All services that provide care to SMI patients were and still are included. This increased the logistic complexity and workload. Therefore, helpdesk workers were appointed to act as liaison between clinicians and researchers. These workers coordinate and instruct the local professionals and facilitate data entry and clinical feedback using individualized patient reports.

Procedure

CNCM interviews are administered by a mental health professional involved in the treatment of the patient under the authority of a psychiatrist or a clinical psychologist. A manual that includes all CNCM instruments is available and interviewers are trained to administer CNCM forms during a basic training and yearly booster sessions in order to prevent drift and maintain reliability. These sessions use video case material that is coded by the trainees and discussed by the instructor in reference to a ‘golden standard’. Furthermore, a team of highly trained helpdesk employees appointed at the mental health care services in the three subregions can provide on-site supervision. Ideally, patients are assessed every year and with every major change in treatment or setting (e.g. hospitalization, start of a new treatment, discharge). In keeping with current legal requirements, patients are informed that anonymized routine clinical data are used for the purpose of regional and scientific analysis, and are given the chance to opt out; in such cases (which are very rare) their data are not used. This has no consequence for their treatment.

Measures

The CNCM interview uses a selection of state-of-the-art standards for assessments, namely the Camberwell Assessment of Need (CAN), the Brief Psychiatric Rating Scale (BPRS), the Global Assessment of Functioning Scale, split into its psychopathology (GAF-p) and impairment (GAF-i) components, a quality of life assessment and a single item on satisfaction with services (quality of care). In addition, custom-made generic assessments were developed for medication and treatment. Quality of life and quality of care are scored by the patient, the CAN combines the ratings from both patient and interviewer (see below), and all other instruments are scored by the interviewer. Finally, some background variables are registered: date of and reason for the interview, date of birth (age), gender, name of clinician in charge, treatment setting and institution.

Alternative choices are possible. Some research groups have relied on the HONOS [48, 49] to assess mental states and dysfunctional domains leading to needs for care. For most clinicians, however, the HONOS set of items was insufficient to assess support in clinical decision-making. In addition, the original CAN set was extended on request (see below). The reference alternative for the psychopathology ratings is the PANSS (Positive And Negative Symptom Scale) [19, 20]. This scale has more items than the BPRS (39 vs. 24 items) while the scope of the BPRS is larger (including mania items and e.g. suicidality). The traditional objection against the BPRS refers to the original BPRS that lacked the anchor scores, reducing reliability. The UCLA adaptation that we use has an extensive scoring manual [43] that substantially improves the reliability of the ratings. For the quality of life assessment, different options are available. In the set of small instruments the MANSA [31] is best known. Unfortunately, it contains some complex branching items (work vs. study related) that were highly confusing for our participants and patients. A simplified alternative was chosen that also included quality of care. The selected instruments for the CNCM are briefly discussed below.

Camberwell Assessment of Needs (CAN)

Need for care is assessed using the CAN [30], which is the core instrument of the CNCM. The CAN is a family of assessment instruments targeted at different treatment populations with SMI. The original CAN Short Assessment Schedule includes 22 items (Table 1) [33]. In general, all items can be scored 0, 1 or 2. If a domain is mastered independently, a score of 0 (=no problem) is assigned. A score of 1 is given if the domain is ‘covered’. In other words, the patient has a need but receives care to meet that need (met need). By whom the care is provided (mental health professionals or family) does not affect the rating. A score of ‘2’ is used to rate ‘unmet needs’ (i.e. there is a need but no or insufficient care to meet the need).

Table 1 The CAN items
Full size table

For the specific purpose of the CNCM, CAN instructions were slightly modified in order to allow reliable assessments by a large and changing group of professional carers. First, a patient and his or her clinician may have different views on needs. In the CNCM, information from clinician and patient is combined using a priori decision rules to maximize the clinical relevance of the rating. These rules imply that per domain, the highest score is filled in on the form. For example, need for care with respect to drug problems that is acknowledged by the clinician but not by the patient is scored as a need. Similarly, if a patient views him- or herself as problematically lonely, this will be acknowledged in the score, even if the clinician thinks that the patient’s social network is large and that contacts with, for example, the family are satisfactory. In addition, if the interviewer thinks a need for care in the domain of occupation/day time activities is met because the patient is involved in supported employment, but the patient would prefer to have a regular job (and this need is realistic), the need is scored as unmet. The point of view of the family is not included in the CAN unless the family’s position is endorsed by the professional carer.

Second, update and sharpening in instructions was necessary because a large number of clinicians is involved in rating. The original CAN manual does not provide a specific reference period; the examples given refer to ‘current’ needs [33], but the CAN is not designed to register temporary disabilities or needs, for example as a result of influenza or a broken arm. To guide clinicians, a reference period of 3 months was defined in the CNCM manual. Consequently, the CAN in the CNCM detects skill-level problems and lasting disabilities that warrant the organization of care substitutions. This was in line with the aims of the original CAN and probably poses no threat to its validity, because when filling in the original CAN-form, interviewers and patients implicitly keep a reference period in mind.

Third, some themes that are relevant in regular care were absent from the original CAN. The need to have a regular job (paid work) was added, because daytime activities as organized by professional carers (i.e. supported employment and vocational rehabilitation) are different from jobs in the ‘real world’. Many SMI patients want regular jobs, although it has been reported that obtaining such jobs is extremely difficult for patients [27]. Another added item is ‘recovery’, which includes all domains of meaningful functioning beyond mere symptom remission, including identity, perspective, future and goal in life. Side effects of medication, problems with sleeping and legal problems (excluding illness-related involuntary admission) have also been added to the CNCM version of the CAN. Informal caregivers are a valuable resource in the care for patients. Therefore, two items assess the needs of the informal caregivers. The first assesses the need for support to reduce the burden and alleviate stress related to care for their ill relative; the second assesses their need for information.

It is not always possible to score a specific need when patients are not challenged because of a particular environment in which they are living. For example, a patient living in residential care may be able to cook, but because meals are cooked for the patient, the clinician does not know that he or she can cook. In the CNCM, this is scored 8 (i.e. the clinician does not know; there may be an ‘overmet’ need).

The CAN is designed to generate sum scores for met needs and unmet needs, since testing all 22 original items would lead to multiple testing and inconclusive results. As the instrument is designed to measure all relevant domains of need, the list is very heterogeneous and a high level of internal consistency cannot be expected and is not necessary [24]. Although the reliability of the CAN has nevertheless been reported as being ‘acceptable’ [24], factor analyses using the CNCM data [10, 11] indicated that childcare, intimate relationships, sexual expression, alcohol and drugs did not load on the main, general need factor (3 sum scores: met needs, unmet needs and total). Reliability of the scales was higher when these items were left out. Therefore, sum scores in several CNCM analyses were based on 17 of the 22 original items [10, 11]. Because an 8 (potential over-met need) can be scored, the scoring is different from the original CAN version and this has consequences for the sum scores. Because a score of 8 and missing values are invalid, the CAN (CNCM version) sum score can be best calculated as a proportion of the valid items. This difference from the original CAN may increase the sensitivity to change.

Individual items can be used for specific research questions, because previous work suggests that relevant individual needs can become ‘hidden’ beneath CAN sum scores [11, 25, 45]. However, in order to avoid multiple testing, researchers selected during a consensus meeting an a priori limited set of items that includes areas of need for care deemed most essential for functioning in society and independent living. Accommodation, household skills, self-care, daytime activities, psychotic symptoms, psychological distress, self-harm, safety to others, alcohol, drugs, money and benefits were selected for this subset [11].

Brief Psychiatric Rating Scale

The Brief Psychiatric Rating Scale (BPRS) is a comprehensive instrument measuring the severity of symptoms. Originally published with 18 items [29], the CNCM version of the BPRS is based on the 24 items of the UCLA version [44]. In the CNCM version, a time window of 2 weeks is used. Clinicians are extensively trained using videotaped material to adhere to the anchor points defined by Ventura et al. [43].

Based on previous BPRS research [44], a confirmatory factor analysis was performed, using the CNCM data [3]. This confirmed the existence of four underlying constructs (also called the BPRS-symptom dimensions): negative symptoms, positive symptoms, manic excitement and depression/anxiety. Blunted affect, motor retardation, emotional withdrawal and self-neglect loaded on negative symptoms; bizarre behaviour, unusual thought content, disorientation, hallucinations and suspiciousness on positive symptoms; motor hyperactivity, elevated mood, excitement, distractibility, hostility and grandiosity on manic excitement; and depression, anxiety, suicidality and guilt on depression/anxiety. These four dimensions were used in previous CNCM research [3].

Over the years, four items were added at the request of clinicians and based on additional scientific evidence: dissociation, obsessive thoughts, lack of motivation and anhedonia (added in 2008).

Remission

Remission is one of the main outcomes used in CNCM research, because it has been shown that the definition of symptomatic remission is clinically meaningful, appears achievable for a significant proportion of patients in routine clinical practice and is applicable across the course of illness [39]. International criteria for symptomatic remission in schizophrenia have been defined [2] and can be assessed using a subset of the BPRS items. Patients are in symptomatic remission if they score 3 or less on all the following items: paranoid delusions, grandiosity, unusual thoughts, hallucinations, incoherent thinking, flat affect and mannerism [2, 39].

According to the criteria, the scores on selected items should remain below 3 for at least 6 months (full remission). However, the 6-month criterion cannot be used in the CNCM data because assessments can be more than a year apart. Nevertheless, a score of less than 3 on the remission items over the scoring period of at least 2 weeks (i.e. symptomatic remission rather than full remission) is considered a clinically relevant outcome in daily life practice in patients with SMI. Previous work using CNCM data has shown good validity of the CNCM-based remission criterion in terms of needs, functioning and patient subjective outcomes [40]. In addition, sensitivity analyses can be performed (defining remission as meeting the criteria at two consecutive CNCM assessments) and previous research revealed that the association between ACT and remission was stronger than when analysing symptomatic remission at one assessment [4].

CGI as alternative for BPRS

Severity of symptoms can also be measured using the Clinical Global Impressions (CGI) scale [16]. The CNCM offers this instrument as an alternative when time restrictions limit the use of the BPRS in services that have a less pronounced psychiatric signature (e.g. the general practitioners’ emergency clinics). The version of the CGI used for the CNCM is a combination of the CGI for schizophrenia patients and the CGI for bipolar patients. Patients can be rated (range 1–7) on five symptom dimensions, namely positive symptoms, negative symptoms, affective symptoms, cognitive symptoms and manic symptoms. The reference period is the preceding 2 weeks. The original CGI also includes change ratings (improvement or deterioration since the last assessment) but these are not included in CNCM.

The reliability of CGI ratings by clinicians can be problematic because scoring should be related to a reference group of patients that share the same diagnosis. This ‘reference’ can be perceived differently from one clinician to the next. Therefore, the CGI was adapted in order to allow reliable assessments by a large and changing group of professional carers. The CNCM manual provides an extensive description of the patients’ severity of symptoms for each of the five domains and each of the seven possible scores per domain (anchor points). This description is based on BPRS and PANSS rules and scored from normal to severely ill.

The BPRS was scored in 69.9% of the CNCM forms (completely), 63.7% of the forms contained CGI assessments, and 48.7% (n = 4509) contained both BPRS and CGI-scores, while 15.1% contained none of the two instruments. This allowed us to compare BPRS and CGI. First, the BPRS sum scores and the CGI-items were highly correlated (positive symptoms: 0.79; negative symptoms: 0.63; depression: 0.71; mania; 0.66; all p < 0.001). Second, we split the sample randomly and used the first half for a stepwise regression of all 24 BPRS-items to predict the 7 CGI-items (one at a time). The other half was used to validate the prediction. This prediction was best for positive symptoms (R 2 = 0.68 in first and 0.66 in second sample; 90% no more than 1 point difference between real CGI and prediction) and manic symptoms (R 2 = 0.53 and 0.44; 95% no more than 1 point difference; Table 2) and worst for negative symptoms (R 2 = 0.48 and 0.37; 88%) and global severity (R 2 = 0.45 and 0.42; 81%). Considering the fact that CGI-scores are one-item ratings of mental state dimensions, a highly significant correlation with predictive value with multi-item scales is considered satisfactory.

Table 2 Prediction of CGI, including all 24 BPRS-items
Full size table

DSM-IV diagnoses and global assessment of functioning

The CNCM includes the five axes of the DSM-IV [1]. Since 2007, diagnostic assessments have to be updated yearly in order to comply with rules for financing in the context of diagnosis-related groups. The clinicians rate axes 1 through 3 in the traditional fashion. No specific training is given. Axis 4 is covered using information collected by the CAN. Axis 5 rates social functioning and is assessed using the Global Assessment of Functioning (GAF) (scores 0–100), split into its psychopathology (GAF-p) and impairment (GAF-i) components [1]. Traditionally, GAF scores are given for ‘actual’ functioning and best functioning in the past 12 months [1]; in the CNCM, however, GAF assesses the worst score for the past month.

The DSM-IV assumes that rating the GAF is self-evident. However, on average, CNCM-based GAF-p scores were 52 (range 4–95) and GAF-i scores were 51 (range 0–97), while the reference score for SMI patients is below 40. GAF scores for SMI in the CNCM database have drifted upwards because professional carers rate their patients compared to their caseload rather than according to the general population reference requested by the manual. Therefore, since the second half of 2006, GAF instructions have been part of the basic and booster training sessions. Most of the trainees acknowledge that they have overrated their patients compared to the scoring instructions.

Quality of life

The quality of life scores are meant to be subjective ratings and are scored by the patients themselves. In the CNCM, patients are asked to rate their quality of life on five themes derived from the sub-scores of the Lancashire Quality of Life Profile [28]: general quality of life, living situation, social relationships, physical health, and mental health (Table 3). These themes are rated on 7-item Likert scales (1 = not, 4 = fair, 7 = very). Although developed independently, the CNCM quality of life scale is very similar to the MANSA [31] and previous research showed strong item-subtest correlations (r > 0.80) with the Lancashire Quality of Life Profile for matching item sets [38].

Table 3 The five CNCM quality of life questions and one quality of care question
Full size table

Quality of care

An extra question on quality of care was added in the same format as the quality of life questions (Table 3). It is logistically impossible to ensure that patients rate the quality of care independently of the clinician who is present during the interview. To minimize contamination, however, clinicians are asked to allow the patient privacy during the rating and to provide an envelope that can be sent sealed to the helpdesk. The quality of care rating can be omitted from the feedback report if the patient requests so.

Definition of SMI

The CNCM is also applied in non-SMI patients, given that it is a broadly applicable method to obtain ROA measures. Most previous analyses were restricted to the subset of SMI patients. Thus, criteria for SMI had to be defined. SMI patients were defined as having a diagnosis of schizophrenia or non-affective psychotic disorder (DSM IV 295, 297 or 298) or having affective disorder (bipolar disorder: 296.48 or 301.13) or depression with psychotic features (296.14, 296.24, 296.34). In addition, several other criteria were defined because registration of diagnosis is not always complete. Patients who score 15 or more on the positive symptoms scale of the BPRS were included in the SMI group, as were patients with a combination of low functioning (one of the two GAF scales <45) and needs for care in at least two of four a priori selected domains, that is accommodation, welfare benefits, alcohol and drugs. The rationale was that ‘accommodation’ and ‘welfare benefits’ refer to needs in daily life essential areas leading to major social problems. Needs related to addiction (alcohol and drugs) refer to a comorbidity that severely disables patients. Patients who attend addiction clinics for a simple addiction diagnosis are not defined as SMI because they do not score less than 45 on any of the GAF scales. SMI is a patient characteristic: if a patient meets the criteria at one assessment, he or she is included in the SMI group for all assessments.

Patients who score less than 45 on one of the two GAF scales and have a need in one of the four above-mentioned CAN domains are defined as moderately mentally ill (MMI), that is, as only slightly less severely mentally ill than those in the SMI group. Not included in the patient group are patients with mild psychiatric symptoms representing common mental disorders (i.e. all patients not meeting the above criteria). As stated, clinicians tend to overestimate the GAF. Therefore, the traditional cut-off of GAF scores below 40 for SMI was raised to 45.

Subjects and prevalence

The CNCM covers a geographically defined area in a small European country (South Limburg catchment area, the Netherlands), in which approximately 382,000 persons aged between 20 and 64 reside (2004–2006). In order to estimate the number of patients who should be in the database, data of the local Psychiatric Case Register (PCR) [9] covering the period 2004–2006 as well as population data were obtained in order to calculate yearly prevalence of treated SMI (the PCR methods are described below). Approximately 6,000 patients in care are diagnosed with schizophrenia, organic psychosis or bipolar disorder; this figure represents 1.6% of the adult population. Prevalence is rather constant over the years. The number of subjects in the CNCM database is also approximately 6,000. However, some professional carers also use the CNCM system for non-SMI patients. These non-SMI patients (approximately 25% of the CNCM database) are excluded when analysing data for the purpose of SMI health services research. Thus, only 4,500 of the 6,000 SMI patients are in the database, but this number is increasing. For illnesses that have a protracted course, prevalence is much higher than incidence. Yearly incidence figures are around 2–3 per 10,000 inhabitants [8, 26].

Of all SMI patients in the region, 75% are diagnosed with schizophrenia or other psychotic disorder (including bipolar disorder); the remaining 25% mainly suffer from adult autism, chronic refractory depression or borderline personality disorder. This is in agreement with an Italian study in which 72% of SMI patients (defined using the Health of the Nation Outcome Scales) were diagnosed with schizophrenic disorders or affective disorders [22].

Linking CNCM data to Psychiatric Case Register data: service consumption component

The Psychiatric Case Register (PCR) monitors all mental health care provided by the psychiatric services in the geographically defined area [9], since 1983. The PCR is an anonymous cumulative register of admissions (including duration), outpatient contacts and days in day care. The register includes data from the psychiatric hospital, the community mental health centre, the psychiatric department of the university hospital, the community psychiatric emergency outreach team, psychogeriatric nursing homes, sheltered housing, child psychiatric services, services for the mentally impaired, and alcohol and drug misuse services. The PCR includes information on demographic and socioeconomic variables (e.g. educational level: high/low; problems in the working situation: yes/no). CNCM and PCR data can be matched anonymously at the case level using an encrypted identification code that is provided through a secure Internet connection. This procedure ensures that case material can be linked to the same case (>99% certainty) without being able to trace information back to specific persons.

Medication monitor

The CNCM optimizes a comprehensive dataset with minimized investment. A limitation of this approach is that not all the crucial information for clinical decision-making is available. For example, although some information on medication is included, no details are available. In addition, recent guidelines stipulate that patients who are on antipsychotics should be examined medically on a regular basis in order to reduce the risks of, for example, diabetes or metabolic syndrome. Therefore, the Medication Monitor (MM) was added to the CNCM.

Psychiatrists can enrol their patients in the MM system by performing a baseline measurement whenever new medication is prescribed or an existing prescription altered. A nurse practitioner examines vital signs, BMI, waist circumference, prolactine, risk factors for metabolic syndrome and diabetes mellitus, and signs of extrapyramidal symptoms and akathisia 3, 6, 12, 18 and 24 months after baseline. MM will then continue annual assessments as long as the medication is not altered.

The MM currently includes a subsample of patients who are prescribed antipsychotics (n=281). At the first assessment, an average of 5.2 (SD 2.9) of 15 thresholds indicating problems were present per patient (waist, diastolic blood pressure, systolic blood pressure, triglycerides, HDL, LDL, body mass index, glucose, fasting glucose, haemoglobin/HbA1c, cholesterol, prolactin, akathisia, rigidity, tardive dyskinesia); in only 11% of the patients none of the thresholds were exceeded. This underlines the importance of the MM. A closer look shows that 98 (35%) of the patients met the criteria for metabolic syndrome, 50% had movement disorder, 37% were overweight, another 25% were obese and 1.6% were underweight.

In addition, CNCM data showed that almost half of the SMI patients were chronically (>3 months) prescribed combinations of medications that were often not in accordance with guidelines [37]. Because the registration of medication in the CNCM is limited, this estimate represents an underestimation of the total prevalence of polypharmacy. Future use of the MM will improve clinical practice and will provide more precise estimates of polypharmacy. In addition, the association between polypharmacy and health parameters (e.g. mental state, physical health, functioning) can be assessed using MM data.

Statistical analysis of CNCM data

In the CNCM, patients are interviewed multiple times over protracted periods of time. Therefore, the dataset contains more than one observation per person, violating the assumption of independence of the observations needed for standard linear regression analyses. Multilevel linear and logistic regression analyses are ideally suited for the analysis of this type of data, yielding more than one observation per person (i.e. several records in the dataset) [34]. The regression coefficients obtained from multilevel linear regression analyses and the odds ratios obtained from multilevel logistic regression analyses can be interpreted in the same way as estimates obtained from standard unilevel analyses.

Although there is an option to derive multilevel frequency tables, these frequencies may be more difficult to interpret. Percentages do not add up to 100 if a subject scores differently at different assessments and he or she is counted twice.

Results of CNCM data analyses

CNCM data have proven useful for health services research analyses. For example, need for care was linked to care consumption as registered in the PCR [10]. Results showed that need for care predicted intramural care consumption, but not outpatient care consumption. This was interpreted as indicating that the more severely ill patients with higher needs for care depended more on hospital resources and thus made less use of outpatient facilities. In a second paper, the validity of the CAN was further analysed by assessing changes from unmet to met needs over time [11]. In recent-onset patients, the number of met needs increased over time, but unmet needs did not decrease, while neither met nor unmet needs changed over time in chronic patients. The reason that sum scores appeared insensitive to change may be that new problems come to light only when need for care in other, more immediately visible areas is addressed. Thus, some items may be reported when the patient is more severely ill, while other items are scored only when the more severe problems are no longer present. As a consequence, the CAN sum scores may be of limited use when following patients over time [25, 32, 45]. The same paper also studied individual CAN items and showed that professional carers can improve the financial and housing situation, as well as independence regarding self-care and household skills, while meeting patients’ needs in the areas of occupation/daytime activities, psychotic symptoms, psychological distress and self-harm proved more difficult [11].

Using CNCM data, Van Os and colleagues [40] assessed the clinical validity of the remission criteria as defined by Andreasen and colleagues [2]. In two later papers, ACT was associated with increased remission rates in a pre-post design [4] and in a matched-controls design [12].

The efficiency and effectiveness of the CNCM for individual treatment is to be assessed further within the scope of the Efficiency of Tailor-made Psychiatric Rehabilitation (E-Tail) study, an externally funded study.

Limitations

The CNCM database is unique in that it will soon hold data on the large majority of SMI patients in a circumscribed catchment area. Assessments are integrated into routine clinical practice.

It has been suggested that an overload of data can obscure results that are truly relevant to the treatment of individual patients [17]. CNCM staff, therefore, strives to unlock these relevant data for the professional carers, using feedback on the individual patient. In addition, the filling in of multiple choice items may restrict the clinician to asking a selective set of questions rather than having an open-ended dialogue with the patient, and this could lead to ‘standard rather than customized care’ [17] (p. 1658). Thus, the CNCM form should be seen not as an alternative to an assessment, but as an addition to it. Therefore, the CNCM form should be brief, so that not all the available time is taken up by this instrument.

Although the basic design of the CNCM represents a strength, it does entail some limitations. First, as mental health care professionals collect data of their patients, patients who are difficult to trace, because they are homeless or avoiding the mental health system, are missed while these patients are at the most severe end of the SMI spectrum. This limitation is inherent to the design. However, local services are based on ACT with extensive outreach. Clinicians actively engage in contact, visit patients at home and keep patients in their caseload. Potential patients who are identified by street workers, the police, general practitioners or housing corporations are visited at home by mental health care professionals and included in the database. Service providers outside the mental health system (e.g. those in the social security system) are invited to refer their potential cases, which are reviewed on a regular basis by the ACT teams. Finally, the alcohol and drugs treatment system has participated in the CNCM since 2007.

Second, although one might suspect that patients are more often assessed during crisis situations, patients on acute wards that have high pass-through rates are underrepresented in the database, given time constraints. Thus, most assessments are collected in stable treatment phases. This does not affect the need assessments (CAN) with a time frame of three months, but does, of course, affects the mental state data (BPRS), which has to be considered a snapshot that, on average, may underrate the true morbidity.

Third, clinicians collect CNCM data. The use of research assistants, as advocated by some ROA systems, may result in more reliable and complete data. However, this is not possible within the mental health budget. More importantly, however, as the CNCM is intended to be a quality improvement tool for regular clinical practice, interviewing is a task for the mental health care workers. Standardized assessments and systematic needs assessment by the carer rather than an outside interviewer are needed for quality improvement. Using external raters therefore will not result in quality improvement and also will not result in the uptake of feedback provided by the instrument. Use of external raters therefore is strongly discouraged in the CNCM system. In order to guarantee reliability, a manual was developed and clinicians are trained extensively to improve their mental state assessment, to focus on treatment needs, and to systematically assess addiction-related problems and somatic needs [6]. In addition, newly appointed mental health professionals require on site supervision before conducting CNCM interviews independently. These efforts have resulted in satisfactory test-retest reliability (unpublished results).

Furthermore, the CNCM database contains ‘dirty’ data that have a number of missing values. SMI patients are not always willing to cooperate. Therefore, imputation and sensitivity analyses are used to verify the validity of the reported results. In addition, the CNCM protocol requires all patients to be assessed at least once a year. However, in practice, the average time between successive interviews can be as much as two years. The helpdesk routinely sent reminders to all interviewers who do not turn in the yearly reassessment to increase the compliance rate.

Extensive mental state assessments using the BPRS are absent from services in which symptoms are assessed with the CGI. However, these services have a less pronounced psychiatric signature serving the less severely ill patients. Thus, BPRS data are rather complete in the more severely ill patient group, but not in the non-SMI group.

Finally, ROA data are required by some insurers that demand a certain proportion of favourable outcomes. As a consequence, a potential bias is introduced in the system of data collection. However, the CNCM in the Netherlands is not dependent on the financial reimbursement systems. Therefore, this risk of bias is avoided.

Conclusion

The CNCM has been operational since 1998, which means that the first cohort now contains a 10-year follow-up. In addition, the PCR had its 25th anniversary in 2008. Although there are (or have been) various PCRs in Europe and the USA (Groningen [35], Rijnmond [21, 47], South Verona [36], Camberwell [46], Belgium [23], Denmark [15], Finland [18], Nottingham [5], Salford [14], Maryland [13]), the longitudinal monitoring of needs for care, severity of symptoms, functioning and other outcomes in the same region, allowing for linking clinical and service consumption data, is rare. Data of the CNCM, merged with PCR data, provide an insight into the care that was provided and what effects were obtained. This enables researchers to answer a wide variety of research questions. However, the CNCM is also part of treatment; it provides information on the individual patient, which can be used to tailor treatment. This makes it not only a valuable research and management tool, but also an asset in clinical practice.