Background

Advances in digital information management systems, including databases, have become important tools in understanding trends and correlations in health care by improving the collection and organisation of information such as demographics, diagnostics, therapeutic interventions, and outcomes. Routine collection of such data has now become commonplace in national health systems like the United Kingdom National Health Service (NHS) with recent imperatives focusing on the most efficient use of such data for key goals such as, improving patient outcomes, increased efficiency, and the development of effective new treatments [1].

Analysis of data collected from large cohorts of patients can have the potential to generate insights into factors identifying treatments and the characteristics of subgroups of patients who respond to certain types of care. Observational studies using large data sets have led to important public health discoveries through epidemiological analysis. For example, the Framingham study led to landmark breakthroughs in the understanding of blood pressure in disease [2].

The advent of machine learning (ML) techniques, a form of artificial intelligence (AI), is starting to change health care in profound ways, including reducing bias in diagnosis, facilitating the use of multiple sources of information to improve the understanding of pathological conditions, and improving the analysis of data sets [3]. [4] ML techniques are derived from traditional statistical methods and do not always provide clarity when trying to understand complex systems [4]. However, ML is beginning to reduce human burden in the task of diagnosis in a world of increasingly complex and interacting factors [4].

Computer technology has also made it easier to create databases that can be tailored for specialised purposes. The Care Response (CR) database has existed for ten years now and has grown steadily over time. It was designed to capture patient-reported outcome measures (PROMs) for chiropractic patients internationally. Different PROMs capture information in one or more of five different categories: quality of life related to health, functional status, symptoms, health behaviours, and patients’ perspective on their health care experience [5, 6]. PROMs can be used to monitor patient progress, guide clinical decisions, and benchmark treatment outcomes [7, 8] The use of PROMs has been associated with improved communication between clinician and patient, better symptom control, and increased patient satisfaction [9].

Specifically, the CR database was developed to help clinicians overcome some of the barriers to using PROMs regularly in day-to-day clinical practice. Electronic PROM systems are believed to reduce the administrative burden (time & cost), simplify the collection and reporting of results as well as increase the completeness of returned assessments [5, 10,11,12,13] PROMs available on the CR system include the Bournemouth Questionnaire (BQ), Measure Your own Medical Outcome Profile (MYMOP), EQ-5D, patient global impression of change (PGIC), numerical pain scale, and Patient Reported Experience Measure (PREM) [14].

The BQ was developed to be a quick, comprehensive assessment instrument. Studies have found the BQ to be valid and reliable in back [15] and in neck [16] pain patients. MYMOP [17], is a health-related quality of life (HRQL) measure. It allows patients to nominate and score the two most important aspects of their lives that contribute most to their overall quality of life. The EuroQol 5 dimensions questionnaire (EQ-5D), is also an HRQL measure. It is commonly used for people in chronic pain [18]. Compared to some other measures, the EQ-5D has produced higher scores in healthier people, and lower scores in severely ill patients. The EQ-5D may also be more sensitive in conditions with primarily physical limitations and disability [19]. The PGIC is a 7-point ordinal Likert scale (very much improved, much improved, minimally improved, not changed, minimally worse, much worse and very much worse) [20]. The numerical pain scale (NPS) is an 11-point (0–10), commonly used pain rating tool which is very quick to administer but it is challenging to use to infer clinically important changes from baseline [21]. The PREM in the CR database is two open-ended questions: (1) Is there anything particularly good about your chiropractic care? and (2) Is there anything that could be improved?

The CR database is currently provided to clinicians as a free-to-use online system. Designed to be used in busy clinics, the advantage of the CR database is that it moves away from reliance on clinicians to provide PROMs themselves during consultations, collecting information directly from patients. After obtaining informed consent, clinics register patients by adding their name, date of birth, email address and date of first appointment either using a ’self-service’ link provided by the clinic or by clinic administrative staff. Once these fields are populated, the CR database generates a PROM questionnaire, based on the clinician’s preference, and this is provided to the patient usually via an automated email link or by the clinic on a PC or tablet device, or in paper form. Any of the available PROM instruments can be selected as a whole; questions cannot be selected from different PROMs and combined by a clinician. Multiple PROMs may be selected for use by any clinic. Subsequent PROM questionnaires are generated either at pre-set timed intervals or by the clinic manually making a request. PROM questionnaires are scored, and the clinician is presented with collated results for an individual patient or group of patients in tabulated or graphic format for longitudinal assessment after serial use of PROMs. The results of these are available immediately they are completed facilitating their use within a clinical encounter.

The CR database has been adopted in clinical practice across diverse settings and multiple countries and 218,770 patients used the CR database up to September 2022. However, up to date information on the number of clinics or countries is not available, as the system is anonymised. There are also no data on the numbers of NHS versus private patients.

Data from the CR database have been used to support multiple publications, including one of the largest UK wide cohort studies within the chiropractic profession to inform the feasibility of online PROM collection [22]. Others include the use of PROMs in clinical education, [23] studies on outcomes [24, 25] and predictors of response to chiropractic care [26, 27] There has also been a series of papers exploring the utility of the STarT Back Screening tool in professions working in the independent sector [28,29,30]. Care Response data have supported 2 PhD studies, including one which resulted in the development of a theoretical model of the impact of PROMs in clinical practice [31, 32] But despite these studies there has not yet been a comprehensive assessment of the contents of the CR database.

We sought detailed participant demographics, signs and symptoms, diagnostic data as well as information on therapeutic interventions, length of treatment sessions and length of care plans. We hoped in a future study to apply ML methods to the large number of variables in the large cohort of participants to glean insight into which interventions were most effective for which conditions. This could then support clinical decision-making and policy development [33]. With information on signs and symptoms, it may be possible to discover previously undiagnosed conditions [34].

This study therefore aimed to generate a description and preliminary analysis of the entire data record, with the following objectives: to document the content of the CR database, and to explore whether the depth and breadth of information contained within the CR database would be appropriate for the application of ML analysis.

Methods

Prior to commencing data collection, data sharing agreements were drawn up between the University of Central Lancashire (UCLan) and AECC-University College (AECC-UC), and between UCLan and Clinical Transparency, Ltd, owned by Jonathan Field, which holds the CR database. Ethical approval was obtained through the UCLan Research Ethics Committee, approval #HEALTH0287. Consent to participate is obtained by Care Response and includes the option of use for research by third parties.

Two authors (KJY and JFitzgerald) first manually scanned the data to assess a broad sense of scope and completeness. The search was not date-limited, including the full 10 + years of data in the CR database. Categories of data, such as demographics, were extracted into separate tables to facilitate analysis. Without knowing what data were collected by the CR system, we were unable to determine a priori which data points we would analyse. We broadly considered that we would explore the association of demographic factors and response/completion rates. We did not know if there may be differences in responsiveness to individual items, therefore we included individual items in the analysis. Descriptive statistics were calculated for timepoints based on relevant patient characteristics available in the dataset. Based on the contents of the database, a judgement was made on the suitability of ML analysis.

Results

Anonymised data were extracted from the database on 12 April 2022. After data extraction from the CR servers, we received an Excel spreadsheet with 32,468 rows, each with 95 columns. 30,940 people (95%) completed the baseline data; 14,695 (45%) completed the 14 days post-clinical-visit questions; 12,764 (39%) completed at 30 days, and 8689 (26%) completed at 90 days. These figures do not reflect completion of all questions up to these time points; that is, for instance, someone could have completed the baseline and only the 90-day questions.

The database contains clinical metrics including various validated PROMs. A list of available data points was obtained which included patient age and sex, region of pain, patient global impression of change, pain rating, and information from the STarT Back Screening tool [35] and Bournemouth Questionnaire [15, 16]. Please see Additional File 1 for a list of the available data points and their codes and further details. However, other variables were limited and did not include information on signs and symptoms, diagnostic data, therapeutic interventions, length of treatment sessions or length of care plans. There were no data returned relating to the EQ-5D, MYMOP, or PREM outcome measures. Of the 32,468 rows 3,210 contained a complete data set, that is, every box filled in, to 90 days. When patients sign up to the system, they are given an option to have their anonymised data used for analysis and only those who gave this consent were included in the data export. That proportion of the total was unknown.

Table 1 Timepoint Completion – numbers of people who completed input at the different points sampled
Full size table

Table 1 shows participants who completed the Baseline, 14 days, 30 days and 90 days; 15.65% of those who completed all database fields completing through to 90 days. It is of note that more female patients completed through to 90 days than male patients, but the data gave no indication of why this might be.

Table 2 Patient characteristics
Full size table

Table 2 was created to explore whether there may have been a link between completion of data points and data such as gender, age, new to practice and same therapist; once again, the completion by female patients was higher than that of male patients. Age was indistinguishable across both genders and completion data.

Table 3 Responses to STarT Back Screening Tool (SBT)
Full size table

Table 3 shows the responses to the STarT Back Screening tool in terms of average pain scores, with little gender difference seen except possibly in “bothersomeness”. However, we did not test whether this difference may be significant or not, as we were only seeking to document the contents and completeness of the database at this time.

Table 4 STarT Back Ranking responses
Full size table

Table 4 compared the level of pain expressed by patients, considering data completion and gender; the highest scores were for low and medium pain, with a slight increase in the “low” sector. Table 5 shows the responses to the NPS. Table 6 reviews the average scores across the pain domains, around 6/10 at baseline and reducing to an average of 2.15, with those completing all time points showing a lower pain score than those who didn’t complete all data points.

Table 5 Responses to numerical pain scale (0–10)
Full size table
Table 6 Average Bournemouth questionnaire information
Full size table
Table 7 Patient Global Impression of Change (PGIC)
Full size table

Table 7 shows a difference in the impression of change in pain level between those who completed all data-points and those who did not. Table 8 shows a marked improvement in satisfaction between those who completed all the data points and those who did not.

Table 8 Patient satisfaction Only data from 30 days post appointment was retrieved. Scale is 0–10 with 10 representing complete satisfaction
Full size table

Discussion

We found the CR database somewhat limiting to use for clinical research. It was designed for practitioners to better understand their practices, and so perhaps unsurprisingly other uses require workarounds. Member clinics get reports on their PROMs free of charge, but there was an additional financial cost to extract the data for the current analysis. Compared to other databases (e.g. healthdata.gov, healthdata.org.uk, Cochrane Open Access), the information in the CR database was limited.

The CR database anonymises responses and so tracking of individual patients was not possible. One patient may be represented by multiple lines, so this is a major limitation. Many patients could have a few rows each and thus the data could quickly be greatly biased. We also have no indication that either responders to care or non-responders would be more likely to participate in CR. Care Response did not include any information on diagnosis, co-morbidities, type of treatment interventions used by the reporting clinic, the number of visits for treatment, or co-treatment by other practitioners. The demographic information included only sex and age. These are points that could be addressed in a future version of CR.

Studies have shown that chiropractic care often produces improvements in clinical outcomes, particularly for low back pain [36,37,38] The generally positive outcomes found in the CR database could be the result of the therapeutic interventions applied. However, the therapeutic alliance and contextual factors are also emerging as important elements in clinical improvement [39,40,41,42,43] There is evidence that chiropractors develop a positive therapeutic alliance with their patients [44]. Therefore, it seems likely that improved clinical outcomes are due to these effects in various combinations and proportions. However, it should be noted that regression to the mean, expected improvement over time due to natural history, and other factors are also likely involved in producing these results.

The data currently available on the CR database are limited. The available data are not useful in terms of helping to determine which therapeutic interventions correlate with better scores on PROMs, which interventions are reported as more effective on which conditions or which demographic groups. Without information on diagnoses and therapeutic interventions, we identified a lack of utility in the current state of the CR database to use ML or statistical methods for clinical prediction.

There are positive elements to the CR database. It is one of few large cohort databases within the realm of chiropractic. Generally, large chiropractic cohort studies have focused on specific research questions [45,46,47,48,49] There is no other system that has broadly collected PROMs and PREMs information on chiropractic patients on an ongoing basis for over 10 years and continuing now. The CR database is not limited to one country, and thus researchers in the future could design studies to compare outcomes in different international regions. Finally, patients can enter information directly into the system electronically via the internet. As noted above, this may facilitate response rates and reduce the time clinic staff use to collect the data.

Strengths of this study

This is the first study to explore the contents of the CR database for its potential to be investigated using machine learning methods, although the result was ultimately negative. In addition, although a number of observational cohort studies of chiropractic patients can be found in the literature, they involved dozens to low thousands of participants, [50,51,52,53,54,55] whereas our analysis of the CR database contained PROMs data for over 33,000 individual courses of care, with the caveat that we do not know how many individual patients this represents. We view Care Response as augmenting the information in other large cohorts collecting data on chiropractic patients. So, understanding the scope and completeness of the data is useful to help guide future research projects using this database.

Limitations

The sample group included in CR is limited to only those patients who chose to include themselves, attending clinics where the practitioners have chosen to enrol on the system. Consequently, the cohort would seem to include only the highly motivated patients of highly motivated practitioners. Thus, there is potential inclusion bias in the sample. The potential inclusion of patients on multiple lines on the spreadsheet of data was another potential source of bias. There was a lot of missing data; about 16% of patients completed all items through to the 90-day follow-up. It was unknown as to how many returning patients were also included as new patients, so there may have been some overlap or mis-categorisation of patients. Despite the large size of the data set, information on outcomes is not generalisable to chiropractic patients at large due to the amount of information missing or not collected.

Further research

It is challenging to know how meaningful correlational analyses might be. Is it useful to know, for instance, if female patients, or an older age group responded better to ‘chiropractic care’? If the system collected more data about diagnostic and treatment information, as well as reporting location, socio-economic, and cultural data about patients, more conclusions could be drawn about which aspects of chiropractic care provide the most benefit to patients.

Conclusions

The CR database is a large set of PROMs for chiropractic patients internationally. However, its utility is limited by a lack of demographic information, diagnoses, and therapeutic interventions. It can offer information about chiropractic care in general and patient satisfaction. It could form the basis for a useful clinical tool in the future, if reformed to be more accessible to researchers and expanded with more patient demographic data points as well as information on diagnoses and therapeutic interventions.