Background

Osteoarthritis (OA) is a leading cause of joint pain and years lived with disability worldwide causing considerable detrimental impact on daily activities and quality of life [1–4]. OA is one of the main reasons for musculoskeletal consultations with a general practitioner by older adults [5].

High quality care is described as clinically effective, personal and safe, which is delivered to all users of a health service in all aspects of care [6]. However, previous studies have shown that the quality of care provided to patients with OA in primary care is suboptimal [7–11] and varies according to patient age and OA severity [12]. Research has shown that patients with OA need more information and education about the condition, diet, exercise, aids, and better support for self-management [13]. However, core recommended treatments such as exercise, weight loss and the provision of written information is underused for patients with OA [7]. Furthermore many core treatment are initiated by the patients themselves rather than doctor initiated [7, 14].

Several international guidelines exist which provide recommendations for the management of OA [3, 4, 15, 16]. International quality standards for OA have also been developed such as those recently published by the National Institute for Health & Care Excellence (NICE) [17], and the European Musculoskeletal Conditions Surveillance and Information Network (eumusc.net) [18]. Yet there are no robust or routinely collected measures used currently in general practice to monitor the quality of care for people with OA [19], although an OA e-template for use during consultations in primary care has recently been developed and tested [20, 21].

Quality indicators (QI) are ‘specific and measurable elements of practice that can be used to assess the quality of care’ [22]. They are used to assess care quality according to defined standards of care (e.g. NICE [17], eumusc.net [18]). QIs typically assess the processes of care given to patients [23] by measuring what the provider can offer patients and examining whether standards of care are being implemented.

A systematic review identified 15 QIs which are broadly applicable with current international guidance for the assessment of non-pharmacological and pharmacological management of OA in primary care [8]; however the authors recommended an increased use of QIs in primary care from the patient perspective.

The Management of OSteoArthritis In ConsultationS (MOSAICS) study [24] developed and evaluated a new model of supported self-management of OA to implement the NICE guidelines for OA in primary care [3]. The MOSAICS study aimed to evaluate the new model of supported self-management for OA in primary care, in terms of the quality of care from both a clinical and a patient perspective (see Additional file 1 for more information about the MOSAICS study). The findings of the MOSAICS study are subject to other papers in production. However, at the time of designing the MOSAICS study, there was a lack of evidence regarding the experiences of patients with OA in primary care and there were few appropriate quality indicators that captured the quality of primary care for OA from the patient perspective. Active and meaningful patient and public involvement (PPI) is increasingly viewed and encouraged as integral part of the research process to improve its quality and relevance [25–28]. Therefore the collaboration between the researchers and the RUG described in this article led to development of patient reported QIs for the MOSAICS model of self-support in primary care.

During the course of the MOSAICS study, the research team became aware of a questionnaire capturing the patient perspective of the quality of OA primary care being developed in Norway. The OsteoArthritis Quality Indicator (OA-QI) has since been validated for use to measure the quality of primary care for OA in a Norwegian population [29]. The Norwegian OA-QI, comprises 17 questions related to patient education and information, regular provider assessments, referrals, and pharmacologic treatment. The tool was developed by team of researchers and OA clinicians, with input from two patient partners who gave feedback on the content of the finalised questionnaire. The results of the validation study are reported elsewhere [29].

We report on the co-production of a set of self-report, RUG-generated QIs to capture the quality of primary care management of OA. We also compare the RUG-generated OA QI (UK) questionnaire with a Norwegian OA-QI questionnaire developed in parallel, leading to a final recommended OA QI (UK) questionnaire.

The manuscript was written using the Guidance for Reporting Involvement of Patients and Public (GRIPP) checklist for reporting Patient and Public Involvement (PPI) in research [30]. This checklist provides a structure for improving the quality of reporting of the PPI and is designed for studies that have included some form of patient and public involvement in research. The authors have referred to the checklist to ensure all the relevant aspects of PPI in this study were reported.

PPI good practice

The PPI described in this article took place at the Arthritis Research UK Primary Care Centre, Keele University. This institution takes an explicit and systematic approach to involvement of patients and the public in research [31]. Formed in 2006, a Research User Group (RUG) was established to embed PPI across the whole of the Centre’s research activities and is supported by a dedicated PPI team and core funding. Currently, the RUG has over 60 members actively involved on over 60 projects, recruited on a basis of ‘expertise by experience’ of musculoskeletal and other long term conditions. Some RUG members have more experience in involvement in research than other members, though all provide the lay perspective of their health condition. Our approach to patient involvement draws on previous experience [32–35] and recommendations for the good practice of PPI [25, 30, 36, 37] so that RUG members can provide meaningful contributions to the research process (see Additional file 2 for more information about the RUG).

Our good practice principles include holding meetings in accessible venues at convenient times; allocated parking; meeting and greeting on arrival; training and support; inclusion of regular breaks during meetings; payment for RUG members’ time and contribution (if wanted); and reimbursement of expenses. Meetings between researchers and RUG members are made actively ‘jargon-free’ and any technical terms are explained in plain English.

Methods

The MOSAICS study investigated whether a new way of supporting self-management, delivered during an OA consultation in primary care, could offer a clinically practical approach to implementing the core NICE recommendations [24]. We describe here the development of the OA QI (UK) questionnaire by the RUG group for use in the MOSAICS study in Stage 1, the comparison of this with a Norwegian OA-QI questionnaire in Stage 2, and review of the finalised OA QI (UK) questionnaire in Stage 3. Figure 1 provides an overview of the process.

Fig. 1
figure 1

Overview of the stages of development of the OA QI

Full size image

Patient and public involvement in this study

In 2009, RUG members with OA were invited to form an OA PPI group to work in partnership with researchers throughout the MOSAICS study, including the development of patient-reported QIs, set within a wider five year programme of research into OA [24, 38]. Furthermore, two members (AH: a former member of the wider Research User group and now PPI Support Worker/Coordinator; and RT: Lay member of the OA Research User Group) have co-authored this article, including writing the plain English and providing detailed comments on the manuscript prior to the final submission.

During the course of the MOSAICS study, members of the research team met with RUG members to co-produce the OA QI (UK) questionnaire for use in the MOSAICS study. The discussion meetings were facilitated by the Centre’s PPI Support Worker/Coordinator, the MOSAICS study Chief Investigator and a trial coordinator. The PPI Support Worker/Coordinator provided a key role by attend the meetings with RUG members to provide assistance and support, prior, during and after meetings. The MOSAICS study Chief Investigator (KD) has collaborated with the RUG on numerous research studies and is currently the senior academic lead for PPI in the Centre. All trial coordinators at the Centre have a responsibility for ensuring PPI in their respective studies and have lots of experience of collaborating with RUG members.

Discussion notes from the meetings were recorded on flip charts and in meeting minutes. Following each meeting, a summary of the outcomes and decisions written in plain English was sent to the RUG members to acknowledge their contribution and verify that all views had been captured. RUG members were also given the opportunity for further comment at the start of the next meeting.

It was not intended to formally evaluate the PPI interaction and the RUG members’ experience in the process. However, the impact of the RUG members is described in this article in the form of the co-produced OA QI (UK) questionnaire for use in the MOSAICS study.

Ethical approval for the PPI activities was not sought because the RUG members were acting as specialist advisers, providing valuable knowledge and expertise based on their experience of a health condition or public health concern and their involvement did not raise any ethical concerns [39]. However, the full MOSAICS research programme was approved by the North West 1 Research Ethics Committee, Cheshire, UK (REC reference: 10/H1017/76) [24].

Stage 1: development of the OA QI questionnaire

Members of the OA RUG group (n = 10) were invited to a series of four discussion groups with the research team to develop the patient-reported QIs for patients with OA treated in primary care. The discussion groups took place over a three year period from 2009–2012. The objectives of the discussion groups were i) to understand the aims of MOSAICS and roles and expectations of the RUG members, ii) to identify important and relevant quality indicators for patients with OA when consulting in primary care, and iii) to develop wording and response options for a self-report OA QI (UK) questionnaire to assess the identified quality indicators (Fig. 1).

  1. i)

    Understanding the aims of MOSAICS and roles and expectations of the RUG members

    In the first meeting, a plain English summary of the MOSAICS study was introduced to set the context for the meetings and outline roles of the RUG members.

  2. ii)

    Identifying important and relevant quality indicators of OA in primary care consultations from a patient’s perspective

    During facilitated discussions, RUG members identified factors they considered to be important to patients with OA consulting their general practitioner (GP) to help identify potential QIs for OA consultations. Each RUG member then ranked (1 to 5) the top five factors they considered the most important. Any factors not selected as ‘most important’ by at least one RUG member were excluded from further discussions.

    The research team then presented RUG members with five QIs identified from a previous systematic review [8] (Fig. 2). These QIs were selected on the basis of their relevance for the MOSAICS model OA consultation of supported self-management and were used to stimulate discussion within the RUG group. The QIs included whether during GP consultations, patients have been offered education and advice about their disease, exercise and weight loss, and offered pain relief in the form of paracetamol and topical (skin applied) non-steroidal anti-inflammatory drugs (NSAIDS). RUG members were asked to 1) consider each QI alongside the initial list of important factors when consulting their GP for OA and add other factors as necessary, and 2) suggest potential QIs and questions which could capture the quality of care from a patient perspective.

    Fig. 2
    figure 2

    Five quality indicators identified from a systematic review used to stimulate discussion with the RUG members

    Full size image
  3. iii)

    Developing the wording and response options for a self-report OA QI (UK) questionnaire

    Based on the list of important factors suggested by RUG members, an initial set of patient-reported OA QIs were generated. From this list, wording for questionnaire items to assess the QIs were drafted for an OA QI (UK) questionnaire. Over a further two meetings, RUG members and the research team worked together to refine and finalise the questions so they were suitable for use in a research trial. (See Fig. 3 for an outline of the process). Feedback and suggestions on the wording of the items and the scoring method (response options) were given on documents mailed to the RUG members between meetings.

    Fig. 3
    figure 3

    Overview of the OA QI (UK) questionnaire item development

    Full size image

Stage 2: comparison of the OA QI (UK) with the Norwegian OA-QI questionnaire

The RUG group and MOSAICS study team had not planned to test the measurement (or psychometric) properties of the OA QI (UK) questionnaire. As mentioned earlier in this article, a similar OA QI questionnaire was being developed for use in primary care in Norway (the Norwegian OA-QI) [29].

In order to establish the measurement properties of the OA QI (UK) questionnaire, the Norwegian developers (NO, KH) produced an English translation of a draft version of the Norwegian OA-QI for item content and scoring comparison. RUG members reviewed this and compared its content with the OA QI (UK) questionnaire during the third meeting. Based on this comparison, RUG members suggested how the OA QI (UK) questionnaire could be refined and modified to include items in the validated Norwegian OA-QI. Where QIs in each questionnaire were similar, RUG members considered the appropriateness of the wording used in the Norwegian OA-QI for potential use.

Stage 3: review of the finalised OA QI (UK) questionnaire

During the last meeting, the RUG reviewed, commented and suggested refinements to the OA QI (UK) questionnaire for primary care. RUG members were also asked to assess the face validity of the questionnaire by commenting further on its appearance and layout, ease of completion, and to identify anything ambiguous or difficult to understand. The research team also compared the content of the OA QI (UK) questionnaire with the other questionnaires and data collected in the MOSAICS to check for unnecessary duplication.

Results

Patient and public involvement

All of the RUG members (males 5, females 5) had OA and were aged 52–80 years. Four meetings were held during the three year period. All meetings were attended by between three and ten members.

Stage 1: development of the OA quality indicators

Identifying important and relevant quality indicators of OA in primary care consultations from a patient’s perspective

RUG members initially discussed and identified 30 factors considered important and relevant to patients consulting their GP for OA. From this list, RUG members chose 20 factors as ‘most important’ (Table 1), which were grouped into the following domains: information about OA; information about treatment for OA; information about self-management for OA; advice about using medications to relieve joint pain; information about exercise and activities; referrals to activity or exercise programmes, and the quality of the consultation with the GP. These domains formed an initial list of seven patient-reported QIs for OA consultations. Following the review of the five quality indicators identified from a systematic review [8], an additional four QIs appropriate from a patient perspective (‘patient has received a follow-up review of his/her joint problem’; ‘patient has received a referral for physiotherapy’; ‘patient has received advice about body weight and joint pain’, ‘patient has received a referral for weight loss services’) were added. Therefore, 11 unique QIs were identified (Table 1).

Table 1 Selection of factors considered most important to patients with OA consulting their GP
Full size table

Developing the wording and response options for a self-report OA QI (UK) questionnaire

Using the list of eleven self-reported QIs for OA consultations, RUG members suggested questionnaire wording and response options to assess each QI. RUG members stated that the 3-month recall period, as determined by the MOSAICS trial design, was an appropriate period to have had at least one consultation with their GP for a joint problem. They also suggested that the response options should be a simple 3-level response format for all questions for the draft questionnaire: “Yes”, “No”, and “Don’t know”.

The RUG members’ suggestions were drafted into a questionnaire. Views of the overall quality of the consultation were captured in other parts of the MOSAICS study, so this item was not included in the questionnaire. To avoid multiple dimensions of care within a single question, one QI (advice about using medications to relieve joint pain) was split into four questions, two QIs (information about self-management for OA; advice about exercise or activities) into three items, and one other QI (follow up review) was split into two questions, respectively.

The resultant draft OA QI (UK) questionnaire comprised 16 items. In the second group meeting, RUG members reviewed the draft questionnaire and worked with the research team to refine its content. RUG members provided further comments on the ease of understanding and relevance of the questions. They suggested wording for the questionnaire instructions and changes to improve the clarity, specificity and order of the questions. RUG members suggested changing the wording for the ‘don’t know’ response option where relevant if the respondent could not remember, if they had not received an aspect of care, or if the question was not applicable.

Stage 2: comparison of the OA QI (UK) with the Norwegian OA-QI questionnaire

After reviewing both questionnaires, RUG members suggested that the draft OA QI (UK) questionnaire and the draft Norwegian OA-QI were similar. Seven items in the draft OA QI (UK) questionnaire capturing six quality indicators: (information about OA, information about treatment for OA, information about self-managing OA, advice about exercise or activities for OA, referral to exercise or activity programmes for OA, advice about the use of medications to relieve joint pain) used comparable wording to those included in the draft Norwegian OA-QI (Table 2). Of these, RUG members and the research team agreed to retain the wording used in the draft OA QI (UK) questionnaire for three items and adopt all or some of wording in the draft Norwegian OA-QI for the other four items for the final questionnaire. Both questionnaires used similar three-level response options.

Table 2 Comparison of the draft OA QI (UK) and Norwegian OA-QI questionnaires
Full size table

RUG members recommended that a further seven items (capturing six QIs) included in the Norwegian OA-QI were relevant and should be added to the UK questionnaire. They suggested minor changes to wording to make them more appropriate for the UK (Table 2). Three other items from the Norwegian OA-QI questionnaire (advice about changing lifestyle; assessment of daily activities; assessment of pain) were captured elsewhere in the MOSAICS study and therefore not required for the OA QI (UK) questionnaire. Nine items from the draft OA QI (UK) that were not present in the Norwegian OA-QI were retained at this stage. Therefore, at the end of Stage 2, the draft version of the OA QI (UK) questionnaire contained 23 items (Fig. 3).

Stage 3: review of the finalised OA QI (UK) questionnaire

The iterative process of redrafting and reviewing the questionnaire continued into the fourth meeting until the RUG members and researchers agreed on the final draft version. Along with subtle changes to item wording suggested by RUG members, the research team and RUG members agreed to retain one item from the draft OA QI (UK) (on support for self-managing OA). Eight items from the draft OA QI (UK) (support from ‘surgery’ or other health care professionals to help you manage your joint problem; follow up review received (2 items); current participation in exercise; exercise programme suggested; referral for physiotherapy; advice about taking paracetamol received; advice about taking capsaicin cream received) were either covered elsewhere in the MOSAICS study [24], too generic or were too similar to other items (Table 2). Therefore, these eight items were not included in the final 15-item OA QI (UK) questionnaire (see Additional file 3). RUG members and researchers agreed that the length of the final version of the questionnaire was appropriate to capture important quality indicators of OA in primary care consultations from a patient’s perspective without overburdening those who complete it.

Discussion

This study describes the development of patient-reported quality indicators questionnaire for the primary care of osteoarthritis, which were derived by members of a Research User Group, working in partnership with researchers. The OA QI (UK) has been successfully used in a large randomized control trial of a new model of supported self-management of OA (the MOSAICS study) [22] and a study to audit the quality of OA primary care practice in the United Kingdom, Norway, Denmark and Portugal. While the full results of these studies are subject to other papers in production, the focus of this article is on the role and impact of PPI to develop the OA QI (UK).

The active, meaningful and on-going involvement of patients as partners in the research process is a strength of this study. The perspectives of patients may differ from the perspectives of healthcare professionals or information recorded by professionals in medical records [40, 41]. Therefore, the unique perspectives of patients with OA based on their experience of the condition and past consultations in primary care has enhanced the development of patient-centred quality indicators for use in OA primary care. We acknowledged that the PPI input in this study incorporated the perspectives of a small group of patients, as small as three people for one meeting. Also, the RUG membership was not greatly diverse, in terms of age, ethnicity, and physical abilities. While obtaining a range of perspectives is the objective of PPI in research and not necessarily ‘representativeness’, it is possible however that the OA QI (UK) does not cover the full range of quality indicators relevant to the population of patients with OA. Nevertheless, the sequential and iterative development of the OA QI (UK) allowed the researchers and RUG members to review and critique earlier suggestions made by the RUG.

The RUG group identified important factors related to the quality of OA care provided by a primary care healthcare professional and suggested item wording for a questionnaire. By comparing the OA QI (UK) questionnaire with a similar one developed in Norway, the RUG members helped redraft and refine the final questionnaire. The RUG collaborated with the research team throughout the development of the OA QI (UK) but were also involved on other aspects of the MOSAICS study such as developing a self-management guidebook for patients with OA and participant information sheets [33, 42]). Regular meetings were set up for these. Though there were extended gaps between meetings regarding the OA QI (UK) development, the timings of the meetings were governed by the MOSAICS study timeline. However, RUG members were provided with feedback of the meeting and given the opportunity to comment. This process built upon existing working relationships and trust between the RUG and researchers.

The research team embraced the contribution of the RUG members and implemented many of their suggestions. The concepts included in the finalised OA QI (UK) were generated by the RUG members. Working in collaboration the RUG members and the research team shaped the items in survey questions suitable for use in a research trial. The Chief Investigator did not make decisions on the final content of the questionnaire without the fully informed RUG and explained if information was already captured elsewhere in the MOSAICS study. For example, RUG members did identify eight other important and relevant QIs not included in the final version of the OA QI (UK). So, to avoid repetition and participant burden in the MOSAICS study, these eight items were not included in the final 15-item OA QI (UK) questionnaire. These decisions were fully explained to the RUG members. Therefore, the RUG members' contribution ensured that the resulting OA QI (UK) incorporated issues relevant to patients with OA, written in a language that patients found easy to understand.

The OA QI (UK) was developed to assess the uptake of treatment recommended by NICE [3] and complements the new NICE Quality Standards of Care for OA [15]. Using evidence from a systematic review [8], the OA QI (UK) is a 15-item questionnaire that covers treatments offered by healthcare professionals in primary care. The OA QI (UK) supports the recently published NICE Quality Standards of Care for OA [17] and the European Musculoskeletal Conditions Surveillance and Information Network (eumusc.net) recommendations for the OA Standards of Care across European member states [43]. For example, it captures five of the eight NICE quality statements either fully or partially from a patient’s viewpoint. The OA QI (UK) and the Norwegian OA-QI also sit alongside established outcome measures of OA management and provide process measures of the quality of OA care. The eumusc.net has developed OA health care quality indicators (HCQI-OA) and an accompanying audit tool for clinicians and health care providers [44]. The OA QI (UK) and HCQI-OA both include six similar quality indicators. This questionnaire, if used in routine practice, will have the patients’ perspectives embedded in an evaluation of care quality. Though developed for use across primary care settings, the OA QI (UK) may be further refined to meet the specific needs and priorities of local health care settings, if required.

Establishing the measurement properties of a questionnaire is an important step in its development. The use of scientifically sound and decision-relevant measures allows the collection of evidence on the benefits of intervention (or care practices) from a patients’ perspective [45]. The OA QI (UK) and the Norwegian OA-QI were developed in parallel. Given the similarity between the construct and wording of the two questionnaires and the direct additions of items from the Norwegian OA-QI, the UK version ‘adopted’ the measurement properties of the validated Norwegian version. The comparison of the questionnaires used a translated, draft version of the Norwegian OA-QI. However, the Norwegian OA-QI was further refined with some changes to the item wording before validity and reliability testing. Though they are very similar in content and wording, the finalised, validated Norwegian OA-QI was published after the OA QI (UK) was produced and implemented in the MOSAICS study. The measurement properties of the OA QI (UK) was not tested because the 14 (out of the 15 items) were identical or contained subtle changes in wording to items in the validated Norwegian questionnaire. Therefore, conducting a full validation study on the OA QI (UK) questionnaire was not justified at this stage. However, the assumption that the measurement properties of the two questionnaires are similar may need further exploration.

The overall positive feedback from RUG members on the Norwegian OA-QI enabled adaptations to the OA QI (UK) to be made with confidence. The development of the both questionnaires was coincidental. Although there was differences in how they were developed (one mainly patient-led and the other mainly researcher/clinician derived), this study has demonstrated that patients and researchers have similar expectations about what constitutes good quality care in OA in different European countries. It also highlights the value of the active, meaningful and useful contribution of patients in the research process. Furthermore, the consistency of quality indicators for OA consultations in two European countries has now provided a unique opportunity to compare QIs across European countries [46]. This may lead to the development of a single, combined questionnaire for use in routine clinical practice to assess the quality of OA care provided to patients.

Conclusion

This study has demonstrated that active involvement of patients in research, working in partnership with researchers, identified important and relevant OA quality indicators, and developed a self-reported questionnaire to measure them. The OA QI (UK) questionnaire aligns with current national and international standards and process measures of OA care (e.g. NICE, eumusc.net) and is consistent with quality indicators validated for Norwegian OA consultations. The development of two OA quality indicator questionnaires was coincidental but has led to further research to compare patient-reported OA QIs across European countries. Following this work, a single refined OA QI questionnaire for use in routine clinical practice is planned.