Skip to main content

Advertisement

SpringerLink
Log in

Implementing Patient-Reported Outcome Measures into Clinical Practice Across NSW: Mixed Methods Evaluation of the First Year

  • Published:
Applied Research in Quality of Life Aims and scope Submit manuscript

Abstract

Use of patient-reported measures (PRMs) in healthcare is increasing internationally. In New South Wales (NSW) Australia the implementation of a system-wide PRMs program is underway. This program is an enabler for value based healthcare. We report an evaluation (2015/16) of this program’s first year. We evaluated the program using mixed methods. People living with chronic conditions from 18 sites across NSW completed PROMIS10 at all sites; DASS21 and CAT administered at selected sites depending on patient cohorts. PRM completion rates and mean scores were calculated. Stakeholder interviews were analysed using thematic analysis. PRM completion rates were high at baseline (69%) but suboptimal at follow-up time-points. Mean scores indicated people with back pain had worse physical health and stress compared to those with other conditions. People with alcohol or drug problems had worse mental health, disease symptoms, depression and anxiety compared to other conditions. Stakeholders reported collection of PRMs could improve health outcomes for patients, enhance service delivery, and reduce avoidable hospital admissions. Routine collection and use of PRMs can be implemented across NSW care settings. Several challenges need to be overcome to realise the full benefits of the PRM program and to improve follow-up completion rates.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. Names of authors and funders replaced with XXX for blind manuscript

References

  • Aaronson, N.K., Elliott, T.E., Greenhalgh, J., Halyard, M.Y., Hess, R., Miller, D., Reeve, B.B., Santana, M., Snyder, C. (2015). User’s Guide to Implementing Patient-Reported Outcomes Assessment in Clinical Practice. http://www.isoqol.org/UserFiles/2015UsersGuide-Version2.pdf. Accessed 11 March 2019.

  • Agency for Clinical Innovation (2019): Patient Reported Measures: Outcomes that Matter to Patients. https://www.aci.health.nsw.gov.au/make-it-happen/prms. Accessed 11 March 2019.

  • Australian Bureau of Statistics (2019): 2017–18 National Health Survey (NHS). https://www.abs.gov.au/ausstats/abs@.nsf/mf/4364.0.55.001. Accessed 16 Oct 2019.

  • Australian Institute of Health and Welfare (2018): Mental health services—in brief 2018. https://www.aihw.gov.au/getmedia/0e102c2f-694b-4949-84fb-e5db1c941a58/aihw-hse-211.pdf.aspx?inline=true. Accessed 16 Oct 2019.

  • Barbera, L., Sutradhar, R., Howell, D., Sussman, J., Seow, H., Dudgeon, D., Atzema, C., Earle, C., Husain, A., Liu, Y., & Krzyzanowska, M. K. (2015). Does routine symptom screening with ESAS decrease ED visits in breast cancer patients undergoing adjuvant chemotherapy? Supportive Care in Cancer, 23(10), 3025–3032. https://doi.org/10.1007/s00520-015-2671-3.

    Article  Google Scholar 

  • Basch, E., Deal, A. M., Dueck, A. C., Scher, H. I., Kris, M. G., Hudis, C., & Schrag, D. (2017). Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine Cancer treatment. JAMA, 318(2), 197–198. https://doi.org/10.1001/jama.2017.7156.

    Article  Google Scholar 

  • Cella, D., Hahn, E.A., Jensen, S.E., et al. (2015): Patient-Reported Outcomes in Performance Measurement. In: Method and Mode of Administration, Data Collection, and Analysis. RTI Press, Research Triangle Park (NC).

  • Chen, J., Ou, L., & Hollis, S. J. (2013). A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Services Research, 13, 211. https://doi.org/10.1186/1472-6963-13-211.

    Article  Google Scholar 

  • Cheung, N.W., Crampton, M., Nesire, V., Hng, T.-M., Chow, C.K., on behalf of the Western Sydney Integrated Care Program Investigators (2019): Model for integrated care for chronic disease in the Australian context: Western Sydney Integrated Care Program. Australian Health Review. https://doi.org/10.1071/AH18152

  • Cleeland, C. S., Wang, X. S., Shi, Q., Mendoza, T. R., Wright, S. L., Berry, M. D., Malveaux, D., Shah, P. K., Gning, I., Hofstetter, W. L., Putnam Jr., J. B., & Vaporciyan, A. A. (2011). Automated symptom alerts reduce postoperative symptom severity after cancer surgery: A randomized controlled clinical trial. Journal of Clinical Oncology : Official Journal of the American Society of Clinical Oncology, 29(8), 994–1000. https://doi.org/10.1200/jco.2010.29.8315.

    Article  Google Scholar 

  • Denis, F., Lethrosne, C., Pourel, N., Molinier, O., Pointreau, Y., Domont, J., Bourgeois, H., Senellart, H., Trémolières, P., Lizée, T., Bennouna, J., Urban, T., El Khouri, C., Charron, A., Septans, A.-L., Balavoine, M., Landry, S., Solal-Céligny, P., Letellier, C. (2017). Randomized Trial Comparing a Web-Mediated Follow-up With Routine Surveillance in Lung Cancer Patients. Journal of the National Cancer Institute 109(9). https://doi.org/10.1093/jnci/djx029

  • Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005). Synthesising qualitative and quantitative evidence: A review of possible methods. Journal of Health Services Research & Policy, 10(1), 45–53. https://doi.org/10.1177/135581960501000110.

    Article  Google Scholar 

  • Fairclough, D. L., Peterson, H. F., Cella, D., & Bonomi, P. (1998a). Comparison of several model-based methods for analysing incomplete quality of life data in cancer clinical trials. Statistics in Medicine, 17(5–7), 781–796.

    Article  Google Scholar 

  • Fairclough, D. L., Peterson, H. F., & Chang, V. (1998b). Why are missing quality of life data a problem in clinical trials of cancer therapy? Statistics in Medicine, 17(5–7), 667–677.

    Article  Google Scholar 

  • Foster, A., Croot, L., Brazier, J., Harris, J., & O'Cathain, A. (2018). The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: A systematic review of reviews. Journal of Patient-Reported Outcomes, 2, 46. https://doi.org/10.1186/s41687-018-0072-3.

    Article  Google Scholar 

  • Glickman, S. W., Ndubuizu, A., Weinfurt, K. P., Hamilton, C. D., Glickman, L. T., Schulman, K. A., & Cairns, C. B. (2011). Perspective: The case for research justice: Inclusion of patients with limited English proficiency in clinical research. Academic Medicine : Journal of the Association of American Medical Colleges, 86(3), 389–393. https://doi.org/10.1097/ACM.0b013e318208289a.

    Article  Google Scholar 

  • Goldstein, D., Bell, M. L., Butow, P., Sze, M., Vaccaro, L., Dong, S., Liauw, W., Hui, R., Tattersall, M., Ng, W., Asghari, R., Steer, C., Vardy, J., Parente, P., Harris, M., Karanth, N. V., King, M., Girgis, A., Eisenbruch, M., & Jefford, M. (2014). Immigrants' perceptions of the quality of their cancer care: an Australian comparative study, identifying potentially modifiable factors. Annals of Oncology: Official Journal of the European Society for Medical Oncology, 25(8), 1643–1649. https://doi.org/10.1093/annonc/mdu182.

    Article  Google Scholar 

  • Greenhalgh, J., & Meadows, K. (1999). The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: A literature review. Journal of Evaluation in Clinical Practice, 5(4), 401–416.

    Article  Google Scholar 

  • Greenhalgh, T., Jackson, C., Shaw, S., & Janamian, T. (2016). Achieving research impact through co-creation in community-based health services: Literature review and case study. The Milbank Quarterly, 94(2), 392–429. https://doi.org/10.1111/1468-0009.12197.

    Article  Google Scholar 

  • Greenhalgh, J., Dalkin, S., Gooding, K., Gibbons, E., Wright, J., Meads, D., Black, N., Valderas, J.M., Pawson, R. (2017): Functionality and feedback: a realist synthesis of the collation, interpretation and utilisation of patient-reported outcome measures data to improve patient care. Health Services and Delivery Research, 5(2). doi:https://doi.org/10.3310/hsdr05020

  • Hays, R. D., Bjorner, J. B., Revicki, D. A., Spritzer, K. L., & Cella, D. (2009). Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Quality of Life Research: an International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 18(7), 873–880. https://doi.org/10.1007/s11136-009-9496-9.

    Article  Google Scholar 

  • Health and Social Policy (2018): NSW Health Strategic Framework for Integrating Care. https://www.health.nsw.gov.au/integratedcare/Publications/strategic-framework-for-integrating-care.PDF. Accessed 11 March 2019.

  • Ishaque, S., Karnon, J., Chen, G., Nair, R., & Salter, A. B. (2019). A systematic review of randomised controlled trials evaluating the use of patient-reported outcome measures (PROMs). Quality of Life Research: an International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 28(3), 567–592. https://doi.org/10.1007/s11136-018-2016-z.

    Article  Google Scholar 

  • Jang, S.M., Kim, K.U., Na, H.J., Song, S.E., Lee, S.H., Lee, H., Kim, Y.S., Lee, M.K., Park, H.-K. (2019). Depression is a major determinant of both disease-specific and generic health-related quality of life in people with severe COPD. Chronic Respiratory Disease 16. https://doi.org/10.1177/1479972318775422

  • Jones, P. W., Harding, G., Berry, P., Wiklund, I., Chen, W. H., & Kline Leidy, N. (2009). Development and first validation of the COPD assessment test. The European Respiratory Journal, 34(3), 648–654. https://doi.org/10.1183/09031936.00102509.

    Article  Google Scholar 

  • Lovibond, S. H., & Lovibond, P. F. (1995). Manual for the depression anxiety stress scales (2nd ed.). Sydney: Psychology Foundation.

    Google Scholar 

  • Mejdahl, C. T., Schougaard, L. M. V., Hjollund, N. H., Riiskjaer, E., & Lomborg, K. (2018). Exploring organisational mechanisms in PRO-based follow-up in routine outpatient care - an interpretive description of the clinician perspective. BMC Health Services Research, 18(1), 546. https://doi.org/10.1186/s12913-018-3352-y.

    Article  Google Scholar 

  • Mercieca-Bebber, R., Palmer, M. J., Brundage, M., Calvert, M., Stockler, M. R., & King, M. T. (2016). Design, implementation and reporting strategies to reduce the instance and impact of missing patient-reported outcome (PRO) data: A systematic review. BMJ Open, 6(6), e010938. https://doi.org/10.1136/bmjopen-2015-010938.

    Article  Google Scholar 

  • NSW Ministry of Health (2018): NSW integrated care journey. https://www.health.nsw.gov.au/integratedcare/Pages/Our-Plan.aspx. Accessed 11 March 2019.

  • OECD (2010): Improving value in health care. https://doi.org/10.1787/9789264094819-en.

  • Porter, I., Goncalves-Bradley, D., Ricci-Cabello, I., Gibbons, C., Gangannagaripalli, J., Fitzpatrick, R., Black, N., Greenhalgh, J., & Valderas, J. M. (2016). Framework and guidance for implementing patient-reported outcomes in clinical practice: Evidence, challenges and opportunities. Journal of Comparative Effectiveness Research, 5(5), 507–519. https://doi.org/10.2217/cer-2015-0014.

    Article  Google Scholar 

  • Snyder, C. F., Aaronson, N. K., Choucair, A. K., Elliott, T. E., Greenhalgh, J., Halyard, M. Y., Hess, R., Miller, D. M., Reeve, B. B., & Santana, M. (2012). Implementing patient-reported outcomes assessment in clinical practice: A review of the options and considerations. Quality of Life Research: an International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 21(8), 1305–1314. https://doi.org/10.1007/s11136-011-0054-x.

    Article  Google Scholar 

  • Solano, J. P., Gomes, B., & Higginson, I. J. (2006). A comparison of symptom prevalence in far advanced cancer, AIDS, heart disease, chronic obstructive pulmonary disease and renal disease. Journal of Pain and Symptom Management, 31(1), 58–69.

    Article  Google Scholar 

  • Stanton, R., Rosenbaum, S., Rebar, A., & Happell, B. (2019). Prevalence of chronic health conditions in Australian adults with depression and/or anxiety. Issues in Mental Health Nursing, 40(10), 902–907. https://doi.org/10.1080/01612840.2019.1613701.

    Article  Google Scholar 

  • Sze, M., Butow, P., Bell, M., Vaccaro, L., Dong, S., Eisenbruch, M., Jefford, M., Girgis, A., King, M., McGrane, J., Ng, W., Asghari, R., Parente, P., Liauw, W., Goldstein, D., & Psycho-oncology co-operative research group, C., linguistically diverse, T. (2015). Migrant health in cancer: Outcome disparities and the determinant role of migrant-specific variables. Oncologist, 20(5), 523–531. https://doi.org/10.1634/theoncologist.2014-0274.

    Article  Google Scholar 

  • Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8, 45. https://doi.org/10.1186/1471-2288-8-45.

    Article  Google Scholar 

  • Velikova, G., Booth, L., Smith, A. B., Brown, P. M., Lynch, P., Brown, J. M., & Selby, P. J. (2004). Measuring quality of life in routine oncology practice improves communication and patient well-being: A randomized controlled trial. Journal of Clinical Oncology : Official Journal of the American Society of Clinical Oncology, 22(4), 714–724. https://doi.org/10.1200/jco.2004.06.078.

    Article  Google Scholar 

Download references

Funding

This study was sponsored by the Agency of Clinical Innovation (ACI) with funding from the NSW Department of Health.

Author information

Authors and Affiliations

  1. University of Sydney, Faculty of Science, School of Psychology, Sydney, NSW, Australia

    Claudia Rutherford, Rachel Campbell & Madeleine King

  2. University of Sydney, Sydney Nursing School, Cancer Nursing Research Unit (CNRU), Sydney, NSW, Australia

    Claudia Rutherford

  3. Agency for Clinical Innovation, Sydney, NSW, Australia

    Melissa Tinsley & Robyn Speerin

  4. Western Sydney Local Health District, Sydney, NSW, Australia

    Linda Soars

  5. Northern Sydney Local health District, Sydney, NSW, Australia

    Anna Butcher

Authors
  1. Claudia Rutherford
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rachel Campbell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Melissa Tinsley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robyn Speerin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Linda Soars
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Anna Butcher
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Madeleine King
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claudia Rutherford.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 13 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rutherford, C., Campbell, R., Tinsley, M. et al. Implementing Patient-Reported Outcome Measures into Clinical Practice Across NSW: Mixed Methods Evaluation of the First Year. Applied Research Quality Life 16, 1265–1284 (2021). https://doi.org/10.1007/s11482-020-09817-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11482-020-09817-2

Keywords

Search

Navigation