Skip to main content
SpringerLink
Log in

Using a multi-stakeholder co-design process to develop a health service organisation-wide patient reported outcome measure collection system

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Limited examples exist of successful Patient Reported Outcome Measure (PROM) implementation across an entire healthcare organisation. The aim of this study was to use a multi-stakeholder co-design process to develop a PROM collection system, which will inform implementation of routine collection of PROMs across an entire healthcare organisation.

Methods

Co-design comprised semi-structured interviews with clinicians (n = 11) and workshops/surveys with consumers (n = 320). The interview guide with clinicians focused on their experience using PROMs, preferences for using PROMs, and facilitators/barriers to using PROMs. Co-design activities specific to consumers focused on: (1) how PROMs will be administered (mode), (2) when PROMs will be administered (timing), (3) who will assist with PROMs collection, and (4) how long a PROM will take to complete. Data were analysed using a manifest qualitative content analysis approach.

Results

Core elements identified during the co-design process included: PROMs collection should be consumer-led and administered by someone other than a clinician; collection at discharge from the healthcare organisation and at 3–6 months post discharge would be most suitable for supporting comprehensive assessment; PROMs should be administered using a variety of modes to accommodate the diversity of consumer preferences, with electronic as the default; and the time taken to complete PROMs should be no longer than 5–10 min.

Conclusion

This study provides new information on the co-design of a healthcare organisation-wide PROM collection system. Implementing a clinician and patient informed strategy for PROMs collection, that meets their preferences across multiple domains, should address known barriers to routine collection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data availability

All relevant data supporting the findings of this study are available within the manuscript and its supplementary information.

References

  1. Porter, M. (2009). A strategy for health care reform - toward a value-based system. New England Journal of Medicine, 361(2), 109–112.

    Article  CAS  PubMed  Google Scholar 

  2. Porter, M. (2010). What is value in health care? New England Journal of Medicine, 363(26), 2477–2481.

    Article  CAS  PubMed  Google Scholar 

  3. Porter, M., Larsson, S., & Lee, T. (2013). The strategy that will fix health care. Harvard Business Review, 91(10), 1–19.

    Google Scholar 

  4. Porter, M., & Teisberg, E. (2006). Redefining health care: creating value-based competition on results. Harvard business press.

    Google Scholar 

  5. Bodenheimer, T., & Sinsky, C. (2014). From triple to quadruple aim: Care of the patient requires care of the provider. Annals of Family Medicine, 12(6), 573–576.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Churruca, K., Pomare, C., Ellis, L. A., Long, J. C., Henderson, S. B., Murphy, L. E. D., Leahy, C. J., & Braithwaite, J. (2021). Patient-reported outcome measures (PROMs): A review of generic and condition-specific measures and a discussion of trends and issues. Health Expectations, 24(4), 1015–1024.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Black, N. (2013). Patient reported outcome measures could help transform healthcare. British Medical Journal, 346, f167.

    Article  PubMed  Google Scholar 

  8. Williams, K., Sansoni, J., Darcy, M., Grootemaat, P., & Thompson, C. (2016). Patient Reported Outcome Measures: Literature Review. 2016, Australian commission on safety and quality in health care.

  9. Sayah, F., Jin, X., & Johnson, J. (2021). Selection of patient-reported outcome measures (PROMs) for use in health systems. Journal of Patient Reported Outcomes. https://doi.org/10.1186/s41687-021-00374-2

    Article  PubMed  PubMed Central  Google Scholar 

  10. NHS. Patient Reported Outcome Measures (PROMs). 2021 [cited 2022 10 January]; Available from: https://www.england.nhs.uk/statistics/statistical-work-areas/proms/.

  11. Cancer, C.P.A. (2017). Effective, efficient, evolving: enhancing cancer control in Canada., CPAC: Toronto.

  12. Ernstsson, O., Janssen, M., & Heintz, E. (2020). Collection and use of EQ-5D for follow-up, decision-making, and quality improvement in health care - the case of the Swedish National Quality Registries. Journal of Patient Reported Outcomes. https://doi.org/10.1186/s41687-020-00231-8

    Article  PubMed  PubMed Central  Google Scholar 

  13. Peters, M., Crocker, H., Jenkinson, C., & Doll, H. (2014). Ray Fitzpatrick The routine collection of patient-reported outcome measures (PROMs) for long-term conditions in primary care: A cohort survey. BMJ Open, 4, e003968.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Gilbert, J., Howell, D., & King, S. (2012). Quality improvement in cancer symptom assessment and control: The provincial palliative care integration project (PPCIP. Journal of Pain and Symptom Management, 43, 663–678.

    Article  PubMed  Google Scholar 

  15. Rutherford, C., Campbell, R., Tinsley, M., Speerin, R., Soars, L., & Butcher, A. (2021). Madeleine king implementing patient-reported outcome measures into clinical practice across NSW: Mixed methods evaluation of the first year. Applied Research in Quality of Life, 16, 1265–1284.

    Article  Google Scholar 

  16. Foster, A., Croot, L., Brazier, J., Harris, J., & O’Cathain, A. (2018). The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: A systematic review of reviews. Journal of Patient Reported Outcomes. https://doi.org/10.1186/s41687-018-0072-3

    Article  PubMed  PubMed Central  Google Scholar 

  17. Stover, A. M., Haverman, L., van Oers, H. A., Greenhalgh, Joanne, Potter, C. M., Ahmed, S., Greenhalgh, J., Gibbons, E., Haverman, L., Manalili, K., Potter, C., Roberts, N., Santana, M., & Stover, A. M. (2021). Using an implementation science approach to implement and evaluate patient reported outcome measures (PROM) initiatives in routine care settings. Quality of Life Research, 30, 3015–33.

    Article  PubMed  Google Scholar 

  18. Briggs, M., Rethman, K., & Crookes, J. (2020). Implementing patient-reported outcome measures in outpatient rehabilitation settings: A systematic review of facilitators and barriers using the consolidated framework for implementation research. Archives of Physical Medicine and Rehabilitation, 101(10), 1796–1812.

    Article  PubMed  Google Scholar 

  19. O’Connell, S., Palmer, R., Withers, K., Saha, N., Puntoni, S., & Carolan-Rees, G. (2018). Requirements for the collection of electronic PROMS either “in clinic” or “at home” as part of the PROMs, PREMs and Effectiveness Programme (PPEP) in Wales: A feasibility study using a generic PROM tool. Pilot and Feasibility Studies. https://doi.org/10.1186/s40814-018-0282-8

    Article  PubMed  PubMed Central  Google Scholar 

  20. Williams, K. & C. Thompson. (2018) Patient-reported outcome measures: Stakeholder interviews, Australian Commission on Safety and Quality in Health Care,: Sydney.

  21. Greenhalgh, T., Jackson, C., Shaw, S., & Janamian, T. (2016). Achieving research impact through co-creation in community-based health services: Literature review and case study. The Milbank Quarterley, 94(2), 392–429.

    Article  Google Scholar 

  22. Dawda, P., & Knight, A. (2018). Experience Based Co-design: a toolkit for Australia. Prestantia Health: Canberra.

  23. Kyte, D., Anderson, N., Auti, R., Aiyegbusi, O. L., Bishop, J., Bissell, A., Brettell, E., Calvert, M., Chadburn, M., Cockwell, P., Dutton, M., Eddington, H., Forster, E., Hadley, G., Ives, N. J., Jackson, L., O’Brien, S., Price, G., Sharpe, K., … Williams, J. (2020). Development of an electronic patient reported outcome measure (ePROM) system to aid the management of patients with advanced chronic kidney disease. Journal of Patient-Reported Outcomes. https://doi.org/10.1186/s41687-020-00223-8

    Article  PubMed  PubMed Central  Google Scholar 

  24. Knowles, S. E., Ercia, A., Caskey, F., Rees, M., Farrington, K., & Van der Veer, S. N. (2021). Participatory co-design and normalisation process theory with staff and patients to implement digital ways of working into routine care: the example of electronic patient-reported outcomes in UK renal services. BMC Health Services Research. https://doi.org/10.1186/s12913-021-06702-y

    Article  PubMed  PubMed Central  Google Scholar 

  25. NSW Council of Social Services, Principles of co-design. 2017.

  26. Sanders, E., & Stappers, P. (2008). Co-creation and the new landscapes of design. International Journal of Cocreation in Design and the Arts, 4(1), 5–18.

    Google Scholar 

  27. Aitken, J., & Shackleton, D. (2014). Co-creation and co-design: Applied research methods in healthcare service design. Sage Publications.

    Book  Google Scholar 

  28. Australian Bureau of Statistics. 2016 Census Quickstats: Mornington Peninsula 2021; Available from: https://www.abs.gov.au/census/find-census-data/quickstats/2021/214.

  29. Snowdon, D. A., Srikanth, V., Beare, R., Marsh, L., Parker, E., Naude, K., & Andrew, N. E. (2023). A landscape assessment of the use of patient reported outcome measures in research, quality improvement and clinical care across a healthcare organisation. BMC Health Services Research. https://doi.org/10.1186/s12913-023-09050-1

    Article  PubMed  PubMed Central  Google Scholar 

  30. Andrew, N., Beare, R., & Ravipati, T. (2023). Developing a linked electronic health record derived data platform to support research into healthy ageing. International Journal of Population Data Science. https://doi.org/10.23889/ijpds.v8i1.2129

    Article  PubMed  PubMed Central  Google Scholar 

  31. Snowdon, D., Srikanth, V., & Beare, R. (2023). Acceptability of the routine use and collection of a generic patient reported outcome measure from the perspective of healthcare staff: A qualitative study. J Patient Rep Outcomes. https://doi.org/10.1186/s41687-023-00617-4

    Article  PubMed  PubMed Central  Google Scholar 

  32. Sayah, F. A., Lahtinen, M., Bonsel, G. J., Ohinmaa, A., & Johnson, J. A. (2021). A multi-level approach for the use of routinely collected patient-reported outcome measures (PROMs) data in healthcare systems. Journal of Patient Reported Outcomes, 5, 98.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Boyce, M., Browne, J., & Greenhalgh, J. (2014). The experiences of professionals with using information from patient-reported outcome measures to improve the quality of healthcare: A systematic review of qualitative research. BMJ Quality and Safety, 23, 508–518.

    Article  PubMed  Google Scholar 

  34. Meadows, K. (2011). Patient-reported outcome measures: An overview. British Journal of Community Nursing, 16(3), 146–151.

    Article  PubMed  Google Scholar 

  35. The EuroQol Group. (1990). EuroQol-a new facility for the measurement of health-related quality of life. Health Policy, 16(3), 199–208.

    Article  Google Scholar 

  36. Hudak, P., Amadio, P., & Bombardier, C. (1996). Development of an upper extremity outcome measure: The DASH (disabilities of the arm, shoulder and hand). American Journal of Ind Med, 29, 602–608.

    Article  CAS  Google Scholar 

  37. Snowdon, D. A., McGill, S., Altmann, C., Brooks, K., Everard, T., Le Fevre, K., & Andrew, N. E. (2022). Client and service factors associated with changes in health-related quality of life following community rehabilitation. Disability and Rehabilitation, 45, 512–522.

    Article  PubMed  Google Scholar 

  38. QSR International Pty Ltd. (2020) NVivo (released in March 2020).

  39. Bengtsson, M. (2016). How to plan and perform a qualitative study using content analysis. NursingPlus Open, 2, 8–14.

    Article  Google Scholar 

  40. Francis, J., Johnston, M., & Robertson, C. (2010). What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychology & Health, 25, 1229–1245.

    Article  Google Scholar 

  41. Heale, R., & Forbes, D. (2013). Understanding triangulation in research. Evidence-Based Nursing, 16, 98.

    Article  PubMed  Google Scholar 

  42. Brusco, N., Atkinson, V., & Woods, J. (2022). Implementing PROMS for elective surgery patients: feasibility, response rate, degree of recovery and patient acceptability. Journal of Patient Reported Outcomes. https://doi.org/10.1186/s41687-022-00483-6

    Article  PubMed  PubMed Central  Google Scholar 

  43. Price, C. M., de Amanda, C., Williams, C., Smith, B. H., & Bottle, A. (2018). Implementation of patient-reported outcomes (PROMs) from specialist pain clinics in England and Wales: Experience from a nationwide study. European Journal of Pain, 23, 1368–1377.

    Article  Google Scholar 

  44. Makhni, E. C., Swantek, A. J., Ziedas, A. C., Geoffrey Patterson, R., Allard, D., Day, C. S., & Betty, C. (2021). The Benefits of Capturing PROMs in the EMR. NEJM Catalyst Innovations in Care Delivery. https://doi.org/10.1056/CAT.21.0134

    Article  Google Scholar 

  45. Olalekan, L. (2020). Key methodological considerations for usability testing of electronic patient-reported outcome (ePRO) systems. Quality of Life Research, 29, 325–333.

    Article  Google Scholar 

  46. Aiyegbusi, O. L., Kyte, D., Cockwell, P., Marshall, T., Dutton, M., Walmsley-Allen, N., & Auti, R. (2018). Calvert M Development and usability testing of an electronic patient-reported outcome measure (ePROM) system for patients with advanced chronic kidney disease. Computers in Biology and Medicine, 101, 120–127.

    Article  PubMed  Google Scholar 

  47. Gabbard, J., McLouth, C., & Murea, M. (2020). Rapid electronic capturing of patient-reported outcome measures in older adults with end-stage renal disease: A feasibility study. American Journal of Hospice and Palliative Medicine, 38(5), 432–440.

    Article  PubMed  Google Scholar 

  48. The University of Melbourne. Co-Design and Implementation Effectiveness. 2022 [cited 2022 February]; Available from: https://clinicalresearch.mdhs.unimelb.edu.au/about-us/co-design-and-implementation-effectiveness.

Download references

Acknowledgements

We would like to thank the clinicians and consumers at Peninsula Health who participated in this study.

Funding

The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Author information

Authors and Affiliations

  1. National Centre for Healthy Ageing, 2 Hastings Road, Frankston, VIC, Australia

    Kim Naude, Nadine E. Andrew, Velandai Srikanth, Emily Parker, Lucy Marsh, Richard Beare & David A. Snowdon

  2. Academic Unit, Peninsula Health, Frankston, VIC, Australia

    Kim Naude, Nadine E. Andrew, Velandai Srikanth, Emily Parker, Lucy Marsh, Richard Beare & David A. Snowdon

  3. Peninsula Clinical School, Central Clinical School, Monash University, Melbourne, VIC, Australia

    Nadine E. Andrew, Velandai Srikanth, Richard Beare & David A. Snowdon

  4. Developmental Imaging, Murdoch Children’s Research Institute, Melbourne, VIC, Australia

    Richard Beare

  5. Action Lab, Department of Human Centered Computing, Monash University, Melbourne, VIC, Australia

    Roisin McNaney

Authors
  1. Kim Naude
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nadine E. Andrew
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Velandai Srikanth
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Emily Parker
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lucy Marsh
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Richard Beare
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Roisin McNaney
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. David A. Snowdon
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation and data collection was conducted by KN, EP and LM. Data analysis was performed by KN and DS. Data Triangulation was performed by KN, EP, LM, NA and DS. The first draft of the manuscript was written by KN and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kim Naude.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Ethical approval

This study was performed in line with the principles of the Declaration of Helsinki. Ethical approval for this study was obtained from the Peninsula Health Human Research Ethics Committee (AM/73610/PH-2021-281834(v1)).

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Naude, K., Andrew, N.E., Srikanth, V. et al. Using a multi-stakeholder co-design process to develop a health service organisation-wide patient reported outcome measure collection system. Qual Life Res 33, 619–636 (2024). https://doi.org/10.1007/s11136-023-03552-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-023-03552-5

Keywords

Search

Navigation