Skip to main content
Log in

Development and user experience testing of an electronic system for routine collection and use of electronic patient-reported outcome measures

  • Original Paper
  • Published:
Health and Technology Aims and scope Submit manuscript

Abstract

Electronic collection of patient-reported outcomes (e-PROM) allows accurate recording of data. They also enable the visualization of longitudinal trends in domain-specific scores for a patient, and may improve patient-physician communication. Several commercial offerings are available but deploying them in countries like India is challenging due to language barriers and literacy levels. Additionally, costs involved remain a major problem. We propose to develop an open-source tool to serve the same purpose. After an exhaustive requirement analysis for a minimum viable product, we decided to proceed with developing the system using an open-source content management system. Additional contributed modules like Webform and Media were used to provide additional functionality. Three tiers of user roles with role-specific privileges were defined. A preliminary user experience testing was done for the patient role. All requirements identified in the requirement analysis section have been met. The system allows users with a patient role to fill in questionnaires presented to them. To ensure that diverse groups of patients can be accommodated, patients are grouped according to disease groups. Health care workers can visualize the results of the questionnaire as well as develop new questionnaires using a graphical interface. Initial user experience testing shows that 85.7% of the users (playing the role of patients) were able to use the website without additional help. An open-source system to collect electronic PROM has been developed with localization in Indian languages. We aim to continue developing, validating and extending the system in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

Data Availability

Can be provided on reasonable request.

Code availability

Can be made available on reasonable request.

References

  1. Center for Drug Evaluation and Research (CDER), Center for Biologics Evaluation and Research (CBER), Center for Devices and Radiological Health (CDRH). Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims [Internet]. U.S. Department of Health and Human Services Food and Drug Administration; 2009, December. Available from: https://www.fda.gov/media/77832/download

  2. Weldring T, Smith SMS. Patient-Reported Outcomes (PROs) and Patient-Reported Outcome Measures (PROMs). Health Serv Insights. 2013;4(6):61–8.

    Google Scholar 

  3. Churruca K, Pomare C, Ellis LA, Long JC, Henderson SB, Murphy LED, et al. Patient-reported outcome measures (PROMs): A review of generic and condition-specific measures and a discussion of trends and issues. Health Expect. 2021;24(4):1015–24.

    Article  Google Scholar 

  4. Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.

  5. Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S. Patient reported outcome measures in practice. BMJ. 2015;350:g7818.

  6. Henry DH, Viswanathan HN, Elkin EP, Traina S, Wade S, Cella D. Symptoms and treatment burden associated with cancer treatment: results from a cross-sectional national survey in the U.S. Support Care Cancer. 2008;16(7):791–801.

  7. Carlson LE, Angen M, Cullum J, Goodey E, Koopmans J, Lamont L, et al. High levels of untreated distress and fatigue in cancer patients. Br J Cancer. 2004;90(12):2297–304.

    Article  Google Scholar 

  8. Knight TG, Deal AM, Dusetzina SB, Muss HB, Choi SK, Bensen JT, et al. Financial Toxicity in Adults With Cancer: Adverse Outcomes and Noncompliance. J Oncol Pract. 2018;JOP1800120.

  9. Sanson-Fisher R, Girgis A, Boyes A, Bonevski B, Burton L, Cook P. The unmet supportive care needs of patients with cancer. Supportive Care Review Group Cancer. 2000;88(1):226–37.

    Google Scholar 

  10. Statistics. Statistics » Patient Reported Outcome Measures (PROMs) [Internet]. [cited 2020 Apr 7]. Available from: https://www.england.nhs.uk/statistics/statistical-work-areas/proms/

  11. Porter I, Gonçalves-Bradley D, Ricci-Cabello I, Gibbons C, Gangannagaripalli J, Fitzpatrick R, et al. Framework and guidance for implementing patient-reported outcomes in clinical practice: evidence, challenges and opportunities. J Comp Eff Res. 2016;5(5):507–19.

    Article  Google Scholar 

  12. Basch E, Deal AM, Kris MG, Scher HI, Hudis CA, Sabbatini P, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol. 2016;34(6):557–65.

    Article  Google Scholar 

  13. Basch E, Deal AM, Dueck AC, Scher HI, Kris MG, Hudis C, et al. Overall Survival Results of a Trial Assessing Patient-Reported Outcomes for Symptom Monitoring During Routine Cancer Treatment. JAMA. 2017;318(2):197–8.

    Article  Google Scholar 

  14. Gibbons C, Porter I, Gonçalves-Bradley DC, Stoilov S, Ricci-Cabello I, Tsangaris E, et al. Routine provision of feedback from patient-reported outcome measurements to healthcare providers and patients in clinical practice. Cochrane Database Syst Rev. 2021;10:CD011589.

  15. Lizán L, Pérez-Carbonell L, Comellas M. Additional Value of Patient-Reported Symptom Monitoring in Cancer Care: A Systematic Review of the Literature. Cancers [Internet]. 2021;13(18). Available from: https://doi.org/10.3390/cancers13184615

  16. Coons SJ, Eremenco S, Lundy JJ, O’Donohoe P, O’Gorman H, Malizia W. Capturing Patient-Reported Outcome (PRO) Data Electronically: The Past, Present, and Promise of ePRO Measurement in Clinical Trials. Patient. 2015;8(4):301–9.

    Article  Google Scholar 

  17. Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health. 2008;11(2):322–33.

    Article  Google Scholar 

  18. Rutherford C, Costa D, Mercieca-Bebber R, Rice H, Gabb L, King M. Mode of administration does not cause bias in patient-reported outcome results: a meta-analysis. Qual Life Res. 2016;25(3):559–74.

    Article  Google Scholar 

  19. Assessment Center [Internet]. [cited 2020 Apr 7]. Available from: https://www.assessmentcenter.net/

  20. Ayva ‐ Bravado Health [Internet]. [cited 2020 Apr 7]. Available from: https://www.bravadohealth.com/ayva/

  21. About Us | Epic [Internet]. [cited 2020 Apr 7]. Available from: https://www.epic.com/about

  22. PatientIQ [Internet]. [cited 2020 Apr 7]. Available from: https://www.patientiq.io/

  23. PRO monitoring [Internet]. [cited 2020 Apr 13]. Available from: https://ches.eortc.be/cms/routine.php

  24. Rosett HA, Herring K, Ratliff W, Koontz BF, Zafar Y, LeBlanc TW. Integration of electronic patient-reported outcomes into clinical workflows within the Epic electronic medical record. J Clin Orthod. 2019;37(31_suppl):102–102.

  25. openPRO [Internet]. Github; [cited 2020 Apr 7]. Available from: https://github.com/goinvo/openPRO

  26. Drupal - Open Source CMS [Internet]. Drupal.org. 2018 [cited 2020 Apr 7]. Available from: https://www.drupal.org/home

  27. Drupal 8 translation status | Translations [Internet]. [cited 2020 Apr 7]. Available from: https://localize.drupal.org/translate/drupal8?sort=asc&order=Language

  28. Charts [Internet]. Drupal.org. 2008 [cited 2020 Apr 7]. Available from: https://www.drupal.org/project/charts

  29. Group [Internet]. Drupal.org. 2011 [cited 2020 Apr 7]. Available from: https://www.drupal.org/project/group

  30. Media [Internet]. Drupal.org. 2005 [cited 2020 Apr 7]. Available from: https://www.drupal.org/project/media

  31. Webform [Internet]. Drupal.org. 2004 [cited 2020 Apr 7]. Available from: https://www.drupal.org/project/webform

  32. Lingotek Translation [Internet]. Drupal.org. 2011 [cited 2020 Apr 7]. Available from: https://www.drupal.org/project/lingotek

  33. MySQL [Internet]. [cited 2020 Apr 7]. Available from: https://www.mysql.com/

  34. PHP: Hypertext Preprocessor [Internet]. [cited 2020 Apr 7]. Available from: https://www.php.net/

  35. NGINX | High Performance Load Balancer, Web Server, & Reverse Proxy [Internet]. NGINX. [cited 2020 Apr 7]. Available from: https://www.nginx.com/

  36. Introduction - Composer [Internet]. [cited 2020 Apr 7]. Available from: https://getcomposer.org/doc/00-intro.md

  37. Honavar SG. Electronic medical records - The good, the bad and the ugly. Indian J Ophthalmol. 2020;68(3):417–8.

    Article  Google Scholar 

  38. Best Practices for a HIPAA Compliant Drupal Platform [Internet]. Drupal.org. 2021 [cited 2022 Feb 7]. Available from: https://www.drupal.org/association/supporters/blog/best-practices-for-a-hipaa-compliant-drupal-platform

Download references

Acknowledgements

We would like to thank all the persons who had expressed their interest in participating in the user experience testing. The full list is available at https://astu.pw/node/14. Additionally, we would like to thank the developers of the Drupal CMS and the contributed modules, whose contribution allowed us to develop this system with minimal coding.

Funding

None.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Santam Chakraborty.

Ethics declarations

Consent to participate

All participants consented to participate in the study voluntarily and signed an electronic consent.

Conflicts of interest

None.

Research involving human and animal participants

No animals were involved in the study. Participants in the user experience testing were volunteers who responded to our request on social media as mentioned above. No identifying information was collected during the testing process. The instructions for the testing process were sent to participants individually as a document. The English version of this instruction is provided as an Appendix (Appendix II) with this manuscript. As this study was not performed on patients or healthy volunteers registered at the institute, an institutional review board was not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (MP4 33520 KB)

Supplementary file2 (DOCX 288 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chakraborty, S., Mallick, I., Bhattacharyya, T. et al. Development and user experience testing of an electronic system for routine collection and use of electronic patient-reported outcome measures. Health Technol. 12, 443–452 (2022). https://doi.org/10.1007/s12553-022-00647-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12553-022-00647-w

Keywords

Navigation