Finding Health Care Usability and Safety Issues in Consumer Product Reviews

  • Helen FullerEmail author
  • Timothy Arnold
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 957)


Multiple techniques exist for eliciting usability and safety information related to product design. These methods work well for some types of use scenarios but have limitations. This work proposes a supplemental method of obtaining information relevant to usability and safety by systematically evaluating consumer reviews of medical devices using Natural Language Processing (NLP) techniques. Results include valuable information about categories of use, user priorities, and sources of user confusion. This project described a new method for gathering information from end users of medical devices. The information extracted may be useful to clinicians, manufacturers, and patient educators.


Human factors Usability Patient safety Safety Natural Language Processing (NLP) 



We would like to thank everybody at the VA National Center for Patient Safety for their commitment to patient safety. There were no relevant financial relationships or any source of support in the forms of grants, equipment, or drugs. The authors declare no conflict of interest. The opinions expressed in this article are those of the authors and do not necessarily represent those of the Veterans Administration.


  1. 1.
    Hornbæk, K.: Current practice in measuring usability: challenges to usability studies and research. Int. J. Hum. Comput. Stud. 64, 79–102 (2006)CrossRefGoogle Scholar
  2. 2.
    ISO 9241-11: Ergonomic requirements for office work with visual display terminals (vdts)–part 11: guidance on usability (1998)Google Scholar
  3. 3.
    ISO 9241-210: Human-centered design process for interactive systems (2010)Google Scholar
  4. 4.
    World Health Organization (WHO): Patient safety, 4 February 2019.
  5. 5.
    Seffah, A., Donyaee, M., Kline, R.B., Padda, H.K.: Usability measurement and metrics: a consolidated model. Softw. Qual. J. 14, 159–178 (2006)CrossRefGoogle Scholar
  6. 6.
    Hedegaard, S., Simonsen, J.G.: Extracting usability and user experience information from online user reviews. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2089–2098. ACM (2013)Google Scholar
  7. 7.
    Lahiri, S., Vydiswaran, V.V., Mihalcea, R.: Identifying usage expression sentences in consumer product reviews. In: Proceedings of the Eighth International Joint Conference on Natural Language Processing, pp. 394–403 (2017)Google Scholar
  8. 8.
    Zhang, J.Q., Craciun, G., Shin, D.: When does electronic word-of-mouth matter? A study of consumer product reviews. J. Bus. Res. 63, 1336–1341 (2010)CrossRefGoogle Scholar
  9. 9.
    Frost, J., Okun, S., Vaughan, T., Heywood, J., Wicks, P.: Patient-reported outcomes as a source of evidence in off-label prescribing: analysis of data from PatientsLikeMe. J. Med. Internet Res. 13, e6 (2011)CrossRefGoogle Scholar
  10. 10.
    Pearson, J.F., Brownstein, C.A., Brownstein, J.S.: Potential for electronic health records and online social networking to redefine medical research. Clin. Chem. 57, 196–204 (2011)CrossRefGoogle Scholar
  11. 11.
    Bian, J., Topaloglu, U., Yu, F.: Towards large-scale Twitter mining for drug-related adverse events. In: Proceedings of 2013 ACM International Workshop on Smart Health and Wellbeing, pp. 25–32 (2012)Google Scholar
  12. 12.
    Vydiswaran, V.V., Mei, Q., Hanauer, D.A., Zheng, K.: Mining consumer health vocabulary from community-generated text. In: AMIA Annual Symposium Proceedings, pp. 1150–1159 (2014)Google Scholar
  13. 13.
    Torii, M., Tilak, S.S., Doan, S., Zisook, D.S., Fan, J.W.: Mining health-related issues in consumer product reviews by using scalable text analytics. Biomed. Inform. Insights 8(S1), 1–11 (2016)Google Scholar
  14. 14.
    Bakiu, E., Guzman, E.: Which feature is unusable? Detecting usability and user experience issues from user reviews. In: IEEE 25th International Requirements Engineering Conference Workshops (REW) (2017)Google Scholar
  15. 15.
    Megaputer Intelligence: PolyAnalyst Professional: Version 6.5.2030. Data & Text Analysis Software. Bloomington, Indiana (2018)Google Scholar
  16. 16.
    Nielsen, J.: Enhancing the explanatory power of usability heuristics. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 152–158 (1994)Google Scholar
  17. 17.
    Anderson, E.W.: Customer satisfaction and word of mouth. J. Serv. Res. 1(1), 5–17 (1998)CrossRefGoogle Scholar
  18. 18.
    Hu, N., Pavlou, P.A., Zhang, J.: Can online reviews reveal a product’s true quality? Empirical findings and analytical modeling of online word-of-mouth communication. In: Proceedings of the 7th ACM Conference on Electronic Commerce, pp. 324–330. Ann Arbor, MI (2006)Google Scholar
  19. 19.
    Food and Drug Administration (FDA): Applying human factors and usability engineering to medical devices: guidance for Industry and Food and Drug Administration Staff (3 February 2016)
  20. 20.
    Wei, W., Gulla, J.A.: Sentiment learning on product reviews via sentiment ontology tree. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 404–413 (2010)Google Scholar

Copyright information

© This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply 2020

Authors and Affiliations

  1. 1.VA National Center for Patient SafetyAnn ArborUSA

Personalised recommendations