Clinical Usability Studies – Clash of Cultures? Study Design Proposal from Lessons Learned

Abstract

We can recently see an increased interest in real-world usability data from various global regulatory bodies for medical devices (MDs) and Software as a Medical Device (SaMD). Notably, the new European Medical Device Regulations from 2017 emphasizes the importance of clinical evaluation of the usability of MDs.

As promising as the combination of clinical trials in a real-world setting and usability tests in simulated use can be, it is challenging in practice to combine these well-established methodologies. This paper discusses the challenges around important “cultural” differences and reports on lessons learned. It highlights the opportunities and strengths that both worlds offer and provides guidance for an appropriate selection or combination of user centered design methodologies. The analysis of the user experience and usability research questions and their translation into a clinical protocol is a key element.

In most cases, the usability of an MD or SaMD can be sufficiently assessed with a human factors engineering evaluation (e.g., a simulated-use usability test), which can provide evidence of safe and effective use from an interactive perspective. However, some cases (e.g., for certain clinical claims) may require assessing use within a real-world environment, requiring the integration of methods from the human factors and clinical worlds. For such demanding cases, we propose a framework for a study design: a proposal on how to consider and integrate usability tests into a clinical trial.