Abstract
The selection of participants for usability assessment, together with the minimum number of subjects required to obtain a set of reliable data, is a hot topic in Human Computer Interaction (HCI). Albeit, prominent contributions through the application of different p estimation models argued that five users provide a good benchmark when seeking to discover interaction problems a lot of studies have complained this five-user assumption. The sample size topic is today a central issue for the assessment of critical-systems, such as medical devices, because lacks in usability and, moreover, in the safety in use of these kind of products may seriously damage the final users. We argue that rely on one-size-fits-all solutions, such as the five-user assumption (for websites) or the mandated size of 15 users for major group (for medical device) lead manufactures to release unsafe product. Nevertheless, albeit there are no magic numbers for determining “a priori” the cohort size, by using a specific procedure it is possible to monitoring the sample discovery likelihood after the first five users in order to obtain reliable information about the gathered data and determine whether the problems discovered by the sample have a certain level of representativeness (i.e., reliability). We call this approach “Grounded Procedure” (GP).The goal of this study is to present the GP assumptions and steps, by exemplifying its application in the assessment of a home medical device.
Chapter PDF
References
Streitz, N.: From cognitive compatibility to the disappearing computer: experience design for smart environments. In: Proceedings of the 15th European Conference on Cognitive Ergonomics: the Ergonomics of Cool Interaction, pp. 1–2. ACM, Funchal (2008)
Soylu, A., Causmaecker, P.D., Desmet, P.: Context and Adaptivity in Pervasive Computing Environments: Links with Software Engineering and Ontological Engineering. Journal of Software 4, 992–1013 (2009)
Herman, W.A., Devey, G.B.: Future Trends in Medical Device Technologies: A Ten-Year Forecast. Food and Drug Administration, Center for Devices and Radiological Health (2011)
IEC: IEC 62366: 2007 Medical devices – Application of usability engineering to medical devices. CEN, Brussels, BE (2007)
ISO: ISO 14971:2000 Medical devices – Application of risk management to medical devices. CEN, Brussels, BE (2000)
ISO: ISO 13485:2003 Medical devices – Quality management systems – Requirements for regulatory purposes. CEN, Brussels, BE (2003)
ISO: ISO 14971:2007 Medical devices – Application of risk management to medical devices. CEN, Brussels, BE (2007)
ISO: ISO 15223-1:2012 Medical devices – Symbols to be used with medical device labels, labelling and information to be supplied – Part 1: General requirements. CEN, Brussels, BE (2012)
ANSI/AAMI: HE75: Human factors engineering-Design of medical devices. Association for the Advancement of Medical Instrumentation, Arlington, VA (2009)
Martin, J., Norris, B.J., Murphy, E., Crowe, J.A.: Medical device development: The challenge for ergonomics. Applied Ergonomics 39, 271–283 (2008)
Money, A., Barnett, J., Kuljis, J., Craven, M., Martin, J., Young, T.: The role of the user within the medical device design and development process: medical device manufacturers’ perspectives. BMC Medical Informatics and Decision Making 11, 15 (2011)
Martin, J., Barnett, J.: Integrating the results of user research into medical device development: insights from a case study. BMC Medical Informatics and Decision Making 12, 74 (2012)
Heneghan, C., Thompson, M., Billingsley, M., Cohen, D.: Medical-device recalls in the UK and the device-regulation process: retrospective review of safety notices and alerts. BMJ Open 1 (2011)
Food and Drug Administration (FDA): Draft Guidance for Industry and Food and Drug Administration Staff - Applying Human Factors and Usability Engineering to Optimize Medical Device Design. U.S. Food and Drug Administration, Silver Spring, MD (2011)
Faulkner, L.: Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods 35, 379–383 (2003)
Nielsen, J.: http://www.useit.com/alertbox/20000319.html
Nielsen, J.: http://www.useit.com/alertbox/number-of-test-users.html
Nielsen, J., Landauer, T.K.: A mathematical model of the finding of usability problems. In: Proceedings of the INTERACT 1993 and CHI 1993 Conference on Human Factors in Computing Systems, pp. 206–213. ACM, Amsterdam (1993)
Virzi, R.A.: Streamlining the Design Process: Running Fewer Subjects. In: Proceedings of the Human Factors Society 34th Annual Meeting, vol. 34, pp. 291–294. ACM, Santa Monica (1990)
Virzi, R.A.: Refining the test phase of usability evaluation: how many subjects is enough? Human Factors 34, 457–468 (1992)
Borsci, S., Londei, A., Federici, S.: The Bootstrap Discovery Behaviour (BDB): a new outlook on usability evaluation. Cognitive Processing 12, 23–31 (2011)
Caulton, D.A.: Relaxing the homogeneity assumption in usability testing. Behaviour & Information Technology 20, 1–7 (2001)
Lewis, J.R.: Evaluation of Procedures for Adjusting Problem-Discovery Rates Estimated From Small Samples. International Journal of Human-Computer Interaction 13, 445–479 (2001)
Schmettow, M.: Heterogeneity in the usability evaluation process. In: Proceedings of the 22nd British HCI Group Annual Conference on People and Computers: Culture, Creativity, Interaction, vol. 1, pp. 89–98. British Computer Society, Liverpool (2008)
Schmettow, M.: Sample size in usability studies. Communications of the ACM 55, 64–70 (2012)
Turner, C.W., Lewis, J.R., Nielsen, J.: Determining Usability Test Sample Size. In: Karwowski, W. (ed.) International Encyclopedia of Ergonomics and Human Factors, vol. 2, pp. 3084–3088. CRC Press, Boca Raton (2006)
Lewis, J.R.: Validation of Monte Carlo estimation of problem discovery likelihood (Tech. Rep. No. 29.3357). IBM (2000)
Borsci, S., Macredie, R.D., Barnett, J., Martin, J., Kuljis, J., Young, T.: Reviewing and Extending the Five–user Assumption: A Grounded Procedure for Interaction Evaluation (manuscript submitted for publication, 2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Borsci, S., Martin, J.L., Barnett, J. (2013). A Grounded Procedure for Managing Data and Sample Size of a Home Medical Device Assessment. In: Kurosu, M. (eds) Human-Computer Interaction. Human-Centred Design Approaches, Methods, Tools, and Environments. HCI 2013. Lecture Notes in Computer Science, vol 8004. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39232-0_19
Download citation
DOI: https://doi.org/10.1007/978-3-642-39232-0_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39231-3
Online ISBN: 978-3-642-39232-0
eBook Packages: Computer ScienceComputer Science (R0)