Redesign Based on Card Sorting: How Universally Applicable are Card Sort Results?
Card sort studies can facilitate developers to create an information structure for their website or application. In addition, this human-centered design method provides researchers with insights into the target group’s mental models regarding the information domain under study. In this method, participants sort cards, with excerpts of the website’s or information source’s information on them, into piles or groups. Even though the method lends itself for large numbers of participants, it can be difficult to include sufficient participants in a study to ensure generalizability among large user groups. Especially when the potential user group is heterogeneous, basing the information structure on a limited participant group may not always be valid. In this study, we investigate if card-sort results among one user group (nurses) are comparable to the results of a second (potential) user group (physicians/residents).
The results of a formative card sort study that were used to create an antibiotic information application are compared to the results of a second card sort study. This second study was conducted with the aim of redesigning the nurse-aimed information application to meet the (overlapping) needs of physicians. During the first card sort study, 10 nurses participated. In the second card sort study, 8 residents participated. The same set of 43 cards were used in both setups. These cards contain fragments of antibiotic protocols and reference documents that nurses and physicians use to be informed about the use and administration of antibiotics. The participants sorted the cards in individual sessions, into as many categories as they liked. The sorts of both user groups were analyzed separately. Dendrograms and similarity matrices were generated using the Optimal Sort online program. Based on the matrices, clusters were identified by two independent researchers. On these resulting clusters of cards, overlap scores were calculated (between nurse and resident clusters). Differences are compared.
The results show that overall, residents reached higher agreement than the nurses. Some overlap between categories is observed in both card sort data matrices. Based on the nurses’ data, more and more specific clusters were created (which in part were observed in the larger residents’ clusters).
Based on our findings we conclude that a redesign may not be necessary. Especially when the target group with the lowest prior knowledge levels of the information domain is included in the card sort study, the results can be translated to other groups as well. However, groups with little knowledge will more likely result in lower agreement in the card sorts. Therefore, a larger sample and/or including participants with low and high knowledge of the information domain is advisable.
KeywordsCard sorting Re-design Human-centered design Information architecture
We thank the nurses and medical residents of the Medisch Spectrum Twente hospital, in Enschede, the Netherlands, for participating in this study. This study was conducted within the Interreg IVa-funded project EurSafety Heath-net (III-1-02 = 73). This is a Dutch-German cross-border network supported by the European Commission, the Federal States of Nordrhein-Westfalen and Niedersachsen(Germany), and the Dutch provinces of Overijssel, Gelderland, and Limburg. Part of the study was executed within a project (Google Glass For VIPS), funded as a Tech4People 2015 grant by the faculty BMS of the University of Twente.
- 2.Morville, P., O’reilly, T.: Information Architecture for the World Wide Web. O’Reilly, Sebastopol (2007)Google Scholar
- 4.Verhoeven, F., Karreman, J., Bosma. A., Hendrix, H.R.M., van Gemert-Pijnen, L.E.W.C.: Toward improved education of the public about methicillin-resistant Staphylococcus aureus: a mental models approach. Inter. J. Infect. Control 6(1) (2010)Google Scholar
- 5.Verhoeven, F., Hendrix, R.M., Daniels-Haardt, I., Friedrich, A.W., Steehouder, M.F., van Gemert-Pijnen, J.E.: The development of a web-based information tool for cross-border prevention and control of Methicillin Resistant Staphylococcus aureus. Inter. J. Infect. Control 4(1), 11 (2008)CrossRefGoogle Scholar
- 7.Wentzel, J., van Drie-Pierik, R., Nijdam, L., Geesing, J., Sanderman, R., van Gemert-Pijnen, J.: Antibiotic information application offers nurses quick support. Am. J. Inf. Control (2016). (in press)Google Scholar
- 8.Optimal Workshop - Optimal sort. https://www.optimalworkshop.com/
- 10.Nurse card sort study results: similarity matrix and dendrogram. https://www.optimalworkshop.com/optimalsort/mrsa-net/68158f/shared-results/ba49d9fe334fe8f3ba780902dc37e6a8
- 11.Resident card sort study results: similarity matrix and dendrogram. https://www.optimalworkshop.com/optimalsort/mrsa-net/b75e78/shared-results/805474e2179ad3089f370cbb6c73eb58
- 13.Nielsen, J.: Card Sorting: How Many Users to Test (2004). http://www.useit.com/alertbox/20040719html. Accessed 2 March 2015