Abstract
Medical doctors need rapid and accurate answers, which they cannot easily find with current search systems. This paper describes a formative evaluation of a comprehensive search system for medical professionals. The study was designed to guide system development. The system features included search in text and 2D images, machine translated summaries of search results, as well as query disambiguation and suggestion features, and a comprehensive search user interface. The study design emphasizes qualitative user feedback, based on realistic simulated work tasks and data collection with spontaneous and prompted self-report, written and spoken feedback in response to questionnaires, was well as audio and video recordings, and log files. Results indicate that this is a fruitful approach to uncovering problems and eliciting requirements that would be harder to find in a component-based evaluation testing each feature separately.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hersh, W.R., Hickam, D.H.: How well do physicians use electronic information retrieval systems? a framework for investigation and systematic review. JAMA 280(15), 1347–1352 (1998)
Hoogendam, A., Stalenhoef, A.F.H., de Vries Robbé, P.F., Overbeke, A.J.P.M.: Answers to Questions Posed During Daily Patient Care Are More Likely to Be Answered by UpToDate Than PubMed. J. Med. Internet Res. 10(4) (2008)
Ely, J.W., Osheroff, J.A., Maviglia, S.M., Rosenbaum, M.E.: Patient-care questions that physicians are unable to answer. JAMA 14, 407–414 (2007)
Kelly, D.: Methods for evaluating interactive information retrieval systems with users. Foundations and Trends in Information Retrieval, vol. 3(1-2) (2009)
Zhang, P., Plettenberg, L., Klavans, J.L., Oard, D.W., Soergel, D.: Task-based interaction with an integrated multilingual, multimedia information system: A formative evaluation. In: JCDL 2007, Vancouver, Canada, June 17-22. ACM (2007)
Beckers, T., Dungs, S., Fuhr, N., Jordan, M., Kriewel, S.: ezDL: An interactive search and evaluation system. In: SIGIR 2012 Workshop on Open Source Information Retrieval, OSIR 2012 (2012)
Cunningham, H., Tablan, V., Roberts, I., Greenwood, M.A., Aswani, N.: Information Extraction and Semantic Annotation for Multi-Paradigm Information Management. In: Lupu, M., Mayer, K., Tait, J., Trippe, A.J. (eds.) Current Challenges in Patent Information Retrieval. The Information Retrieval Series, vol. 29. Springer (2011)
García Seco de Herrera, A., Markonis, D., Eggel, I., Müller, H.: The medGIFT Group in ImageCLEFmed 2012. In: CLEF (Online Working Notes/Labs/Workshop) (2012)
Kiryakov, A., Ognyanov, D., Manov, D.: OWLIM – A Pragmatic Semantic Repository for OWL. In: Dean, M., Guo, Y., Jun, W., Kaschek, R., Krishnaswamy, S., Pan, Z., Sheng, Q.Z. (eds.) WISE 2005 Workshops. LNCS, vol. 3807, pp. 182–192. Springer, Heidelberg (2005)
Health On the Net Foundation: The HON Code of Conduct for medical and health Web sites, HONcode (2013), http://www.hon.ch/HONcode/Patients/Conduct.html
Kritz, M., Gschwandtner, M., Stefanov, V., Hanbury, A., Samwald, M.: Utilization and perceived problems of online medical resources and search tools among different groups of European physicians. J. Med. Internet Res. (forthcoming, 2013)
Borlund, P.: The IIR evaluation model: A framework for evaluation of interactive information retrieval systems. Information Research 8(3), 152 (2003)
Brooke, J.: SUS: A “quick and dirty” usability scale. In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, A.L. (eds.) Usability Evaluation in Industry. Taylor and Francis, London (1996)
TechSmith Corporation: Morae usability testing software, version 3.3.2 (2013), http://www.techsmith.com/morae.html
Hornbaek, K., Law, E.L.C.: Meta-analysis of correlations among usability measures. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 617–626 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Stefanov, V., Sachs, A., Kritz, M., Samwald, M., Gschwandtner, M., Hanbury, A. (2013). A Formative Evaluation of a Comprehensive Search System for Medical Professionals. In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds) Information Access Evaluation. Multilinguality, Multimodality, and Visualization. CLEF 2013. Lecture Notes in Computer Science, vol 8138. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40802-1_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-40802-1_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40801-4
Online ISBN: 978-3-642-40802-1
eBook Packages: Computer ScienceComputer Science (R0)