Journal of Grid Computing

, Volume 10, Issue 4, pp 665–688 | Cite as

Assessing the Usability of a Science Gateway for Medical Knowledge Bases with TRENCADIS

  • Cristina Maestre
  • J. Damiàn Segrelles Quilis
  • Erik Torres
  • Ignacio Blanquer
  • Rosana Medina
  • Vicente Hernández
  • Luis Martí


Biomedical applications are often built on top of knowledge bases that contain medical images and clinical reports. Currently, these bases are being used to improve diagnosis, research and teaching, but in many cases, the infrastructure required has a prohibitive cost for many medical centres. However, resources can be attached from existing e-Science infrastructures. Therefore, many efforts have been made to establish best practices that allow the use of such infrastructures. However, e-Science relies on open, distributed, collaborative environments, built on top of very specialized technologies, such as Grid and Cloud computing, which require reasonable technical skills for their usage. Therefore, science gateways have become essential tools that assist users in interacting with e-Science applications. This paper describes TRENCADIS, a technology that supports the creation and operation of virtual knowledge bases. To this end, it provides developers with components and APIs for building secure data services that can be annotated and queried through ontology templates, based on DICOM and DICOM-SR. This technology was used in this paper to build a gateway for assisting diagnosis and research in breast cancer. We also present here the results of a study conducted to evaluate the gateway, from the point of view of the usability perceived by a group of physicians and radiologists.


DICOM DICOM-SR Knowledge base Usability 


  1. 1.
    Afgan, E., Goecks, J., Baker, D., Coraor, N., Team The Galaxy, Nekrutenko, A., Taylor, J.: Galaxy—a gateway to tools in e-Science. In: Yang, X., Wang, L., Jie, W. (eds.) Guide to e-Science: Next Generation Scientific Research and Discovery, pp. 145–180. Springer (2011)Google Scholar
  2. 2.
    Aisen, A.M., Broderick, L.S., Winer-Muram, H., Brodley, C.E., Kak, A.C., Pavlopoulou, C., Dy, J., Shyu, C.R., Marchiori, A.: Automated storage and retrieval of thin-section CT images to assist diagnosis: system description and preliminary assessment. Radiology 228(1), 265–270 (2003)CrossRefGoogle Scholar
  3. 3.
    Allcock, W.: GridFTP: protocol extensions to FTP for the Grid. Tech. rep., Argonne National Laboratory (2003).
  4. 4.
    Andronico, G., Ardizzone, V., Barbera, R., Becker, B., Bruno, R., Calanducci, A., Carvalho, D., Ciuffo, L., Fargetta, M., Giorgio, E., Rocca, G., Masoni, A., Paganoni, M., Ruggieri, F., Scardaci, D.: e-Infrastructures for e-Science: a global view. J. Grid Computing 9(2), 155–184 (2011). doi:10.1007/s10723-011-9187-y CrossRefGoogle Scholar
  5. 5.
    Bastien, J.M.C.: Usability testing: a review of some methodological and technical aspects of the method. Int. J. Med. Inform. 79(4), e18–e23 (2010)CrossRefGoogle Scholar
  6. 6.
    Bastien, J.M.C., Scapin, D.L.: Ergonomic Criteria for the Evaluation of Human-Computer Interfaces (1993)Google Scholar
  7. 7.
    Blanquer, I., Hernández, V., Segrelles, D., Torres, E.: Enhancing privacy and authorization control scalability in the Grid through ontologies. IEEE Trans. Inf. Technol. Biomed. 13(1), 16–24 (2009). doi:10.1109/TITB.2008.2003369 CrossRefGoogle Scholar
  8. 8.
    Blanquer, I., Hernández, V., Salavert, J., Segrelles, D.: Integrating TRENCADIS components in gLite to share DICOM medical images and structured reports. Stud. Health Technol. Inform. 159, 64–75 (2010)Google Scholar
  9. 9.
    Blanquer Espert, I., Hernández García, V., Meseguer Anastásio, F.J., Segrelles Quilis, J.D.: Content-based organisation of virtual repositories of DICOM objects. Future Gener. Comput. Syst. 25(6), 627–637 (2009). doi:10.1016/j.future.2008.12.004 CrossRefGoogle Scholar
  10. 10.
    Buyya, R., Yeo, C.S., Venugopal, S., Broberg, J., Brandic, I.: Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst. 25(6), 599–616 (2009). doi:10.1016/j.future.2008.12.001 CrossRefGoogle Scholar
  11. 11.
    Clunie, D.A.: DICOM Structured Reporting. PixelMed, Bangor (2000)Google Scholar
  12. 12.
    Cockburn, A.: Writing Effective Use Cases. Addison-Wesley Professional (2000)Google Scholar
  13. 13.
    Costa, C., Ferreira, C., Bastião, L., Ribeiro, L., Silva, A., Oliveira, J.: Dicoogle—an open source peer-to-peer PACS. J. Digit. Imaging 24(5), 848–856 (2011). doi:10.1007/s10278-010-9347-9 CrossRefGoogle Scholar
  14. 14.
    Cunsolo, V.D., Distefano, S., Puliafito, A., Scarpa, M.L.: GS3: a Grid storage system with security features. J. Grid Computing 8(3), 391–418 (2010). doi:10.1007/s10723-010-9157-9 CrossRefGoogle Scholar
  15. 15.
    Dumas, J.S., Redish, J.: A Practical Guide to Usability Testing. Intellect Ltd (1999)Google Scholar
  16. 16.
    European Commission: A digital agenda for Europe. Communication 5(245 final/2), 42 (2010)Google Scholar
  17. 17.
    European Commission: Riding the wave—how Europe can gain from the rising tide of scientific data. Final report of the High Level Expert Group on Scientific Data. Tech. Rep. October, European Commission (2010)Google Scholar
  18. 18.
    European Commission: Safeguarding privacy in a connected world a European data protection framework for the 21st century. Tech. rep., European Commission, Brussels, Belgium (2012)Google Scholar
  19. 19.
    Freund, J., Comaniciu, D., Ioannis, Y., Liu, P., McClatchey, R., Morley-Fletcher, E., Pennec, X., Pongiglione, G., Xiang, Zhou: Health-e-child: an integrated biomedical platform for Grid-based paediatric applications. Stud. Health Technol. Inform. 120, 259–270 (2006)Google Scholar
  20. 20.
    Fuster-Garcia, E., Navarro, C., Vicente, J., Tortajada, S., García-Gómez, J.M., Sáez, C., Calvar, J., Griffiths, J., Julià-Sapé, M., Howe, F.A., Pujol, J., Peet, A.C., Heerschap, A., Moreno-Torres, A., Martínez-Bisbal, M.C., Martínez-Granados, B., Wesseling, P., Semmler, W., Capellades, J., Majós, C., Alberich-Bayarri, A., Capdevila, A., Monleón, D., Martí-Bonmatí, L., Arús, C., Celda, B., Robles, M.: Compatibility between 3T 1H SV-MRS data and automatic brain tumour diagnosis support systems based on databases of 1.5T 1H SV-MRS spectra. Magma (N.Y. N.Y.) 24(1), 35–42 (2011). doi:10.1007/s10334-010-0241-8 Google Scholar
  21. 21.
    Geer, L.Y., Marchler-Bauer, A., Geer, R.C., Han, L., He, J., He, S., Liu, C., Shi, W., Bryant, S.H.: The NCBI BioSystems database. Nucleic Acids Res. 38(Database issue), D492–6 (2010). doi:10.1093/nar/gkp858 Google Scholar
  22. 22.
    Giger, M.L.: Intelligent CAD workstation for breast imaging using similarity to known lesions and multiple visual prompt aids. Proc. SPIE 4684, 768–773 (2002). doi:10.1117/12.467222 CrossRefGoogle Scholar
  23. 23.
    Grethe, J.S., Baru, C., Gupta, A., James, M., Ludaescher, B., Martone, M.E., Papadopoulos, P.M., Peltier, S.T., Rajasekar, A., Santini, S., Zaslavsky, I.N., Ellisman, M.H.: Biomedical informatics research network: building a national collaboratory to hasten the derivation of new understanding and treatment of disease. Stud. Health Technol. Inform. 112, 100–109 (2005)Google Scholar
  24. 24.
    Hornbak, K.: Current practice in measuring usability: challenges to usability studies and research. Int. J. Human-Comput. Stud. 64(2), 79–102 (2006). doi:10.1016/j.ijhcs.2005.06.002 CrossRefGoogle Scholar
  25. 25.
    ISO—International Organization for Standardization: Ergonomic requirements for office work with visual display terminals (VDTs)—part 11: guidance on usability (1998)Google Scholar
  26. 26.
    ISO—International Organization for Standardization: Software engineering—product quality—part 4: quality in use metrics (2004)Google Scholar
  27. 27.
    Jaspers, M.W.M.: A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int. J. Med. Inform. 78(5), 340–353 (2009)CrossRefGoogle Scholar
  28. 28.
    Kranzlmüller, D., Lucas, J.M., Öster, P.: The European Grid Initiative (EGI). In: Davoli, F., Pugliese, R., Meyer, N., Zappatore, S. (eds.) Remote Instrumentation and Virtual Laboratories, pp. 61–66. Springer (2010). doi:10.1007/978-1-4419-5597-5_6
  29. 29.
    Langlotz, C.P.: RadLex: a new method for indexing online educational materials. Radiographics 26(6), 1595–1597. doi:10.1148/rg.266065168
  30. 30.
    Lee, K.P., Hu, J.: XML schema representation of DICOM structured reporting. J. Am. Med. Inform. Assoc. 10(2), 213–223 (2003)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Lenert, L., Sundwall, D.N.: Public health surveillance and meaningful use regulations: a crisis of opportunity. Am. J. Publ. Health 102(3), e1–e7 (2012). doi:10.2105/AJPH.2011.300542 CrossRefGoogle Scholar
  32. 32.
    Lewis, J.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum.-Comput. Interact. 7(1), 57–78 (1995). doi:10.1080/10447319509526110 CrossRefGoogle Scholar
  33. 33.
    Lewis, J.R.: Psychometric evaluation of the PSSUQ using data from five years of usability studies. Int. J. Hum.-Comput. Interact. 14(3–4), 463–488 (2002). doi:10.1080/10447318.2002.9669130 Google Scholar
  34. 34.
    Lindland, O.I., Sindre, G., Solvberg, A.: Understanding quality in conceptual modeling. IEEE Softw. 11(2), 42–49 (1994). doi:10.1109/52.268955 CrossRefGoogle Scholar
  35. 35.
    Llorente, I.M., Moreno-Vozmediano, R., Montero, R.S.: Cloud computing for on-demand resource provisioning. In: Gentzsch, W., Grandinetti, L., Joubert, G. (eds.) High Speed and Large Scale Scientific Computing, pp. 177–191. IOS Press, Amsterdam (2009). doi:10.3233/978-1-60750-073-5-177
  36. 36.
    Napel, S.A., Beaulieu, C.F., Rodriguez, C., Cui, J., Xu, J., Gupta, A., Korenblum, D., Greenspan, H., Ma, Y., Rubin, D.L.: Automated retrieval of CT images of liver lesions on the basis of image similarity: method and preliminary results. Radiology 256(1), 243–252 (2010)CrossRefGoogle Scholar
  37. 37.
    Nielsen, J.: Usability Engineering, vol. 44. Morgan Kaufmann (1993)Google Scholar
  38. 38.
    OGF—Open Grid Forum: The Storage Resource Manager Interface Specification Version 2.2. Tech. rep. (2008).
  39. 39.
    OGF—Open Grid Forum: A simple API for Grid applications (SAGA). Tech. rep. (2011).
  40. 40.
    OGF—Open Grid Forum: Data Format Description Language (DFDL) v1.0 Specification. Tech. rep. (2011).
  41. 41.
    OGF—Open Grid Forum: Distributed Resource Management Application API Version 2 (DRMAA). Tech. rep. (2012).
  42. 42.
    Pereira, J.A., Quach, S., Hamid, J.S., Heidebrecht, C.L., Quan, S.D., Nassif, J., Diniz, A.J., Exan, R.V., Malawski, J., Gentry, A., Finkelstein, M., Guay, M., Buckeridge, D.L., Bettinger, J.A., Kalailieff, D., Kwong, J.C.: Exploring the feasibility of integrating barcode scanning technology into vaccine inventory recording in seasonal influenza vaccination clinics. Vaccine 30(4), 794–802 (2012). doi:10.1016/j.vaccine.2011.11.043 CrossRefGoogle Scholar
  43. 43.
    Ramos-Pollán, R., Guevara-López, M.A., Suárez-Ortega, C., Díaz-Herrero, G., Franco-Valiente, J.M., Rubio-Del-Solar, M., González-de Posada, N., Vaz, M.A.P., Loureiro, J., Ramos, I.: Discovering mammography-based machine learning classifiers for breast cancer diagnosis. J. Med. Syst. 36(4), 2259–2269 (2012). doi:10.1007/s10916-011-9693-2 CrossRefGoogle Scholar
  44. 44.
    Redolfi, A., McClatchey, R., Anjum, A., Zijdenbos, A., Manset, D., Barkhof, F., Spenger, C., Legré, Y., Wahlund, L.O., Pietro, C.B.d.S., B Frisoni, G.: Grid infrastructures for computational neuroscience: the neuGRID example. Future Neurol. 4(6), 703–722 (2009). doi:10.2217/fnl.09.53 CrossRefGoogle Scholar
  45. 45.
    Rimal, B.P., Jukan, A., Katsaros, D., Goeleven, Y.: Architectural requirements for cloud computing cystems: an enterprise cloud approach. J. Grid Computing 9(1), 3–26 (2010). doi:10.1007/s10723-010-9171-y CrossRefGoogle Scholar
  46. 46.
    Rings, T., Caryer, G., Gallop, J., Grabowski, J., Kovacikova, T., Schulz, S., Stokes-Rees, I.: Grid and cloud computing: opportunities for integration with the next generation network. J. Grid Computing 7(3), 375–393 (2009). doi:10.1007/s10723-009-9132-5 CrossRefGoogle Scholar
  47. 47.
    Rodero-Merino, L., Vaquero, L.M., Gil, V., Galán, F., Fontán, J., Montero, R.S., Llorente, I.M.: From infrastructure delivery to service management in clouds. Future Gener. Comput. Syst. 26(8), 1226–1240 (2010). doi:10.1016/j.future.2010.02.013 CrossRefGoogle Scholar
  48. 48.
    Rosenthal, A., Mork, P., Li, M.H., Stanford, J., Koester, D., Reynolds, P.: Cloud computing: a new business paradigm for biomedical information sharing. Journal of Biomedical Informatics 43(2), 342–353 (2010)CrossRefGoogle Scholar
  49. 49.
    Ruch, P., Gobeill, J., Lovis, C., Geissbühler, A.: Automatic medical encoding with SNOMED categories. BMC Med. Inform. Decis. Mak. 8(Suppl 1), S6 (2008). doi:10.1186/1472-6947-8-S1-S6 CrossRefGoogle Scholar
  50. 50.
    Salavert Torres, J., Maestre Urbano, C.V., Segrelles Quilis, J.D., Blanquer Espert, I., Garcia, H.V., Martí Bonmatí, L.: Grid prototype to support cancer of breast diagnostics in clinic practice. In: IBERGRID’2010—4th Iberian Grid Infrastructure Conference. Netbiblo, Braga, pp. 285–294 (2010)Google Scholar
  51. 51.
    Saleem, J., Haggstrom, D., Militello, L., Flanagan, M., Kiess, C., Arbuckle, N., Doebbeling, B.: Redesign of a computerized clinical reminder for colorectal cancer screening: a human-computer interaction evaluation. BMC Med. Inform. Decis. Mak. 11(1), 74 (2011). doi:10.1186/1472-6947-11-74 CrossRefGoogle Scholar
  52. 52.
    Schwiegelshohn, U., Badia, R.M., Bubak, M., Danelutto, M., Dustdar, S., Gagliardi, F., Geiger, A., Hluchy, L., Kranzlmüller, D., Laure, E.: Perspectives on Grid computing. Future Gener. Comput. Syst. 26(8), 1104–1115 (2010). doi:10.1016/j.future.2010.05.010 CrossRefGoogle Scholar
  53. 53.
    Segrelles, D., Blanquer, I., Salavert, J., Hernandez, V., Franco, J.M., Diaz, G., Ramos, R., Medina, R., Marti, L., Guevara, M.A., Gonzalez, N., Loureiro, J., Ramos, I.: Exchanging data for breast cancer diagnosis on heterogeneous Grid platforms. Comput. Inform. 31(1), 3–15 (2012)Google Scholar
  54. 54.
    Serco Usability Services: Performance Measurement Handbook, version 3 edn. Middlesex, UK (1995)Google Scholar
  55. 55.
    Shapiro, S.S., Wilk, M.B.: An analysis of variance test for normality (complete samples). Biometrika 52(3–4), 591–611 (1965). doi:10.1093/biomet/52.3-4.591 MathSciNetMATHGoogle Scholar
  56. 56.
    Silva, L., Costa, C., Oliveira, J.: A PACS archive architecture supported on cloud services. Int. J. Comput. Assist. Radiol. Surg. 7(3), 349–358 (2012). doi:10.1007/s11548-011-0625-x CrossRefGoogle Scholar
  57. 57.
    Teng, C.C., Mitchell, J., Walker, C., Swan, A., Davila, C., Howard, D., Needham, T.: A medical image archive solution in the cloud. In: 2010 IEEE International Conference on Software Engineering and Service Sciences (ICSESS), pp. 431–434 (2010). doi:10.1109/ICSESS.2010.5552343
  58. 58.
    Van Rijsbergen, C.J.: Information retrieval. In: The Kluwer International Series on Information Retrieval, vol. 30. Butterworths (1979). doi:10.1016/0020-0271(68)90016-8
  59. 59.
    Weiss, D.L., Langlotz, C.P.: Structured reporting: patient care enhancement or productivity nightmare? Radiology 249(3), 739–747 (2008). doi:10.1148/radiol.2493080988 CrossRefGoogle Scholar
  60. 60.
    Wess, M.L., Saleem, J.J., Tsevat, J., Luckhaupt, S.E., Johnston, J.A., Wise, R.E., Kopke, J.E., Eckman, M.H.: Usability of an atrial fibrillation anticoagulation decision-support tool. J. Prim. Care Community Health 2(2), 100–106 (2011). doi:10.1177/2150131910387608 CrossRefGoogle Scholar
  61. 61.
    Zentner, L.K., Clark, S.M., Smith, P.M., Shivarajapura, S., Farnsworth, V., Madhavan, K.P.C., Klimeck, G.: Practical considerations in cloud utilization for the science gateway In: 2011 Fourth IEEE International Conference on Utility and Cloud Computing (UCC), pp. 287–292 (2011). doi:10.1109/UCC.2011.46
  62. 62.
    Zhao, L., Lee, K.P., Hu, J.: Generating XML schemas for DICOM structured reporting templates. J. Am. Med. Inform. Assoc. 12(1), 72–83 (2005)CrossRefGoogle Scholar
  63. 63.
    Zheng, B., Mello-Thoms, C., Wang, X.H., Abrams, G.S., Sumkin, J.H., Chough, D.M., Ganott, M.A., Lu, A., Gur, D.: Interactive computer-aided diagnosis of breast masses: computerized selection of visually similar image sets from a reference library. Acad. Radiol. 14(8), 917–927 (2007)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2012

Authors and Affiliations

  • Cristina Maestre
    • 1
  • J. Damiàn Segrelles Quilis
    • 1
  • Erik Torres
    • 1
  • Ignacio Blanquer
    • 1
  • Rosana Medina
    • 2
  • Vicente Hernández
    • 1
  • Luis Martí
    • 3
  1. 1.Institut d’Instrumentació per a Imatge Molecular (I3M)Universitat Politècnica de ValènciaValenciaSpain
  2. 2.Hospital Universitari Doctor PesetValenciaSpain
  3. 3.Hospital Universitari i Politècnic La FeValenciaSpain

Personalised recommendations