Abstract
In the digital era, patient-specific 3D models (3D-PSMs) are becoming increasingly relevant in computer-assisted diagnosis, surgery training on digital models, or implant design. While advanced imaging and reconstruction techniques can create accurate and detailed 3D models of patients’ anatomy, software tools that are able to fully exploit the potential of 3D-PSMs are still far from being satisfactory. In particular, there is still a lack of integrated approaches for extracting, coding, sharing and retrieving medically relevant information from 3D-PSMs and use it concretely as a support to diagnosis and treatment. In this article, we propose the SemAnatomy3D framework, which demonstrates how the ontology-driven annotation of 3D-PSMs and of their anatomically relevant features (parts of relevance) can assist clinicians to document more effectively pathologies and their evolution. We exemplify the idea in the context of the diagnosis of rheumatoid arthritis of the hand district, and show how feature extraction tools and semantic 3D annotation can provide a rich characterization of anatomical landmarks (e.g., articular facets, prominent features, ligament attachments) and pathological markers (erosions, bone loss). The core contributions are an ontology-driven part-based annotation method for the 3D-PSMs and a novel automatic localization of erosion and quantification of the OMERACT RAMRIS erosion score. Finally, our results have been compared against a medical ground truth.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat. 2(4), 433–459 (2010)
Banerjee, I., Agibetov, A., Catalano, C., Patanè, G., Spagnuolo, M.: Semantic annotation of patient-specific 3D anatomical models. International Conference on cyberworlds (CW) (2015)
Banerjee, I., Catalano, C.E., Patané, G., Spagnuolo, M.: Semantic annotation of 3D anatomical models to support diagnosis and follow-up analysis of musculoskeletal pathologies. Int. J. Comput. Assist. Radiol. Surg. 1–14 (2015)
Banerjee, I., Laga, H., Patanè, G., Kurtek, S., Srivastava, A., Spagnuolo, M.: Generation of 3D canonical anatomical models: an experience on carpal bones. In: New trends in image analysis and processing—ICIAP 2015 workshops, pp. 167–174. Springer (2015)
Banerjee, I., Patanè, G., Spagnuolo, M.: SemAnatomy3D: annotation of patient-specific anatomy. In: Giachetti, A., Biasotti, S., Tarini, M (eds.) Smart tools and apps for graphics—eurographics Italian chapter conference. The Eurographics Association (2015). doi:10.2312/stag.20151292
Boeker, M., Schober, D., Raufie, D., Grewe, N., Röhl, J., Jansen, L., Schulz, S.: Teaching good biomedical ontology design. In: Proceedings of international conference on biomedical ontologies (2012)
Ejbjerg, B., McQueen, F., Lassere, M., Haavardsholm, E., Conaghan, P., O’Connor, P., Bird, P., Peterfy, C., Edmonds, J., Szkudlarek, M., et al.: The EULAR-OMERACT rheumatoid arthritis MRI reference image atlas: the wrist joint. Ann. Rheum. Dis. 64(suppl 1), i23–i47 (2005)
Hunter, J., Cole, T., Sanderson, R., Van de Sompel, H.: The open annotation collaboration: a data model to support sharing and interoperability of scholarly annotations. In: Digital humanities 2010, pp. 175–178. Office for Humanities Communication, Centre for Computing in the Humanities (2010)
Jimnez-Ruiz, E., Grau, B.C., Sattler, U., Schneider, T., Berlanga, R.: Safe and economic re-use of ontologies: a logic-based methodology and tool support. The Semantic Web: research and applications, no. 5021. Lecture notes in computer science, pp. 185–199. Springer, Berlin (2008)
Kahan, J., Koivunen, M.R., Prud’Hommeaux, E., Swick, R.R.: Annotea: an open RDF infrastructure for shared web annotations. Comput. Netw. 39(5), 589–608 (2002)
Langlotz, C.P.: RadLex: a new method for indexing online educational materials. Radiographics 26(6), 1595–1597 (2006)
Lukovic, V.: Integrating biomedical ontologies OBR-scolio ontology. In: Special issue ICIT—bioinformatics and image, 3, vol. 4, pp. 664–669 (2009)
Mejino, J.L., Rubin, D.L., Brinkley, J.F.: FMA-RadLex: an application ontology of radiological anatomy derived from the foundational model of anatomy reference ontology. AMIA Annu. Symp. Proceed. 2008, 465–469 (2008)
Mediare project. http://www.research.softeco.it/mediare.aspx. Accessed 1 July 2013
Möller, M., Regel, S., Sintek, M.: Radsem: Semantic annotation and retrieval for medical images. In: The Semantic Web: research and applications, pp. 21–35. Springer (2009)
MSH: MSH ontology: deliverable reports D8.2 (m24, m36) and OWL file. Accessed 28 July 2015. http://multiscalehuman.miralab.ch/repository/Public_download/D8.2_MSD-Ontology/
MSH: Multiscalehuman project. http://multiscalehuman.miralab.ch/. Accessed 1 June 2014
Organization, W.H., et al.: The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines. World Health Organization, Geneva (1992)
Østergaard, M., Peterfy, C., Conaghan, P., McQueen, F., Bird, P., Ejbjerg, B., Shnier, R., O’Connor, P., Klarlund, M., Emery, P., et al.: Omeract rheumatoid arthritis magnetic resonance imaging studies. core set of MRI acquisitions, joint pathology definitions, and the omeract ra-MRI scoring system. J. Rheumatol. 30(6), 1385–1386 (2003)
Parascandolo, P., Cesario, L., Vosilla, L., Viano, G.: Rheumascore: A cad system for rheumatoid arthritis diagnosis and follow-up. In: New trends in image analysis and processing—ICIAP 2015 workshops, pp. 135–142. Springer (2015)
Rosse, C., Mejino, J.L.V.: A reference ontology for biomedical informatics: the foundational model of anatomy. J. Biomed. Inform. 36, 500 (2003)
Politecmed consortium. http://www.camelotbio.com/politecmed. Accessed 15 July 2015
Rubin, D.L., Rodriguez, C., Shah, P., Beaulieu, C.: iPad: semantic annotation and markup of radiological images. In: AMIA annual symposium proceedings, vol. 2008, p. 626. American Medical Informatics Association (2008)
Sanderson, R., Ciccarese, P., Van de Sompel, H.: Open annotation data model. W3C Community Draft. http://www.openannotation.org/spec/core/20130208/index.html (2013). Accessed 8 Feb 2013
Seifert, S., Kelm, M., Moeller, M., Mukherjee, S., Cavallaro, A., Huber, M., Comaniciu, D.: Semantic annotation of medical images. In: SPIE medical imaging, pp. 762, 808–762, 808. International Society for Optics and Photonics (2010)
Tomatis, V., Cimmino, M.A., Barbieri, F., Troglio, G., Parascandolo, P., Cesario, L., Viano, G., Vosilla, L., Pitikakis, M., Schiappacasse, A., et al.: A database of segmented MRI images of the wrist and the hand in patients with rheumatic diseases. In: New Trends in image analysis and processing—CIAP 2015 workshops, pp. 143–150. Springer (2015)
Woolf, A.D., Pfleger, B.: Burden of major musculoskeletal conditions. Bull. World Health Org. 81(9), 646–656 (2003)
Acknowledgments
This work is supported by the FP7 Marie Curie Initial Training Network “MultiScaleHuman”: Multi-scale Biological Modalities for Physiological Human Articulation (2011–2015). This work is also partially supported by the Project FAS-MEDIARE “Nuove metodologie di Imaging Diagnostico per patologie reumatiche”. The carpal data set is provided by Softeco Sismat Srl in collaboration with DIMI (Dipartimento di Medicina Interna, Clinica Reumatologica, Universita degli Studi di Genova).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Banerjee, I., Agibetov, A., Catalano, C.E. et al. Semantics-driven annotation of patient-specific 3D data: a step to assist diagnosis and treatment of rheumatoid arthritis. Vis Comput 32, 1337–1349 (2016). https://doi.org/10.1007/s00371-016-1226-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-016-1226-z