Skip to main content


Log in

Semantics-driven annotation of patient-specific 3D data: a step to assist diagnosis and treatment of rheumatoid arthritis

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript


In the digital era, patient-specific 3D models (3D-PSMs) are becoming increasingly relevant in computer-assisted diagnosis, surgery training on digital models, or implant design. While advanced imaging and reconstruction techniques can create accurate and detailed 3D models of patients’ anatomy, software tools that are able to fully exploit the potential of 3D-PSMs are still far from being satisfactory. In particular, there is still a lack of integrated approaches for extracting, coding, sharing and retrieving medically relevant information from 3D-PSMs and use it concretely as a support to diagnosis and treatment. In this article, we propose the SemAnatomy3D framework, which demonstrates how the ontology-driven annotation of 3D-PSMs and of their anatomically relevant features (parts of relevance) can assist clinicians to document more effectively pathologies and their evolution. We exemplify the idea in the context of the diagnosis of rheumatoid arthritis of the hand district, and show how feature extraction tools and semantic 3D annotation can provide a rich characterization of anatomical landmarks (e.g., articular facets, prominent features, ligament attachments) and pathological markers (erosions, bone loss). The core contributions are an ontology-driven part-based annotation method for the 3D-PSMs and a novel automatic localization of erosion and quantification of the OMERACT RAMRIS erosion score. Finally, our results have been compared against a medical ground truth.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others




  1. Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat. 2(4), 433–459 (2010)

    Article  Google Scholar 

  2. Banerjee, I., Agibetov, A., Catalano, C., Patanè, G., Spagnuolo, M.: Semantic annotation of patient-specific 3D anatomical models. International Conference on cyberworlds (CW) (2015)

  3. Banerjee, I., Catalano, C.E., Patané, G., Spagnuolo, M.: Semantic annotation of 3D anatomical models to support diagnosis and follow-up analysis of musculoskeletal pathologies. Int. J. Comput. Assist. Radiol. Surg. 1–14 (2015)

  4. Banerjee, I., Laga, H., Patanè, G., Kurtek, S., Srivastava, A., Spagnuolo, M.: Generation of 3D canonical anatomical models: an experience on carpal bones. In: New trends in image analysis and processing—ICIAP 2015 workshops, pp. 167–174. Springer (2015)

  5. Banerjee, I., Patanè, G., Spagnuolo, M.: SemAnatomy3D: annotation of patient-specific anatomy. In: Giachetti, A., Biasotti, S., Tarini, M (eds.) Smart tools and apps for graphics—eurographics Italian chapter conference. The Eurographics Association (2015). doi:10.2312/stag.20151292

  6. Boeker, M., Schober, D., Raufie, D., Grewe, N., Röhl, J., Jansen, L., Schulz, S.: Teaching good biomedical ontology design. In: Proceedings of international conference on biomedical ontologies (2012)

  7. Ejbjerg, B., McQueen, F., Lassere, M., Haavardsholm, E., Conaghan, P., O’Connor, P., Bird, P., Peterfy, C., Edmonds, J., Szkudlarek, M., et al.: The EULAR-OMERACT rheumatoid arthritis MRI reference image atlas: the wrist joint. Ann. Rheum. Dis. 64(suppl 1), i23–i47 (2005)

    Article  Google Scholar 

  8. Hunter, J., Cole, T., Sanderson, R., Van de Sompel, H.: The open annotation collaboration: a data model to support sharing and interoperability of scholarly annotations. In: Digital humanities 2010, pp. 175–178. Office for Humanities Communication, Centre for Computing in the Humanities (2010)

  9. Jimnez-Ruiz, E., Grau, B.C., Sattler, U., Schneider, T., Berlanga, R.: Safe and economic re-use of ontologies: a logic-based methodology and tool support. The Semantic Web: research and applications, no. 5021. Lecture notes in computer science, pp. 185–199. Springer, Berlin (2008)

  10. Kahan, J., Koivunen, M.R., Prud’Hommeaux, E., Swick, R.R.: Annotea: an open RDF infrastructure for shared web annotations. Comput. Netw. 39(5), 589–608 (2002)

    Article  Google Scholar 

  11. Langlotz, C.P.: RadLex: a new method for indexing online educational materials. Radiographics 26(6), 1595–1597 (2006)

    Article  Google Scholar 

  12. Lukovic, V.: Integrating biomedical ontologies OBR-scolio ontology. In: Special issue ICIT—bioinformatics and image, 3, vol. 4, pp. 664–669 (2009)

  13. Mejino, J.L., Rubin, D.L., Brinkley, J.F.: FMA-RadLex: an application ontology of radiological anatomy derived from the foundational model of anatomy reference ontology. AMIA Annu. Symp. Proceed. 2008, 465–469 (2008)

    Google Scholar 

  14. Mediare project. Accessed 1 July 2013

  15. Möller, M., Regel, S., Sintek, M.: Radsem: Semantic annotation and retrieval for medical images. In: The Semantic Web: research and applications, pp. 21–35. Springer (2009)

  16. MSH: MSH ontology: deliverable reports D8.2 (m24, m36) and OWL file. Accessed 28 July 2015.

  17. MSH: Multiscalehuman project. Accessed 1 June 2014

  18. Organization, W.H., et al.: The ICD-10 Classification of Mental and Behavioural Disorders: Clinical Descriptions and Diagnostic Guidelines. World Health Organization, Geneva (1992)

    Google Scholar 

  19. Østergaard, M., Peterfy, C., Conaghan, P., McQueen, F., Bird, P., Ejbjerg, B., Shnier, R., O’Connor, P., Klarlund, M., Emery, P., et al.: Omeract rheumatoid arthritis magnetic resonance imaging studies. core set of MRI acquisitions, joint pathology definitions, and the omeract ra-MRI scoring system. J. Rheumatol. 30(6), 1385–1386 (2003)

    Google Scholar 

  20. Parascandolo, P., Cesario, L., Vosilla, L., Viano, G.: Rheumascore: A cad system for rheumatoid arthritis diagnosis and follow-up. In: New trends in image analysis and processing—ICIAP 2015 workshops, pp. 135–142. Springer (2015)

  21. Rosse, C., Mejino, J.L.V.: A reference ontology for biomedical informatics: the foundational model of anatomy. J. Biomed. Inform. 36, 500 (2003)

    Article  Google Scholar 

  22. Politecmed consortium. Accessed 15 July 2015

  23. Rubin, D.L., Rodriguez, C., Shah, P., Beaulieu, C.: iPad: semantic annotation and markup of radiological images. In: AMIA annual symposium proceedings, vol. 2008, p. 626. American Medical Informatics Association (2008)

  24. Sanderson, R., Ciccarese, P., Van de Sompel, H.: Open annotation data model. W3C Community Draft. (2013). Accessed 8 Feb 2013

  25. Seifert, S., Kelm, M., Moeller, M., Mukherjee, S., Cavallaro, A., Huber, M., Comaniciu, D.: Semantic annotation of medical images. In: SPIE medical imaging, pp. 762, 808–762, 808. International Society for Optics and Photonics (2010)

  26. Tomatis, V., Cimmino, M.A., Barbieri, F., Troglio, G., Parascandolo, P., Cesario, L., Viano, G., Vosilla, L., Pitikakis, M., Schiappacasse, A., et al.: A database of segmented MRI images of the wrist and the hand in patients with rheumatic diseases. In: New Trends in image analysis and processing—CIAP 2015 workshops, pp. 143–150. Springer (2015)

  27. Woolf, A.D., Pfleger, B.: Burden of major musculoskeletal conditions. Bull. World Health Org. 81(9), 646–656 (2003)

    Google Scholar 

Download references


This work is supported by the FP7 Marie Curie Initial Training Network “MultiScaleHuman”: Multi-scale Biological Modalities for Physiological Human Articulation (2011–2015). This work is also partially supported by the Project FAS-MEDIARE “Nuove metodologie di Imaging Diagnostico per patologie reumatiche”. The carpal data set is provided by Softeco Sismat Srl in collaboration with DIMI (Dipartimento di Medicina Interna, Clinica Reumatologica, Universita degli Studi di Genova).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Imon Banerjee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Banerjee, I., Agibetov, A., Catalano, C.E. et al. Semantics-driven annotation of patient-specific 3D data: a step to assist diagnosis and treatment of rheumatoid arthritis. Vis Comput 32, 1337–1349 (2016).

Download citation

  • Published:

  • Issue Date:

  • DOI: