Estimating the Quality of Ontology-Based Annotations by Considering Evolutionary Changes

  • Anika Gross
  • Michael Hartung
  • Toralf Kirsten
  • Erhard Rahm
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5647)


Ontology-based annotations associate objects, such as genes and proteins, with well-defined ontology concepts to semantically and uniformly describe object properties. Such annotation mappings are utilized in different applications and analysis studies whose results strongly depend on the quality of the used annotations. To study the quality of annotations we propose a generic evaluation approach considering the annotation generation methods (provenance) as well as the evolution of ontologies, object sources, and annotations. Thus, it facilitates the identification of reliable annotations, e.g., for use in analysis applications. We evaluate our approach for functional protein annotations in Ensembl and Swiss-Prot using the Gene Ontology.


annotation evolution quality 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Berriz, G.F., King, O.D., Bryant, B., et al.: Characterizing gene sets with FuncAssociate. Bioinformatics 19(18), 2502–2504 (2003)CrossRefPubMedGoogle Scholar
  2. 2.
    Bose, R., Frew, J.: Lineage retrieval for scientific data processing: A survey. ACM Computing Surveys 37(1), 1–28 (2005)CrossRefGoogle Scholar
  3. 3.
    Boutet, E., Lieberherr, D., Tognolli, M.: UniProtKB/Swiss-Prot. Methods in Molecular Biology 406, 89–112 (2007)PubMedGoogle Scholar
  4. 4.
    Boyle, E.I., Weng, S., Gollub, J., et al.: GO:TermFinder - open source software for accessing Gene Ontology information and finding significantly enriched Gene Ontology terms associated with a list of genes. Bioinformatics 20(18), 3710–3715 (2004)CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Buneman, P., Chapman, A., Cheney, J.: Provenance management in curated databases. In: Proc. of the 2006 ACM SIGMOD International Conference on Management of Data, pp. 539–550 (2006)Google Scholar
  6. 6.
    Buza, T.J., McCarty, F.M., Wang, N.: Gene Ontology annotation quality analysis in model eukaryotes. Nucleic Acids Research 36(2), e12 (2008)CrossRefGoogle Scholar
  7. 7.
    Dahlquist, K.D., Salomonis, N., Vranizan, K., et al.: GenMAPP, a new tool for viewing and analyzing microarray data on biological pathways. Nature Genetics 31(1), 19–20 (2002)CrossRefPubMedGoogle Scholar
  8. 8.
    Gene Ontology - Evidence Codes:
  9. 9.
    The Gene Ontology Consortium: The Gene Ontology project in 2008. Nucleic Acids Research 36, D440–D441 (2008) (Database issue)Google Scholar
  10. 10.
    Hartung, M., Kirsten, T., Rahm, E.: Analyzing the Evolution of Life Science Ontologies and Mappings. In: Bairoch, A., Cohen-Boulakia, S., Froidevaux, C. (eds.) DILS 2008. LNCS (LNBI), vol. 5109, pp. 11–27. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Hubbard, T.J., Aken, B.L., Ayling, S., et al.: Ensembl 2009. Nucleic Acids Research 37, D690–D697 (2009) (Database issue) CrossRefGoogle Scholar
  12. 12.
    Jones, C.E., Brown, A.L., Baumann, U.: Estimating the annotation error rate of curated GO database sequence annotations. BMC Bioinformatics 8(1), 170 (2007)CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Kirsten, T., Thor, A., Rahm, E.: Instance-based matching of large life science ontologies. In: Cohen-Boulakia, S., Tannen, V. (eds.) DILS 2007. LNCS (LNBI), vol. 4544, pp. 172–187. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  14. 14.
    Klein, M.: Change Management for Distributed Ontologies. PhD thesis, Vrije Universiteit Amsterdam (2004)Google Scholar
  15. 15.
    Klein, M., Fensel, D.: Ontology versioning on the Semantic Web. In: Proceedings of the International Semantic Web Working Symposium (SWWS), pp. 75–91 (2001)Google Scholar
  16. 16.
    Naumann, F., Leser, U., Freytag, J.C.: Quality-driven Integration of Heterogeneous In-formation Systems. In: Proc. of the International Conference on Very Large Data Bases (VLDB), pp. 447–458 (1999)Google Scholar
  17. 17.
    Noy, N., Klein, M.: Ontology evolution: Not the same as schema evolution. Knowledge and Information Systems 6(4), 428–440 (2004)CrossRefGoogle Scholar
  18. 18.
    Rahm, E., Do, H.H.: Data Cleaning: Problems and Current Approaches. IEEE Data Engineering Bulletin 23(4), 3–13 (2000)Google Scholar
  19. 19.
    Redman, T.C.: Data Quality for the Information Age. Artech House (1996)Google Scholar
  20. 20.
    Stojanovic, L., Maedche, A., Motik, B., Stojanovic, N.: User-driven ontology evolution management. In: Gómez-Pérez, A., Benjamins, V.R. (eds.) EKAW 2002. LNCS, vol. 2473, pp. 285–300. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  21. 21.
    Stojanovic, L., Motik, B.: Ontology evolution within ontology editors. In: Proceedings of the International Workshop on Evaluation of Ontology-based Tools, pp. 53–62 (2002)Google Scholar
  22. 22.
    Thomas, P.D., Mi, H., Lewis, S.: Ontology annotation: mapping genomic regions to biological function. Current Opinion in Chemical Biology 11(1), 4–11 (2007)CrossRefPubMedGoogle Scholar
  23. 23.
    Thor, A., Hartung, M., Gross, A., Kirsten, T., Rahm, E.: An evolution-based approach for assessing ontology mappings - A case study in the life sciences. In: Proc. Conference of the Business, Technology and Web (BTW), pp. 277–286 (2009)Google Scholar
  24. 24.
    Yang, Z., Zhang, D., Ye, C.: Ontology Analysis on Complexity and Evolution Based on Conceptual Model. In: Leser, U., Naumann, F., Eckman, B. (eds.) DILS 2006. LNCS (LNBI), vol. 4075, pp. 216–223. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Anika Gross
    • 1
  • Michael Hartung
    • 1
  • Toralf Kirsten
    • 1
    • 2
  • Erhard Rahm
    • 1
    • 3
  1. 1.Interdisciplinary Centre for BioinformaticsUniversity of LeipzigGermany
  2. 2.Institute for Medical Informatics, Statistics and EpidemiologyUniversity of LeipzigGermany
  3. 3.Department of Computer ScienceUniversity of LeipzigGermany

Personalised recommendations