Learning to Tag Text from Rules and Examples

  • Michelangelo Diligenti
  • Marco Gori
  • Marco Maggini
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6934)


Tagging has become a popular way to improve the access to resources, especially in social networks and folksonomies. Most of the resource sharing tools allow a manual labeling of the available items by the community members. However, the manual approach can fail to provide a consistent tagging especially when the dimension of the vocabulary of the tags increases and, consequently, the users do not comply to a shared semantic knowledge. Hence, automatic tagging can provide an effective way to complete the manual added tags, especially for dynamic or very large collections of documents like the Web. However, when an automatic text tagger is trained over the tags inserted by the users, it may inherit the inconsistencies of the training data. In this paper, we propose a novel approach where a set of text categorizers, each associated to a tag in the vocabulary, are trained both from examples and a higher level abstract representation consisting of FOL clauses that describe semantic rules constraining the use of the corresponding tags. The FOL clauses are compiled into a set of equivalent continuous constraints, and the integration between logic and learning is implemented in a multi-task learning scheme. In particular, we exploit the kernel machine mathematical apparatus casting the problem as primal optimization of a function composed of the loss on the supervised examples, the regularization term, and a penalty term deriving from forcing the constraints resulting from the conversion of the logic knowledge. The experimental results show that the proposed approach provides a significant accuracy improvement on the tagging of bibtex entries.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bengio, S., Weston, J., Grangier, D.: Label embedding trees for large multi-class tasks. Advances in Neural Information Processing Systems 23, 163–171 (2010)Google Scholar
  2. 2.
    Caponnetto, A., Micchelli, C., Pontil, M., Ying, Y.: Universal Kernels for Multi-Task Learning. Journal of Machine Learning Research (2008)Google Scholar
  3. 3.
    Diligenti, M., Gori, M., Maggini, M., Rigutini, L.: Multitask Kernel-based Learning with Logic Constraints. In: Proceedings of the 19th European Conference on Artificial Intelligence, pp. 433–438. IOS Press, Amsterdam (2010)Google Scholar
  4. 4.
    Katakis, I., Tsoumakas, G., Vlahavas, I.: Multilabel text classification for automated tag suggestion. ECML PKDD Discovery Challenge 75 (2008)Google Scholar
  5. 5.
    Klement, E.P., Mesiar, R., Pap, E.: Triangular Norms. Kluwer Academic Publishers, Dordrecht (2000)CrossRefzbMATHGoogle Scholar
  6. 6.
    Laclavik, M., Seleng, M., Gatial, E., Balogh, Z., Hluchy, L.: Ontology based text annotation. In: Proceedings of the 18th International Conference on Information Modelling and Knowledge Bases, pp. 311–315. IOS Press, Amsterdam (2007)Google Scholar
  7. 7.
    Liu, D., Hua, X., Yang, L., Wang, M., Zhang, H.: Tag ranking. In: Proceedings of the 18th International Conference on World Wide Web, pp. 351–360. ACM, New York (2009)CrossRefGoogle Scholar
  8. 8.
    Matusiak, K.: Towards user-centered indexing in digital image collections. OCLC Systems & Services 22(4), 283–298 (2006)CrossRefGoogle Scholar
  9. 9.
    Peters, S., Denoyer, L., Gallinari, P.: Iterative Annotation of Multi-relational Social Networks. In: Proceedings of the International Conference on Advances in Social Networks Analysis and Mining, pp. 96–103. IEEE, Los Alamitos (2010)Google Scholar
  10. 10.
    Sebastiani, F.: Machine learning in automated text categorization. ACM Computing Surveys (CSUR) 34(1), 1–47 (2002)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Weinberger, K., Saul, L.: Distance metric learning for large margin nearest neighbor classification. The Journal of Machine Learning Research 10, 207–244 (2009)zbMATHGoogle Scholar
  12. 12.
    Zavitsanos, E., Tsatsaronis, G., Varlamis, I., Paliouras, G.: Scalable Semantic Annotation of Text Using Lexical and Web Resources. Artificial Intelligence: Theories, Models and Applications, 287–296 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Michelangelo Diligenti
    • 1
  • Marco Gori
    • 1
  • Marco Maggini
    • 1
  1. 1.Dipartimento di Ingegneria dell’InformazioneUniversità di SienaItaly

Personalised recommendations