Scalable Relation Prediction Exploiting Both Intrarelational Correlation and Contextual Information

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7523)


We consider the problem of predicting instantiated binary relations in a multi-relational setting and exploit both intrarelational correlations and contextual information. For the modular combination we discuss simple heuristics, additive models and an approach that can be motivated from a hierarchical Bayesian perspective. In the concrete examples we consider models that exploit contextual information both from the database and from contextual unstructured information, e.g., information extracted from textual documents describing the involved entities. By using low-rank approximations in the context models, the models perform latent semantic analyses and can generalize across specific terms, i.e., the model might use similar latent representations for semantically related terms. All the approaches we are considering have unique solutions. They can exploit sparse matrix algebra and are thus highly scalable and can easily be generalized to new entities. We evaluate the effectiveness of nonlinear interaction terms and reduce the number of terms by applying feature selection. For the optimization of the context model we use an alternating least squares approach. We experimentally analyze scalability. We validate our approach using two synthetic data sets and using two data sets derived from the Linked Open Data (LOD) cloud.


Singular Value Decomposition Contextual Information Context Model Combination Scheme Latent Semantic Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bell, R.M., Koren, Y., Volinsky, C.: All together now: A perspective on the netflix prize. Chance (2010)Google Scholar
  2. 2.
    Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. Journal of Machine Learning Research (2012)Google Scholar
  3. 3.
    Bloehdorn, S., Sure, Y.: Kernel methods for mining instance data in ontologies. In: ESWC (2007)Google Scholar
  4. 4.
    Candes, E.J., Recht, B.: Exact matrix completion via convex optimization. Computing Research Repository - CORR (2008)Google Scholar
  5. 5.
    Cumby, C.M., Roth, D.: On kernel methods for relational learning. In: ICML (2003)Google Scholar
  6. 6.
    D’Amato, C., Fanizzi, N., Esposito, F.: Non-parametric statistical learning methods for inductive classifiers in semantic knowledge bases. In: IEEE International Conference on Semantic Computing, ICSC (2008)Google Scholar
  7. 7.
    Gärtner, T., Lloyd, J.W., Flach, P.A.: Kernels and distances for structured data. Machine Learning (2004)Google Scholar
  8. 8.
    Getoor, L., Diehl, C.P.: Link mining: a survey. SIGKDD Explorations (2005)Google Scholar
  9. 9.
    Getoor, L., Friedman, N., Koller, D., Pfeffer, A., Taskar, B.: Probabilistic relational models. In: Introduction to Statistical Relational Learning (2007)Google Scholar
  10. 10.
    Huang, Y., Tresp, V., Bundschus, M., Rettinger, A., Kriegel, H.-P.: Multivariate Prediction for Learning on the Semantic Web. In: Frasconi, P., Lisi, F.A. (eds.) ILP 2010. LNCS, vol. 6489, pp. 92–104. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  11. 11.
    Järvelin, K., Kekäläinen, J.: IR evaluation methods for retrieving highly relevant documents. In: SIGIR 2000 (2000)Google Scholar
  12. 12.
    Jiang, X., Huang, Y., Nickel, M., Tresp, V.: Combining Information Extraction, Deductive Reasoning and Machine Learning for Relation Prediction. In: Simperl, E., Cimiano, P., Polleres, A., Corcho, O., Presutti, V. (eds.) ESWC 2012. LNCS, vol. 7295, pp. 164–178. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  13. 13.
    Jiang, X., Tresp, V., Huang, Y., Nickel, M., Kriegel, H.-P.: Link Prediction in Multi-relational Graphs using Additive Models (submitted, 2012)Google Scholar
  14. 14.
    Kann, M.G.: Advances in translational bioinformatics: computational approaches for the hunting of disease genes. In: Briefings in Bioinformatics (2010)Google Scholar
  15. 15.
    Kemp, C., Tenenbaum, J.B., Griffiths, T.L., Yamada, T., Ueda, N.: Learning systems of concepts with an infinite relational model. In: AAAI (2006)Google Scholar
  16. 16.
    Koller, D., Pfeffer, A.: Probabilistic frame-based systems. In: AAAI (1998)Google Scholar
  17. 17.
    Landwehr, N., Passerini, A., De Raedt, L., Frasconi, P.: kFOIL: Learning simple relational kernels. In: AAAI (2006)Google Scholar
  18. 18.
    Lösch, U., Bloehdorn, S., Rettinger, A.: Graph Kernels for RDF Data. In: Simperl, E., Cimiano, P., Polleres, A., Corcho, O., Presutti, V. (eds.) ESWC 2012. LNCS, vol. 7295, pp. 134–148. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  19. 19.
    Muggleton, S.H., Lodhi, H., Amini, A., Sternberg, M.J.E.: Support Vector Inductive Logic Programming. In: Hoffmann, A., Motoda, H., Scheffer, T. (eds.) DS 2005. LNCS (LNAI), vol. 3735, pp. 163–175. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  20. 20.
    Nickel, M., Tresp, V., Kriegel, H.-P.: A three-way model for collective learning on multi-relational data. In: ICML (2011)Google Scholar
  21. 21.
    Nickel, M., Tresp, V., Kriegel, H.-P.: Factorizing YAGO: scalable machine learning for linked data. In: WWW (2012)Google Scholar
  22. 22.
    Popescul, A., Ungar, L.H.: Statistical relational learning for link prediction. In: Workshop on Learning Statistical Models from Relational Data (2003)Google Scholar
  23. 23.
    Rettinger, A., Nickles, M., Tresp, V.: Statistical Relational Learning with Formal Ontologies. In: Buntine, W., Grobelnik, M., Mladenić, D., Shawe-Taylor, J. (eds.) ECML PKDD 2009, Part II. LNCS, vol. 5782, pp. 286–301. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  24. 24.
    Richardson, M., Domingos, P.: Markov logic networks. In: Machine Learning (2006)Google Scholar
  25. 25.
    Takacs, G., Pilaszy, I., Nemeth, B., Tikk, D.: On the gravity recommendation system. In: Proceedings of KDD Cup 2007 (2007)Google Scholar
  26. 26.
    Taskar, B., Wong, M.F., Abbeel, P., Koller, D.: Link prediction in relational data. In: NIPS (2003)Google Scholar
  27. 27.
    Vishwanathan, S.V.N., Schraudolph, N., Kondor, R.I., Borgwardt, K.: Graph kernels. Journal of Machine Learning Research - JMLR (2008)Google Scholar
  28. 28.
    Xu, Z., Kersting, K., Tresp, V.: Multi-relational learning with gaussian processes. In: IJCAI (2009)Google Scholar
  29. 29.
    Xu, Z., Tresp, V., Yu, K., Kriegel, H.-P.: Infinite hidden relational models. In: UAI (2006)Google Scholar
  30. 30.
    Yu, K., Chu, W., Yu, S., Tresp, V., Xu, Z.: Stochastic relational models for discriminative link prediction. In: NIPS (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Corporate TechnologySiemens AGMunichGermany
  2. 2.Ludwig Maximilian University of MunichMunichGermany

Personalised recommendations