VC-Dimension Based Generalization Bounds for Relational Learning

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11052)


In many applications of relational learning, the available data can be seen as a sample from a larger relational structure (e.g. we may be given a small fragment from some social network). In this paper we are particularly concerned with scenarios in which we can assume that (i) the domain elements appearing in the given sample have been uniformly sampled without replacement from the (unknown) full domain and (ii) the sample is complete for these domain elements (i.e. it is the full substructure induced by these elements). Within this setting, we study bounds on the error of sufficient statistics of relational models that are estimated on the available data. As our main result, we prove a bound based on a variant of the Vapnik-Chervonenkis dimension which is suitable for relational data.



OK’s work was partially supported by the Research Foundation - Flanders (project G.0428.15). SS is supported by ERC Starting Grant 637277.

Supplementary material

478890_1_En_16_MOESM1_ESM.pdf (358 kb)
Supplementary material 1 (pdf 358 KB)


  1. 1.
    Clémençon, S., Lugosi, G., Vayatis, N.: Ranking and empirical minimization of u-statistics. Annal. Stat. 36, 844–874 (2008)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Devroye, L., Györfi, L., Lugosi, G.: A Probabilistic Theory of Pattern Recognition. Stochastic Modelling and Applied Probability, vol. 31. Springer, New York (1996). Scholar
  3. 3.
    Dhurandhar, A., Dobra, A.: Distribution-free bounds for relational classification. Knowl. Inf. Syst. 31(1), 55–78 (2012)CrossRefGoogle Scholar
  4. 4.
    Hoeffding, W.: A class of statistics with asymptotically normal distribution. Annal. Math. Stat. 19, 293–325 (1948)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Hoeffding, W.: Probability inequalities for sums of bounded random variables. J. Am. Stat. Assoc. 58(301), 13–30 (1963)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Juba, B.: Implicit learning of common sense for reasoning. In: Proceedings of the 23rd International Joint Conference on Artificial Intelligence, pp. 939–946 (2013)Google Scholar
  7. 7.
    Kuželka, O., Wang, Y., Davis, J., Schockaert, S.: PAC-reasoning in relational domains. In: Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence, UAI 2018 (2018)Google Scholar
  8. 8.
    Kuželka, O., Wang, Y., Davis, J., Schockaert, S.: Relational marginal problems: theory and estimation. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18) (2018)Google Scholar
  9. 9.
    Kuželka, O., Davis, J., Schockaert, S.: Induction of interpretable possibilistic logic theories from relational data. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 1153–1159 (2017)Google Scholar
  10. 10.
    Langford, J., Shawe-Taylor, J.: PAC-Bayes & margins. In: Proceedings of the Annual Conference on Neural Information Processing Systems, pp. 423–430 (2002)Google Scholar
  11. 11.
    Nandi, H., Sen, P.: On the properties of u-statistics when the observations are not independent: Part two unbiased estimation of the parameters of a finite population. Calcutta Stat. Assoc. Bull. 12(4), 124–148 (1963)CrossRefGoogle Scholar
  12. 12.
    Rocktäschel, T., Riedel, S.: End-to-end differentiable proving. In: Proceedings of the Annual Conference on Neural Information Processing Systems, pp. 3791–3803 (2017)Google Scholar
  13. 13.
    Shalev-Shwartz, S., Ben-David, S.: Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, New York (2014)CrossRefGoogle Scholar
  14. 14.
    Valiant, L.G.: Knowledge infusion. In: Proceedings of the 21st National Conference on Artificial Intelligence, pp. 1546–1551 (2006)Google Scholar
  15. 15.
    Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (2000). Scholar
  16. 16.
    Vapnik, V., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16(2), 264 (1971)CrossRefGoogle Scholar
  17. 17.
    Vishwanathan, S.V.N., Schraudolph, N.N., Kondor, R., Borgwardt, K.M.: Graph kernels. J. Mach. Learn. Res. 11, 1201–1242 (2010)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Šourek, G., Aschenbrenner, V., Železný, F., Kuželka, O.: Lifted relational neural networks. In: Proceedings of the NIPS Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches (2015)Google Scholar
  19. 19.
    Xiang, R., Neville, J.: Relational learning with one network: an asymptotic analysis. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 779–788 (2011)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceKU LeuvenLeuvenBelgium
  2. 2.DISCO GroupETH ZurichZurichSwitzerland
  3. 3.School of Computer Science and InformaticsCardiff UniversityCardiffUK

Personalised recommendations