Semi-supervised Learning in Causal and Anticausal Settings

  • Bernhard Schölkopf
  • Dominik Janzing
  • Jonas Peters
  • Eleni Sgouritsa
  • Kun Zhang
  • Joris Mooij


We consider the problem of learning in the case where an underlying causal model can be inferred. Causal knowledge may facilitate some approaches for a given problem, and rule out others. We formulate the hypothesis that semi-supervised learning can help in an anti-causal setting, but not in a causal setting, and corroborate it with empirical results.



We thank Ulf Brefeld and Stefan Wrobel who kindly shared their detailed experimental results with us, allowing for our meta-analysis. We thank Bob Williamson, Vladimir Vapnik, and Jakob Zscheischler for helpful discussions.


  1. 1.
    Brefeld, U., Gärtner, T., Scheffer, T., Wrobel, S.: Efficient co-regularised least squares regression. In: ICML, Pittsburgh (2006)Google Scholar
  2. 2.
    Chapelle, O., Schölkopf, B., Zien, A.: Semi-supervised Learning. MIT, Cambridge (2006)CrossRefGoogle Scholar
  3. 3.
    Daniušis, P., Janzing, D., Mooij, J., Zscheischler, J., Steudel, B., Zhang, K., Schölkopf, B.: Inferring deterministic causal relations. In: UAI, Catalina Island (2010)Google Scholar
  4. 4.
    Fetzer, J.H., Almeder, R.F.: Glossary of Epistemology/Philosophy of Science. Paragon House, New York (1993)Google Scholar
  5. 5.
    Guo, Y., Niu, X., Zhang, H.: An extensive empirical study on semi-supervised learning. In: ICDM, Sydney (2010)Google Scholar
  6. 6.
    Hoyer, P.O., Janzing, D., Mooij, J.M., Peters, J., Schölkopf, B.: Nonlinear causal discovery with additive noise models. In: NIPS, Vancouver (2008)Google Scholar
  7. 7.
    Janzing, D., Schölkopf, B.: Causal inference using the algorithmic Markov condition. IEEE Trans. Inf. Theory 56(10), 5168–5194 (2010)CrossRefGoogle Scholar
  8. 8.
    Lemeire, J., Dirkx, E.: Causal models as minimal descriptions of multivariate systems. (2007)
  9. 9.
    Mooij, J., Janzing, D., Peters, J., Schölkopf, B.: Regression by dependence minimization and its application to causal inference in additive noise models. In: ICML, Montreal (2009)Google Scholar
  10. 10.
    Pearl, J.: Causality. Cambridge University Press, New York (2000)zbMATHGoogle Scholar
  11. 11.
    Pearl, E., Bareinboim, E.: Transportability of causal and statistical relations: a formal approach. In: Proceedings of the 25th AAAI Conference on Artificial Intelligence, Menlo Park, pp. 247–254 (2011)Google Scholar
  12. 12.
    Reichenbach, H.: The Direction of Time. University of California Press, Berkeley (1956)Google Scholar
  13. 13.
    Schölkopf, B., Janzing, D., Peters, J., Sgouritsa, E., Zhang, K., Mooij, J.: On causal and anticausal learning. In: Langford, J., Pineau, J. (eds.) Proceedings of the 29th International Conference on Machine Learning, Edinburgh, pp. 1255–1262. Omnipress, New York (2012)Google Scholar
  14. 14.
    Schweikert, G., Widmer, C., Schölkopf, B., Rätsch, G.: An empirical analysis of domain adaptation algorithms for genomic sequence analysis. In: NIPS, Vancouver (2009)Google Scholar
  15. 15.
    Seeger, M.: Learning with labeled and unlabeled data. Technical Report (Tech. rep.), University of Edinburgh (2001)Google Scholar
  16. 16.
    Seidenfeld, T.: Direct inference and inverse inference. J. Philos. 75(12), 709–730 (1978)CrossRefGoogle Scholar
  17. 17.
    Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search. Springer, New York (1993). (2nd edn.: MIT, Cambridge, 2000)CrossRefzbMATHGoogle Scholar
  18. 18.
    Storkey, A.: When training and test sets are different: characterizing learning transfer. In: Dataset Shift in Machine Learning. MIT, Cambridge (2009)Google Scholar
  19. 19.
    Vapnik, V.: Estimation of Dependences Based on Empirical Data (in Russian). Nauka, Moscow (1979). English translation: Springer, New York, 1982Google Scholar
  20. 20.
    Zhang, K., Hyvärinen, A.: On the identifiability of the post-nonlinear causal model. In: UAI, Montreal (2009)Google Scholar
  21. 21.
    Zhu, X., Goldberg, A.: Introduction to semi-supervised learning. In: Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 3, pp. 1–130. Morgan & Claypool, San Rafael (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Bernhard Schölkopf
    • 1
  • Dominik Janzing
    • 1
  • Jonas Peters
    • 1
  • Eleni Sgouritsa
    • 1
  • Kun Zhang
    • 1
  • Joris Mooij
    • 2
  1. 1.Max Planck Institute for Intelligent SystemsTübingenGermany
  2. 2.Institute for Computing & Information SciencesRadboud UniversityNijmegenNetherlands

Personalised recommendations