Encyclopedia of Machine Learning

2010 Edition
| Editors: Claude Sammut, Geoffrey I. Webb

Phase Transitions in Machine Learning

  • Lorenza Saitta
  • Michele Sebag
Reference work entry
DOI: https://doi.org/10.1007/978-0-387-30164-8_635



Phase transition (PT) is a term originally used in physics to denote the transformation of a system from a liquid, solid, or gas state (phase) to another. It is used, by extension, to describe any abrupt and sudden change in one of the order parameters describing an arbitrary system, when a control parameter approaches a critical value (While early studies on PTs in computer science inverted the notions of order and control parameters, this article will stick to the original definition used in Statistical Physics.).

Far from being limited to physical systems, PTs are ubiquitous in sciences, notably in computational science. Typically, hard combinatorial problems display a PT with regard to the probability of existence of a solution. Note that the notion of PT cannot be studied in relation to single-problem instances: it refers to emergent phenomena in an ensembleof...

This is a preview of subscription content, log in to check access.

Recommended Reading

  1. Ales Bianchetti, J., Rouveirol, C., & Sebag, M. (2002). Constraint-based learning of long relational concepts. In C. Sammut (Ed.), Proceedings of international conference on machine learning, ICML’02, (pp. 35–42). San Francisco, CA: Morgan Kauffman.Google Scholar
  2. Alphonse, E., & Osmani, A. (2008). On the connection between the phase transition of the covering test and the learning success rate. Machine Learning, 70(2–3), 135–150.CrossRefGoogle Scholar
  3. Baskiotis, N., & Sebag, M. (2004). C4.5 competence map: A phase transition-inspired approach. In Proceedings of international conference on machine learning, Banff, Alberta, Canada (pp. 73–80). Morgan Kaufman.Google Scholar
  4. Botta, M., Giordana, A., & Saitta, L. (1999). An experimental study of phase transitions in matching. In Proceedings of the 16th international joint conference on artificial intelligence, Stockholm, Sweden (pp. 1198–1203).Google Scholar
  5. Botta, M., Giordana, A., Saitta, L., & Sebag, M. (2003). Relational learning as search in a critical region. Journal of Machine Learning Research, 4, 431–463.MathSciNetCrossRefGoogle Scholar
  6. Cands, E. J. (2008). The restricted isometry property and its implications for compressed sensing. Compte Rendus de l’Academie des Sciences, Paris, Serie I, 346, 589–592.Google Scholar
  7. Cheeseman, P., Kanefsky, B., & Taylor, W. (1991). Where the really hard problems are. In R. Myopoulos & J. Reiter (Eds.), Proceedings of the 12th international joint conference on artificial intelligence, Sydney, Australia (pp. 331–340). San Francisco, CA: Morgan Kaufmann.Google Scholar
  8. Cornuéjols, A., & Sebag, M. (2008). A note on phase transitions and computational pitfalls of learning from sequences. Journal of Intelligent Information Systems, 31(2), 177–189.CrossRefGoogle Scholar
  9. Cortes, C., & Vapnik, V. N. (1995). Support-vector networks. Machine Learning, 20, 273–297.zbMATHGoogle Scholar
  10. De Raedt, L. (1997). Logical setting for concept-learning. Artificial Intelligence, 95, 187–202.MathSciNetzbMATHCrossRefGoogle Scholar
  11. De Raedt, L. (1998). Attribute-value learning versus inductive logic programming: The missing links. In Proceedings inductive logic programming, ILP, LNCS, (Vol. 2446, pp. 1–8). London: Springer.Google Scholar
  12. Demongeot, J., & Sené, S. (2008). Boundary conditions and phase transitions in neural networks. Simulation results. Neural Networks, 21(7), 962–970.Google Scholar
  13. Dietterich, T., Lathrop, R., & Lozano-Perez, T. (1997). Solving the multiple-instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1–2), 31–71.zbMATHCrossRefGoogle Scholar
  14. Donoho, D. L., & Tanner, J. (2005). Sparse nonnegative solution of underdetermined linear equations by linear programming. Proceedings of the National Academy of Sciences, 102(27), 9446–9451.MathSciNetCrossRefGoogle Scholar
  15. Engel, A., & Van den Broeck, C. (2001). Statistical mechanics of learning. Cambridge: Cambridge University Press.zbMATHGoogle Scholar
  16. Gaudel, R., Sebag, M., & Cornuéjols, A. (2007). A phase transition-based perspective on multiple instance kernels. In Proceedings of international conference on inductive logic programming, ILP, Corvallis, OR (pp. 112–121).Google Scholar
  17. Gaudel, R., Sebag, M., & Cornuéjols, A. (2008). A phase transition-based perspective on multiple instance kernels. Lecture notes in computer sciences, (Vol. 4894, pp. 112–121).Google Scholar
  18. Giordana, A., & Saitta, L. (2000). Phase transitions in relational learning. Machine Learning, 41(2), 17–251.CrossRefGoogle Scholar
  19. Haussler, D. (1999). Convolutional kernels on discrete structures. Tech. Rep., Computer Science Department, University of California at Santa Cruz.Google Scholar
  20. Hogg, T., Huberman, B. A., & Williams, C. P. (Eds.). (1996). Artificial intelligence: Special Issue on frontiers in problem solving: Phase transitions and complexity, (Vol. 81(1–2)) . Elsevier.Google Scholar
  21. Kramer, S., Lavrac, N., & Flach, P. (2001). Propositionalization approaches to relational data mining. In S. Dzeroski & N. Lavrac (Eds.), Relational data mining, (pp. 262–291). New York: Springer.Google Scholar
  22. Maloberti, J., & Sebag, M. (2004). Fast theta-subsumption with constraint satisfaction algorithms. Machine Learning Journal, 55, 137–174.zbMATHCrossRefGoogle Scholar
  23. Mitchell, T. M. (1982). Generalization as search. Artificial Intelligence, 18, 203–226.MathSciNetCrossRefGoogle Scholar
  24. Plotkin, G. (1970). A note on inductive generalization. In Machine Intelligence, (Vol. 5). Edinburgh University Press.Google Scholar
  25. Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Francisco, CA: Morgan Kaufmann.Google Scholar
  26. Rückert, U., & De Raedt, L. (2008). An experimental evaluation of simplicity in rule learning. Artificial Intelligence, 172(1), 19–28.zbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Lorenza Saitta
    • 1
  • Michele Sebag
    • 1
  1. 1.Università del Piemonte OrientaleAlessandriaItaly