Data Mining and Knowledge Discovery

, Volume 21, Issue 2, pp 259–276 | Cite as

Maximal exceptions with minimal descriptions

  • Matthijs van LeeuwenEmail author
Open Access


We introduce a new approach to Exceptional Model Mining. Our algorithm, called EMDM, is an iterative method that alternates between Exception Maximisation and Description Minimisation. As a result, it finds maximally exceptional models with minimal descriptions. Exceptional Model Mining was recently introduced by Leman et al. (Exceptional model mining 1–16, 2008) as a generalisation of Subgroup Discovery. Instead of considering a single target attribute, it allows for multiple ‘model’ attributes on which models are fitted. If the model for a subgroup is substantially different from the model for the complete database, it is regarded as an exceptional model. To measure exceptionality, we propose two information-theoretic measures. One is based on the Kullback–Leibler divergence, the other on Krimp. We show how compression can be used for exception maximisation with these measures, and how classification can be used for description minimisation. Experiments show that our approach efficiently identifies subgroups that are both exceptional and interesting.


Exceptional Model Mining Subgroup Discovery Information theory 


  1. Andritsos P, Tsaparas P, Miller RJ, Sevcik KC (2004) LIMBO: scalable clustering of categorical data. In: Proceedings of the EDBT, pp 124–146Google Scholar
  2. Asuncion A, Newman DJ (2007) UCI machine learning repository.
  3. Cohen WW (1995) Fast effective rule induction. In: Proceedings of the ICML’95, pp 115–123Google Scholar
  4. Garriga GC, Heikinheimo H, Seppänen JK (2007) Cross-mining binary and numerical attributes. In: Proceedings of the ICDM’07, pp 481–486Google Scholar
  5. Heikinheimo H, Fortelius M, Eronen J, Mannila H (2007) Biogeography of european land mammals shows environmentally distinct and spatially coherent clusters. J Biogeogr 34(6): 1053–1064CrossRefGoogle Scholar
  6. Klösgen W (2002) Subgroup discovery chapter 16.3. Oxford University Press, OxfordGoogle Scholar
  7. Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat 22(1): 79–86zbMATHCrossRefMathSciNetGoogle Scholar
  8. Leeuwen M, Vreeken J, Siebes A (2006) Compression picks the item sets that matter. In: Proceedings of the ECML PKDD’06 pp 585–592Google Scholar
  9. Leeuwen M, Bonchi F, Sigurbjörnsson B, Siebes A (2009) Compressing tags to find interesting media groups. In: Proceedings of the CIKM’09, pp 1147–1156Google Scholar
  10. Leman D, Feelders A, Knobbe A (2008) Exceptional model mining. In: Proceedings of the ECML/ PKDD’08, 2:1–16Google Scholar
  11. Mitchell-Jones AJ, Amori G, Bogdanowicz W, Krystufek B, Reijnders PJH, Spitzenberger F, Stubbe M, Thissen JBM, Vohralik V, Zima J (1999) The atlas of european mammals. Academic Press, LondonGoogle Scholar
  12. Rissanen J (1978) Modeling by shortest data description. Automatica 14(1): 465–471zbMATHCrossRefGoogle Scholar
  13. Siebes A, Vreeken J, van Leeuwen M (2006) Item sets that compress. In: Proceedings of the SDM’06, pp 393–404Google Scholar
  14. Slonim N, Tishby N (1999) Agglomerative information bottleneck. In: Proceedings of the NIPS’99, pp 617–623Google Scholar
  15. Tsoumakas G, Vilcek J, Spyromitros L (2010) MULAN: a java library for multi-label learning.
  16. Umek L, Zupan B, Toplak M, Morin A, Chauchat J-H, Makovec G, Smrke D (2009) Subgroup discovery in data sets with multi-dimensional responses: A method and a case study in traumatology. In: Proceedings of AIME’09, pp 265–274Google Scholar
  17. Warner HR, Toronto AF, Veasey LR, Stephenson R (1961) A mathematical model for medical diagnosis, application to congenital heart disease. J Am Med Assoc 177: 177–184Google Scholar
  18. Witten IH, Frank Eibe (2005) Data mining: practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San FranciscozbMATHGoogle Scholar

Copyright information

© The Author(s) 2010

Open AccessThis is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Authors and Affiliations

  1. 1.Department of Information and Computing SciencesUniversiteit UtrechtUtrechtThe Netherlands

Personalised recommendations