Identifying exceptional (dis)agreement between groups

  • Adnene Belfodil
  • Sylvie Cazalens
  • Philippe Lamarre
  • Marc PlantevitEmail author


Under the term behavioral data, we consider any type of data featuring individuals performing observable actions on entities. For instance, voting data depict parliamentarians who express their votes w.r.t. legislative procedures. In this work, we address the problem of discovering exceptional (dis)agreement patterns in such data, i.e., groups of individuals that exhibit an unexpected (dis)agreement under specific contexts compared to what is observed in overall terms. To tackle this problem, we design a generic approach, rooted in the Subgroup Discovery/Exceptional Model Mining framework, which enables the discovery of such patterns in two different ways. A branch-and-bound algorithm ensures an efficient exhaustive search of the underlying search space by leveraging closure operators and optimistic estimates on the interestingness measures. A second algorithm abandons the completeness by using a sampling paradigm which provides an alternative when an exhaustive search approach becomes unfeasible. To illustrate the usefulness of discovering exceptional (dis)agreement patterns, we report a comprehensive experimental study on four real-world datasets relevant to three different application domains: political analysis, rating data analysis and healthcare surveillance.


Supervised pattern mining Subgroup discovery Exceptional model mining Behavioral data analysis 



This work has been partially supported by the project ContentCheck ANR-15-CE23-0025 funded by the French National Research Agency. The authors would like to thank the reviewers for their valuable remarks. Their thoughtful and deep comments allowed us to considerably improve this paper. They also warmly thank Wouter Duivesteijn, Albrecht Zimmermann and Aimene Belfodil for interesting discussions.


  1. Amelio A, Pizzuti C (2012) Analyzing voting behavior in italian parliament: group cohesion and evolution. In: International conference on advances in social networks analysis and mining, ASONAM 2012, Istanbul, Turkey, 26–29 August 2012, pp 140–146.
  2. Amer-Yahia S, Kleisarchaki S, Kolloju NK, Lakshmanan LVS, Zamar RH (2017) Exploring rated datasets with rating maps. In: Proceedings of the 26th international conference on World Wide Web, WWW 2017, Perth, Australia, April 3–7, 2017, pp 1411–1419.
  3. Atzmueller M (2015) Subgroup discovery. Wiley Interdiscip Rev Data Min Knowl Discov 5(1):35–49. CrossRefGoogle Scholar
  4. Atzmüller M, Puppe F (2006) Sd-map—a fast algorithm for exhaustive subgroup discovery. In: Knowledge discovery in databases: PKDD 2006, 10th European conference on principles and practice of knowledge discovery in databases, Berlin, Germany, September 18–22, 2006, Proceedings, pp 6–17. Google Scholar
  5. Bay SD, Pazzani MJ (2001) Detecting group differences: mining contrast sets. Data Min Knowl Discov 5(3):213–246. CrossRefzbMATHGoogle Scholar
  6. Belfodil A, Cazalens S, Lamarre P, Plantevit M (2017) Flash points: discovering exceptional pairwise behaviors in vote or rating data. In: Machine learning and knowledge discovery in databases—European conference, ECML PKDD 2017, Skopje, Macedonia, September 18–22, 2017, Proceedings, Part II, pp 442–458. CrossRefGoogle Scholar
  7. Belfodil A, Cazalens S, Lamarre P, Plantevit M (2019) Identifying exceptional (dis)agreement between groups. Technical report, LIRIS UMR CNRS 5205.
  8. Bendimerad AA, Cazabet R, Plantevit M, Robardet C (2017) Contextual subgraph discovery with mobility models. In: Complex networks and their applications VI—proceedings of complex networks 2017 (The sixth international conference on complex networks and their applications), Complex networks 2017, Lyon, France, November 29–December 1, 2017, pp 477–489. Google Scholar
  9. Bendimerad AA, Plantevit M, Robardet C (2016) Unsupervised exceptional attributed sub-graph mining in urban data. In: IEEE 16th international conference on data mining, ICDM 2016, December 12–15, 2016, Barcelona, Spain, pp 21–30.
  10. Boley M, Horváth T, Poigné A, Wrobel S (2010b) Listing closed sets of strongly accessible set systems with applications to data mining. Theor Comput Sci 411(3):691–700. MathSciNetCrossRefzbMATHGoogle Scholar
  11. Boley M, Gärtner T, Grosskreutz H (2010a) Formal concept sampling for counting and threshold-free local pattern mining. In: Proceedings of the SIAM international conference on data mining, SDM 2010, April 29–May 1, 2010, Columbus, Ohio, USA, pp 177–188.
  12. Boley M, Lucchese C, Paurat D, Gärtner T (2011) Direct local pattern sampling by efficient two-step random procedures. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining, San Diego, CA, USA, August 21–24, 2011, pp 582–590.
  13. Boley M, Moens S, Gärtner T (2012) Linear space direct pattern sampling using coupling from the past. In: The 18th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’12, Beijing, China, August 12–16, 2012, pp 69–77.
  14. Bosc G, Boulicaut J, Raïssi C, Kaytoue M (2018) Anytime discovery of a diverse set of patterns with monte carlo tree search. Data Min Knowl Discov 32(3):604–650. MathSciNetCrossRefzbMATHGoogle Scholar
  15. Bosc G, Golebiowski J, Bensafi M, Robardet C, Plantevit M, Boulicaut J, Kaytoue M (2016) Local subgroup discovery for eliciting and understanding new structure-odor relationships. In: Discovery science—19th international conference, DS 2016, Bari, Italy, October 19–21, 2016, Proceedings, pp 19–34. CrossRefGoogle Scholar
  16. Charalabidis Y, Alexopoulos C, Loukis E (2016) A taxonomy of open government data research areas and topics. J Organ Comput Electron Commer 26(1–2):41–63CrossRefGoogle Scholar
  17. Csisz I et al (1967) Information-type measures of difference of probability distributions and indirect observations. Stud Sci Math Hungar 2:299–318MathSciNetGoogle Scholar
  18. Das M, Amer-Yahia S, Das G, Yu C (2011) MRI: meaningful interpretations of collaborative ratings. PVLDB 4(11):1063–1074Google Scholar
  19. de Sá CR, Duivesteijn W, Azevedo PJ, Jorge AM, Soares C, Knobbe AJ (2018) Discovering a taste for the unusual: exceptional models for preference mining. Mach Learn 107(11):1775–1807. MathSciNetCrossRefzbMATHGoogle Scholar
  20. de Sá CR, Duivesteijn W, Soares C, Knobbe AJ (2016) Exceptional preferences mining. In: Discovery science—19th international conference, DS 2016, Bari, Italy, October 19–21, 2016, Proceedings, pp 3–18. CrossRefGoogle Scholar
  21. Dong G, Li J (1999) Efficient mining of emerging patterns: discovering trends and differences. In: Proceedings of the fifth ACM SIGKDD international conference on knowledge discovery and data mining, San Diego, CA, USA, August 15–18, 1999, pp 43–52.
  22. Downar L, Duivesteijn W (2017) Exceptionally monotone models–the rank correlation model class for exceptional model mining. Knowl Inf Syst 51(2):369–394. CrossRefGoogle Scholar
  23. Duivesteijn W, Feelders A, Knobbe AJ (2016) Exceptional model mining—supervised descriptive local pattern mining with complex target concepts. Data Min Knowl Discov 30(1):47–98. MathSciNetCrossRefzbMATHGoogle Scholar
  24. Duivesteijn W, Knobbe AJ, Feelders A, van Leeuwen M (2010) Subgroup discovery meets bayesian networks—an exceptional model mining approach. In: ICDM 2010, the 10th IEEE international conference on data mining, Sydney, Australia, 14–17 December 2010, pp 158–167.
  25. Dzyuba V, van Leeuwen M, Raedt LD (2017) Flexible constrained sampling with guarantees for pattern mining. Data Min Knowl Discov 31(5):1266–1293. MathSciNetCrossRefzbMATHGoogle Scholar
  26. Etter V, Herzen J, Grossglauser M, Thiran P (2014) Mining democracy. In: Proceedings of the second ACM conference on Online social networks, COSN 2014, Dublin, Ireland, October 1–2, 2014, pp 1–12.
  27. Fürnkranz J, Gamberger D, Lavrac N (2012) Foundations of rule learning. Cognitive technologies. Springer, Berlin. CrossRefzbMATHGoogle Scholar
  28. Ganter B, Wille R (1999) Formal concept analysis—mathematical foundations. Springer, Berlin. CrossRefzbMATHGoogle Scholar
  29. Ganter B, Kuznetsov SO (2001) Pattern structures and their projections. In: Delugach HS, Stumme G (eds) Conceptual structures: broadening the base, 9th international conference on conceptual structures, ICCS 2001, Stanford, CA, USA, July 30–August 3, 2001, Proceedings, Springer, Lecture notes in computer science, vol 2120, pp 129–142. CrossRefGoogle Scholar
  30. Giacometti A, Soulet A (2016) Frequent pattern outlier detection without exhaustive mining. In: Advances in knowledge discovery and data mining—20th Pacific-Asia conference, PAKDD 2016, Auckland, New Zealand, April 19–22, 2016, Proceedings, Part II, pp 196–207. CrossRefGoogle Scholar
  31. Grosskreutz H, Rüping S (2009) On subgroup discovery in numerical domains. Data Min Knowl Discov 19(2):210–226. MathSciNetCrossRefGoogle Scholar
  32. Grosskreutz H, Boley M, Krause-Traudes M (2010) Subgroup discovery for election analysis: a case study in descriptive data mining. In: Discovery science—13th international conference, DS 2010, Canberra, Australia, October 6–8, 2010. Proceedings, pp 57–71. CrossRefGoogle Scholar
  33. Grosskreutz H, Lang B, Trabold D (2013) A relevance criterion for sequential patterns. In: Machine learning and knowledge discovery in databases—European conference, ECML PKDD 2013, Prague, Czech Republic, September 23–27, 2013, Proceedings, Part I, pp 369–384. CrossRefGoogle Scholar
  34. Grosskreutz H, Rüping S, Wrobel S (2008) Tight optimistic estimates for fast subgroup discovery. In: Machine learning and knowledge discovery in databases, European conference, ECML/PKDD 2008, Antwerp, Belgium, September 15–19, 2008, Proceedings, Part I, pp 440–456.
  35. Harper FM, Konstan JA (2016) The movielens datasets: history and context. TiiS 5(4):19:1–19:19. CrossRefGoogle Scholar
  36. Hasan MA, Zaki MJ (2009) Output space sampling for graph patterns. PVLDB 2(1):730–741. CrossRefGoogle Scholar
  37. Hayes AF, Krippendorff K (2007) Answering the call for a standard reliability measure for coding data. Commun Methods Meas 1(1):77–89CrossRefGoogle Scholar
  38. Herrera F, Carmona CJ, González P, del Jesús MJ (2011) An overview on subgroup discovery: foundations and applications. Knowl Inf Syst 29(3):495–525. CrossRefGoogle Scholar
  39. Hix S, Noury A, Roland G (2005) Power to the parties: cohesion and competition in the european parliament, 1979–2001. Br J Polit Sci 35(2):209–234CrossRefGoogle Scholar
  40. Jakulin A (2004) Analyzing the us senate in 2003: similarities, networks, clusters and blocs. Accessed 18 Oct 2019
  41. Johnson D, Sinanovic S (2001) Symmetrizing the Kullback-Leibler distance. IEEE Trans Inf Theory 47:1–8Google Scholar
  42. Kaytoue M, Kuznetsov SO, Napoli A, Duplessis S (2011) Mining gene expression data with pattern structures in formal concept analysis. Inf Sci 181(10):1989–2001. MathSciNetCrossRefGoogle Scholar
  43. Kaytoue M, Plantevit M, Zimmermann A, Bendimerad AA, Robardet C (2017) Exceptional contextual subgraph mining. Mach Learn 106(8):1171–1211. MathSciNetCrossRefzbMATHGoogle Scholar
  44. Klösgen W (1996) Explora: a multipattern and multistrategy discovery assistant. In: Fayyad UM, Piatetsky-Shapiro G, Smyth P, Uthurusamy R (eds) Advances in knowledge discovery and data mining. MIT Press, Cambridge, pp 249–271Google Scholar
  45. Kuznetsov SO, Obiedkov SA (2002) Comparing performance of algorithms for generating concept lattices. J Exp Theor Artif Intell 14(2–3):189–216. CrossRefzbMATHGoogle Scholar
  46. Lavrac N, Kavsek B, Flach PA, Todorovski L (2004) Subgroup discovery with CN2-SD. J Mach Learn Res 5:153–188MathSciNetGoogle Scholar
  47. Leman D, Feelders A, Knobbe AJ (2008) Exceptional model mining. In: Machine learning and knowledge discovery in databases, European conference, ECML/PKDD 2008, Antwerp, Belgium, September 15–19, 2008, Proceedings, Part II, pp 1–16.
  48. Lemmerich F, Atzmueller M, Puppe F (2016) Fast exhaustive subgroup discovery with numerical target concepts. Data Min Knowl Discov 30(3):711–762. MathSciNetCrossRefzbMATHGoogle Scholar
  49. Lemmerich F, Becker M (2018) pysubgroup: Easy-to-use subgroup discovery in python. In: Machine learning and knowledge discovery in databases—European conference, ECML PKDD 2018, Dublin, Ireland, September 10–14, 2018, Proceedings, Part III, pp 658–662. CrossRefGoogle Scholar
  50. Li G, Zaki MJ (2016) Sampling frequent and minimal boolean patterns: theory and application in classification. Data Min Knowl Discov 30(1):181–225. MathSciNetCrossRefzbMATHGoogle Scholar
  51. Liu B, Hsu W, Ma Y (1998) Integrating classification and association rule mining. In: Proceedings of the fourth international conference on knowledge discovery and data mining (KDD-98), New York City, NY, USA, August 27–31, 1998, pp 80–86.
  52. Moens S, Boley M (2014) Instant exceptional model mining using weighted controlled pattern sampling. In: Advances in intelligent data analysis XIII—13th international symposium, IDA 2014, Leuven, Belgium, October 30–November 1, 2014. Proceedings, pp 203–214. CrossRefGoogle Scholar
  53. Moens S, Goethals B (2013) Randomly sampling maximal itemsets. In: Proceedings of the ACM SIGKDD workshop on interactive data exploration and analytics, IDEA@KDD 2013, Chicago, IL, USA, August 11, 2013, pp 79–86.
  54. Novak PK, Lavrac N, Webb GI (2009) Supervised descriptive rule discovery: a unifying survey of contrast set, emerging pattern and subgroup mining. J Mach Learn Res 10:377–403zbMATHGoogle Scholar
  55. Omidvar-Tehrani B, Amer-Yahia S, Dutot P, Trystram D (2016) Multi-objective group discovery on the social web. In: Machine learning and knowledge discovery in databases—European conference, ECML PKDD 2016, Riva del Garda, Italy, September 19–23, 2016, Proceedings, Part I, pp 296–312. CrossRefGoogle Scholar
  56. Orueta JF, Nuño-Solinis R, Mateos M, Vergara I, Grandes G, Esnaola S (2012) Monitoring the prevalence of chronic conditions: which data should we use? BMC Health Serv Res 12(1):365CrossRefGoogle Scholar
  57. Pajala A, Jakulin A, Buntine W (2004) Parliamentary group and individual voting behavior in finnish parliament in year 2003: a group cohesion and voting similarity analysis. Accessed 18 Oct 2019
  58. Pasquier N, Bastide Y, Taouil R, Lakhal L (1999) Discovering frequent closed itemsets for association rules. In: Database theory—ICDT ’99, 7th international conference, Jerusalem, Israel, January 10–12, 1999, Proceedings, pp 398–416. Google Scholar
  59. Roddy E, Doherty M (2010) Epidemiology of gout. Arthritis Res Ther 12(6):223CrossRefGoogle Scholar
  60. Roman S (2008) Lattices and ordered sets. Springer, BerlinzbMATHGoogle Scholar
  61. Terada A, Okada-Hatakeyama M, Tsuda K, Sese J (2013) Statistical significance of combinatorial regulations. Proc Natl Acad Sci 110(32):12996–13001MathSciNetCrossRefGoogle Scholar
  62. Tukey JW (1977) Exploratory data analysis. Addison-Wesley series in behavioral science: quantitative methods. Addison-Wesley. Accessed 18 Oct 2019
  63. van Leeuwen M, Knobbe AJ (2012) Diverse subgroup set discovery. Data Min Knowl Discov 25(2):208–242. MathSciNetCrossRefGoogle Scholar
  64. Wang C, Crapo LM (1997) The epidemiology of thyroid disease and implications for screening. Endocrinol Metab Clin 26(1):189–218CrossRefGoogle Scholar
  65. Wrobel S (1997) An algorithm for multi-relational discovery of subgroups. In: Principles of data mining and knowledge discovery, first European symposium, PKDD ’97, Trondheim, Norway, June 24–27, 1997, Proceedings, pp 78–87. CrossRefGoogle Scholar

Copyright information

© The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.LIRIS UMR5205, CNRSINSA LyonVilleurbanneFrance
  2. 2.LIRIS UMR5205, CNRSUniversité Lyon 1VilleurbanneFrance

Personalised recommendations