Empirical Software Engineering

, Volume 23, Issue 2, pp 987–1035 | Cite as

Aggregating Association Rules to Improve Change Recommendation

  • Thomas Rolfsnes
  • Leon Moonen
  • Stefano Di Alesio
  • Razieh Behjati
  • Dave Binkley


As the complexity of software systems grows, it becomes increasingly difficult for developers to be aware of all the dependencies that exist between artifacts (e.g., files or methods) of a system. Change recommendation has been proposed as a technique to overcome this problem, as it suggests to a developer relevant source-code artifacts related to her changes. Association rule mining has shown promise in deriving such recommendations by uncovering relevant patterns in the system’s change history. The strength of the mined association rules is captured using a variety of interestingness measures. However, state-of-the-art recommendation engines typically use only the rule with the highest interestingness value when more than one rule applies. In contrast, we argue that when multiple rules apply, this indicates collective evidence, and aggregating those rules (and their evidence) will lead to more accurate change recommendation. To investigate this hypothesis we conduct a large empirical study of 15 open source software systems and two systems from our industry partners. We evaluate association rule aggregation using four variants of the change history for each system studied, enabling us to compare two different levels of granularity in two different scenarios. Furthermore, we study 40 interestingness measures using the rules produced by two different mining algorithms. The results show that (1) between 13 and 90% of change recommendations can be improved by rule aggregation, (2) rule aggregation almost always improves change recommendation for both algorithms and all measures, and (3) fine-grained histories benefit more from rule aggregation.


Evolutionary coupling Targeted association rule mining Rule aggregation Interestingness aggregator Change recommendations Change impact analysis 



This work is supported by the Research Council of Norway through the EvolveIT project (#221751/F20) and the Certus SFI (#203461/030). Dr. Binkley is supported by NSF grant IIA-1360707 and a J. William Fulbright award.


  1. Aggarwal CC, Yu PS (1998) A new framework for itemset generation. In: ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS), 2. ACM, pp 18–24.
  2. Agrawal R, Imielinski T, Swami A (1993) Mining association rules between sets of items in large databases. In: ACM SIGMOD International Conference on Management of Data. ACM, pp 207–216.
  3. Azė J, Kodratoff Y (2002) Evaluation de la résistance au bruit de quelques mesures d’extraction de règles d’association. In: Extraction et gestion des connaissances (EGC), vol 1. Hermes Science Publications, pp 143–154Google Scholar
  4. Ball T, Kim J, Siy HP (1997) If your version control system could talk. In: Workshop on Process Modelling and Empirical Studies of Software Engineering, ICSE. Scholar
  5. Baralis E, Cagliero L, Cerquitelli T, Garza P (2012) Generalized association rule mining with constraints. Inf Sci 194:68–84. CrossRefGoogle Scholar
  6. Bayardo RJ (1998) Efficiently mining long patterns from databases. ACM SIGMOD Record 27(2):85–93. CrossRefGoogle Scholar
  7. Bernard JM, Charron C (1996) Bayesian implicative analysis, a method for the study of oriented dependencies. Mathématiques. Informatique et Sci Humaines 135:5–18zbMATHGoogle Scholar
  8. Beyer D, Noack A (2005) Clustering software artifacts based on frequent common changes. In: International Workshop on Program Comprehension (IWPC). IEEE, pp 259–268.
  9. Bird C, Menzies T, Zimmermann T (2015) Past, present, and future of analyzing software data. In: The Art and Science of Analyzing Software Data, pp 1–13.
  10. Bohner S, Arnold R (1996) Software change impact analysis. IEEE, CA, USAGoogle Scholar
  11. Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and Regression Trees, vol. 19Google Scholar
  12. Brin S, Motwani R, Ullman JD, Tsur S (1997) Dynamic itemset counting and implication rules for market basket data. In: ACM SIGMOD International Conference on Management of Data (SIGMOD), vol 26. ACM, pp 255–264.
  13. Canfora G, Cerulo L (2005) Impact analysis by mining software and change request repositories. In: International Software Metrics Symposium (METRICS). IEEE, pp 29–37x.
  14. Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46. MathSciNetCrossRefGoogle Scholar
  15. Cohen J (1992) A power primer. Psychol Bull 112(1):155–159. CrossRefGoogle Scholar
  16. Collard ML, Decker MJ, Maletic JI (2013) srcML: an infrastructure for the exploration, analysis, and manipulation of source code: a tool demonstration. In: IEEE International conference on software maintenance (ICSM). IEEE, pp 516–519.
  17. Eick S, Graves TL, Karr A, Marron J, Mockus A (2001) Does code decay? Assessing the evidence from change management data. IEEE Trans Softw Eng 27(1):1–12. 10.1109/32.895984CrossRefGoogle Scholar
  18. Gall H, Hajek K, Jazayeri M (1998) Detection of logical coupling based on product release history. In: IEEE International conference on software maintenance (ICSM). IEEE, pp 190–198.
  19. Geng L, Hamilton HJ (2006) Interestingness measures for data mining. ACM Computing Surveys 38(3).
  20. Good IJ (1966) The estimation of probabilities: an essay on modern Bayesian methods. MIT PressGoogle Scholar
  21. Gray B, Orlowska ME (1998) CCAIIA: Clustering categorical attributes into interesting association rules. In: Lecture Notes in Computer Science (LNCS), vol 1394, pp 132–143.
  22. Hassan AE, Holt R (2004) Predicting change propagation in software systems. In: IEEE International conference on software maintenance (ICSM). IEEE, pp 284–293.
  23. Hofmann H, Wilhelm A (2001) Visual comparison of association rules. Comput Stat 16(3):399–415. MathSciNetCrossRefzbMATHGoogle Scholar
  24. Järvelin K, Kekäläinen J (2002) Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst 20(4):422–446. CrossRefGoogle Scholar
  25. Jashki MA, Zafarani R, Bagheri E (2008) Towards a more efficient static software change impact analysis method. In: ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering (PASTE). ACM, pp 84–90.
  26. Jorge AM, Azevedo PJ (2005) An experiment with association rules and classification: post-bagging and conviction. In: Hoffmann A, Motoda H, Scheffer T (eds) Proceedings of the 8th International Conference on Discovery Science DS 2005, Lecture Notes in Computer Science, vol 3735. Springer, Berlin, pp 137–149.
  27. Kamber M, Shinghal R (1996) Evaluating the interestingness of characteristic rules. In: SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp 263–266Google Scholar
  28. Kannan S, Bhaskaran R (2009) Association rule pruning based on interestingness measures with clustering. J Comput Sci 6(1):35–43Google Scholar
  29. Klösgen W (1992) Problems for knowledge discovery in databases and their treatment in the statistics interpreter explora. Int J Intell Syst 7(7):649–673. CrossRefzbMATHGoogle Scholar
  30. Kodratoff Y (2001) Comparing machine learning and knowledge discovery in databases: an application to knowledge discovery in texts. In: Machine Learning and Its Applications, LNAI 2049, chap. 1. Springer, pp 1–21.
  31. Kulczyński S (1928) Die Pflanzenassoziationen der Pieninen Imprimerie de l’universitéGoogle Scholar
  32. Le TDB, Lo D (2015) Beyond support and confidence: exploring interestingness measures for rule-based specification mining. IEEE, pp 331–340. In: International Conference on Software Analysis, Evolution, and Reengineering (SANER).
  33. Lin DI, Kedem ZM (1998) Pincer-search: a new algorithm for discovering the maximum frequent set. pp 103–119.
  34. Liu B, Hsu W, Ma Y (1999) Pruning and summarizing the discovered associations. In: SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, pp 125–134.
  35. Loevinger J (1947) A systematic approach to the construction and evaluation of tests of ability, vol 61.
  36. Lucia, Lo D, Xia X (2014) Fusion fault localizers. In: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering - ASE ’14. ACM Press, New York, pp 127–138.
  37. McGarry K (2005) A survey of interestingness measures for knowledge discovery. Knowl Eng Rev 20(01):39. CrossRefGoogle Scholar
  38. Messaoud RB, Rabaséda S L, Boussaid O, Missaoui R (2006) Enhanced mining of association rules from data cubes. In: International Workshop on Data Warehousing and OLAP (DOLAP). ACM, p 11.
  39. Moonen L, Di Alesio S, Rolfsnes T, Binkley DW (2016) Exploring the effects of history length and age on mining software change impact. In: IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM), pp 207–216.
  40. Mosteller F (1968) Association and estimation in contingency tables. J Am Stat Assoc 63(321):1–28. MathSciNetGoogle Scholar
  41. Pearson K (1896) Mathematical contributions to the theory of evolution. III. Regression, Heredity, and Panmixia. Philosophical Transactions of the Royal Society A: Mathematical. Phys Eng Sci 187:253–318. CrossRefGoogle Scholar
  42. Piatetsky-Shapiro G (1991) Discovery, analysis, and presentation of strong rules. Knowledge discovery in databases pp 229—-238Google Scholar
  43. Podgurski A, Clarke L (1990) A formal model of program dependences and its implications for software testing, debugging, and maintenance. IEEE Trans Softw Eng 16(9):965–979. CrossRefGoogle Scholar
  44. Ren X, Shah F, Tip F, Ryder BG, Chesley O (2004) Chianti: a tool for change impact analysis of java programs. In: ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pp 432–448.
  45. Robbes R, Pollet D, Lanza M (2008) Logical coupling based on Fine-Grained change information. In: Working Conference on Reverse Engineering (WCRE). IEEE, pp 42–46.
  46. Rolfsnes T, Di Alesio S, Behjati R, Moonen L, Binkley DW (2016) Generalizing the analysis of evolutionary coupling for software change impact analysis. In: International Conference on Software Analysis, Evolution, and Reengineering (SANER). IEEE, pp 201–212.
  47. Rolfsnes T, Moonen L, Di Alesio S, Behjati R, Binkley DW (2016) Improving change recommendation using aggregated association rules. In: International Conference on Mining Software Repositories (MSR). ACM, pp 73–84.
  48. Rosenthal R (1991) Meta-analytic procedures for social research. SAGEGoogle Scholar
  49. Sebag M, Schoenauer M (1988) Generation of rules with certainty and confidence factors from incomplete and incoherent learning bases. In: Proceedings of the european knowledge acquisition workshop (EKAW), p 28Google Scholar
  50. Wang S, Lo D, Jiang L, Lucia, Lau HC (2011) Search-based fault localization. In: 2011 26Th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011). IEEE, pp 556–559.
  51. Smyth P, Goodman R (1992) An information theoretic approach to rule induction from databases. IEEE Trans Knowl Data Eng 4(4):301–316. CrossRefGoogle Scholar
  52. Srikant R, Vu Q, Agrawal R (1997) Mining association rules with item constraints. In: International Conference on Knowledge Discovery and Data Mining (KDD). AASI, pp 67–73Google Scholar
  53. Tan PN, Kumar V, Srivastava J (2004) Selecting the right objective measure for association analysis. Inf Syst 29(4):293–313. CrossRefGoogle Scholar
  54. Toivonen H, Klemettinen M, Ronkainen P, Hätönen K, Mannila H (1995) Pruning and grouping discovered association rules. In: Workshop on Statistics, Machine Learning, and Knowledge Discovery in Databases, pp 47–52Google Scholar
  55. Vaillant B, Lenca P, Lallich S (2004) A Clustering of Interestingness Measures. In: Lecture Notes in Artificial Intelligence (LNAI), vol 3245, pp 290–297.
  56. Van Rijsbergen CJ (1979) Information retrieval. Butterworth-HeinemannGoogle Scholar
  57. Wu T, Chen Y, Han J (2010) Re-examination of interestingness measures in pattern mining: a unified framework. Data Min Knowl Disc 21(3):371–397. MathSciNetCrossRefGoogle Scholar
  58. Yao YY, Zhong N (1999) An analysis of quantitative measures associated with rules. In: Methodologies for Knowledge Discovery and Data Mining (LNCS 1574). Springer, pp 479–488.
  59. Yazdanshenas AR, Moonen L (2011) Crossing the boundaries while analyzing heterogeneous component-based software systems. In: IEEE International conference on software maintenance (ICSM). IEEE, pp 193–202.
  60. Ying ATT, Murphy G, Ng RT, Chu-Carroll M (2004) Predicting source code changes by mining change history. IEEE Trans Softw Eng 30(9):574–586. CrossRefGoogle Scholar
  61. Yong SH, Horwitz S (2002) Reducing the overhead of dynamic analysis. Electron Notes Theor Comput Sci 70(4):158–178. CrossRefGoogle Scholar
  62. Yule GU (1900) On the association of attributes in statistics. Philos Trans R Soc Lond 194:257–319CrossRefzbMATHGoogle Scholar
  63. Yule GU (1912) On the methods of measuring association between two attributes. J R Stat Soc LXXV:579–652. CrossRefGoogle Scholar
  64. Zaki MJ (2000) Generating non-redundant association rules SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, pp 34–43.
  65. Zaki MJ, Hsiao CJ (1999) CHARM: an efficient algorithm for closed association rule mining. In: 2nd SIAM International Conference on Data Mining, pp 457–473.
  66. Zanjani MB, Swartzendruber G, Kagdi H (2014) Impact analysis of change requests on source code based on interaction and commit histories. In: International Working Conference on Mining Software Repositories (MSR), pp 162–171.
  67. Zhang T (2000) Association rules. In: Knowledge Discovery and Data Mining. Current Issues and New Applications, c, pp 245–256.
  68. Zimmermann T, Zeller A, Weissgerber P, Diehl S (2005) Mining version histories to guide software changes. IEEE Trans Softw Eng 31(6):429–445. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Simula Research LaboratoryOsloNorway
  2. 2.Loyola University MarylandBaltimoreUSA

Personalised recommendations