Empirical Software Engineering

, Volume 23, Issue 2, pp 987–1035 | Cite as

Aggregating Association Rules to Improve Change Recommendation



As the complexity of software systems grows, it becomes increasingly difficult for developers to be aware of all the dependencies that exist between artifacts (e.g., files or methods) of a system. Change recommendation has been proposed as a technique to overcome this problem, as it suggests to a developer relevant source-code artifacts related to her changes. Association rule mining has shown promise in deriving such recommendations by uncovering relevant patterns in the system’s change history. The strength of the mined association rules is captured using a variety of interestingness measures. However, state-of-the-art recommendation engines typically use only the rule with the highest interestingness value when more than one rule applies. In contrast, we argue that when multiple rules apply, this indicates collective evidence, and aggregating those rules (and their evidence) will lead to more accurate change recommendation. To investigate this hypothesis we conduct a large empirical study of 15 open source software systems and two systems from our industry partners. We evaluate association rule aggregation using four variants of the change history for each system studied, enabling us to compare two different levels of granularity in two different scenarios. Furthermore, we study 40 interestingness measures using the rules produced by two different mining algorithms. The results show that (1) between 13 and 90% of change recommendations can be improved by rule aggregation, (2) rule aggregation almost always improves change recommendation for both algorithms and all measures, and (3) fine-grained histories benefit more from rule aggregation.


Evolutionary coupling Targeted association rule mining Rule aggregation Interestingness aggregator Change recommendations Change impact analysis 



This work is supported by the Research Council of Norway through the EvolveIT project (#221751/F20) and the Certus SFI (#203461/030). Dr. Binkley is supported by NSF grant IIA-1360707 and a J. William Fulbright award.


  1. Aggarwal CC, Yu PS (1998) A new framework for itemset generation. In: ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems (PODS), 2. ACM, pp 18–24.  https://doi.org/10.1145/275487.275490
  2. Agrawal R, Imielinski T, Swami A (1993) Mining association rules between sets of items in large databases. In: ACM SIGMOD International Conference on Management of Data. ACM, pp 207–216.  https://doi.org/10.1145/170035.170072
  3. Azė J, Kodratoff Y (2002) Evaluation de la résistance au bruit de quelques mesures d’extraction de règles d’association. In: Extraction et gestion des connaissances (EGC), vol 1. Hermes Science Publications, pp 143–154Google Scholar
  4. Ball T, Kim J, Siy HP (1997) If your version control system could talk. In: Workshop on Process Modelling and Empirical Studies of Software Engineering, ICSE. Scholar
  5. Baralis E, Cagliero L, Cerquitelli T, Garza P (2012) Generalized association rule mining with constraints. Inf Sci 194:68–84.  https://doi.org/10.1016/j.ins.2011.05.016 CrossRefGoogle Scholar
  6. Bayardo RJ (1998) Efficiently mining long patterns from databases. ACM SIGMOD Record 27(2):85–93.  https://doi.org/10.1145/276305.276313 CrossRefGoogle Scholar
  7. Bernard JM, Charron C (1996) Bayesian implicative analysis, a method for the study of oriented dependencies. Mathématiques. Informatique et Sci Humaines 135:5–18MATHGoogle Scholar
  8. Beyer D, Noack A (2005) Clustering software artifacts based on frequent common changes. In: International Workshop on Program Comprehension (IWPC). IEEE, pp 259–268.  https://doi.org/10.1109/WPC.2005.12
  9. Bird C, Menzies T, Zimmermann T (2015) Past, present, and future of analyzing software data. In: The Art and Science of Analyzing Software Data, pp 1–13.  https://doi.org/10.1016/B978-0-12-411519-4.00001-X
  10. Bohner S, Arnold R (1996) Software change impact analysis. IEEE, CA, USAGoogle Scholar
  11. Breiman L, Friedman J, Stone CJ, Olshen RA (1984) Classification and Regression Trees, vol. 19Google Scholar
  12. Brin S, Motwani R, Ullman JD, Tsur S (1997) Dynamic itemset counting and implication rules for market basket data. In: ACM SIGMOD International Conference on Management of Data (SIGMOD), vol 26. ACM, pp 255–264.  https://doi.org/10.1145/253260.253325
  13. Canfora G, Cerulo L (2005) Impact analysis by mining software and change request repositories. In: International Software Metrics Symposium (METRICS). IEEE, pp 29–37x.  https://doi.org/10.1109/METRICS.2005.28
  14. Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46.  https://doi.org/10.1177/001316446002000104 MathSciNetCrossRefGoogle Scholar
  15. Cohen J (1992) A power primer. Psychol Bull 112(1):155–159.  https://doi.org/10.1037/0033-2909.112.1.155 CrossRefGoogle Scholar
  16. Collard ML, Decker MJ, Maletic JI (2013) srcML: an infrastructure for the exploration, analysis, and manipulation of source code: a tool demonstration. In: IEEE International conference on software maintenance (ICSM). IEEE, pp 516–519.  https://doi.org/10.1109/ICSM.2013.85
  17. Eick S, Graves TL, Karr A, Marron J, Mockus A (2001) Does code decay? Assessing the evidence from change management data. IEEE Trans Softw Eng 27(1):1–12. 10.1109/32.895984CrossRefGoogle Scholar
  18. Gall H, Hajek K, Jazayeri M (1998) Detection of logical coupling based on product release history. In: IEEE International conference on software maintenance (ICSM). IEEE, pp 190–198.  https://doi.org/10.1109/ICSM.1998.738508
  19. Geng L, Hamilton HJ (2006) Interestingness measures for data mining. ACM Computing Surveys 38(3).  https://doi.org/10.1145/1132960.1132963
  20. Good IJ (1966) The estimation of probabilities: an essay on modern Bayesian methods. MIT PressGoogle Scholar
  21. Gray B, Orlowska ME (1998) CCAIIA: Clustering categorical attributes into interesting association rules. In: Lecture Notes in Computer Science (LNCS), vol 1394, pp 132–143.  https://doi.org/10.1007/3-540-64383-4_12
  22. Hassan AE, Holt R (2004) Predicting change propagation in software systems. In: IEEE International conference on software maintenance (ICSM). IEEE, pp 284–293.  https://doi.org/10.1109/ICSM.2004.1357812
  23. Hofmann H, Wilhelm A (2001) Visual comparison of association rules. Comput Stat 16(3):399–415.  https://doi.org/10.1007/s001800100075 MathSciNetCrossRefMATHGoogle Scholar
  24. Järvelin K, Kekäläinen J (2002) Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst 20(4):422–446.  https://doi.org/10.1145/582415.582418 CrossRefGoogle Scholar
  25. Jashki MA, Zafarani R, Bagheri E (2008) Towards a more efficient static software change impact analysis method. In: ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering (PASTE). ACM, pp 84–90.  https://doi.org/10.1145/1512475.1512493
  26. Jorge AM, Azevedo PJ (2005) An experiment with association rules and classification: post-bagging and conviction. In: Hoffmann A, Motoda H, Scheffer T (eds) Proceedings of the 8th International Conference on Discovery Science DS 2005, Lecture Notes in Computer Science, vol 3735. Springer, Berlin, pp 137–149.  https://doi.org/10.1007/11563983_13
  27. Kamber M, Shinghal R (1996) Evaluating the interestingness of characteristic rules. In: SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pp 263–266Google Scholar
  28. Kannan S, Bhaskaran R (2009) Association rule pruning based on interestingness measures with clustering. J Comput Sci 6(1):35–43Google Scholar
  29. Klösgen W (1992) Problems for knowledge discovery in databases and their treatment in the statistics interpreter explora. Int J Intell Syst 7(7):649–673.  https://doi.org/10.1002/int.4550070707 CrossRefMATHGoogle Scholar
  30. Kodratoff Y (2001) Comparing machine learning and knowledge discovery in databases: an application to knowledge discovery in texts. In: Machine Learning and Its Applications, LNAI 2049, chap. 1. Springer, pp 1–21.  https://doi.org/10.1007/3-540-44673-7_1
  31. Kulczyński S (1928) Die Pflanzenassoziationen der Pieninen Imprimerie de l’universitéGoogle Scholar
  32. Le TDB, Lo D (2015) Beyond support and confidence: exploring interestingness measures for rule-based specification mining. IEEE, pp 331–340. In: International Conference on Software Analysis, Evolution, and Reengineering (SANER).  https://doi.org/10.1109/SANER.2015.7081843
  33. Lin DI, Kedem ZM (1998) Pincer-search: a new algorithm for discovering the maximum frequent set. pp 103–119.  https://doi.org/10.1007/BFb0100980
  34. Liu B, Hsu W, Ma Y (1999) Pruning and summarizing the discovered associations. In: SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, pp 125–134.  https://doi.org/10.1145/312129.312216
  35. Loevinger J (1947) A systematic approach to the construction and evaluation of tests of ability, vol 61.  https://doi.org/10.1037/h0093565
  36. Lucia, Lo D, Xia X (2014) Fusion fault localizers. In: Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering - ASE ’14. ACM Press, New York, pp 127–138.  https://doi.org/10.1145/2642937.2642983
  37. McGarry K (2005) A survey of interestingness measures for knowledge discovery. Knowl Eng Rev 20(01):39.  https://doi.org/10.1017/S0269888905000408 CrossRefGoogle Scholar
  38. Messaoud RB, Rabaséda S L, Boussaid O, Missaoui R (2006) Enhanced mining of association rules from data cubes. In: International Workshop on Data Warehousing and OLAP (DOLAP). ACM, p 11.  https://doi.org/10.1145/1183512.1183517
  39. Moonen L, Di Alesio S, Rolfsnes T, Binkley DW (2016) Exploring the effects of history length and age on mining software change impact. In: IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM), pp 207–216.  https://doi.org/10.1109/SCAM.2016.9
  40. Mosteller F (1968) Association and estimation in contingency tables. J Am Stat Assoc 63(321):1–28.  https://doi.org/10.1080/01621459.1968.11009219 MathSciNetGoogle Scholar
  41. Pearson K (1896) Mathematical contributions to the theory of evolution. III. Regression, Heredity, and Panmixia. Philosophical Transactions of the Royal Society A: Mathematical. Phys Eng Sci 187:253–318.  https://doi.org/10.1098/rsta.1896.0007 CrossRefGoogle Scholar
  42. Piatetsky-Shapiro G (1991) Discovery, analysis, and presentation of strong rules. Knowledge discovery in databases pp 229—-238Google Scholar
  43. Podgurski A, Clarke L (1990) A formal model of program dependences and its implications for software testing, debugging, and maintenance. IEEE Trans Softw Eng 16(9):965–979.  https://doi.org/10.1109/32.58784 CrossRefGoogle Scholar
  44. Ren X, Shah F, Tip F, Ryder BG, Chesley O (2004) Chianti: a tool for change impact analysis of java programs. In: ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pp 432–448.  https://doi.org/10.1145/1035292.1029012
  45. Robbes R, Pollet D, Lanza M (2008) Logical coupling based on Fine-Grained change information. In: Working Conference on Reverse Engineering (WCRE). IEEE, pp 42–46.  https://doi.org/10.1109/WCRE.2008.47
  46. Rolfsnes T, Di Alesio S, Behjati R, Moonen L, Binkley DW (2016) Generalizing the analysis of evolutionary coupling for software change impact analysis. In: International Conference on Software Analysis, Evolution, and Reengineering (SANER). IEEE, pp 201–212.  https://doi.org/10.1109/SANER.2016.101
  47. Rolfsnes T, Moonen L, Di Alesio S, Behjati R, Binkley DW (2016) Improving change recommendation using aggregated association rules. In: International Conference on Mining Software Repositories (MSR). ACM, pp 73–84.  https://doi.org/10.1145/2901739.2901756
  48. Rosenthal R (1991) Meta-analytic procedures for social research. SAGEGoogle Scholar
  49. Sebag M, Schoenauer M (1988) Generation of rules with certainty and confidence factors from incomplete and incoherent learning bases. In: Proceedings of the european knowledge acquisition workshop (EKAW), p 28Google Scholar
  50. Wang S, Lo D, Jiang L, Lucia, Lau HC (2011) Search-based fault localization. In: 2011 26Th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011). IEEE, pp 556–559.  https://doi.org/10.1109/ASE.2011.6100124
  51. Smyth P, Goodman R (1992) An information theoretic approach to rule induction from databases. IEEE Trans Knowl Data Eng 4(4):301–316.  https://doi.org/10.1109/69.149926 CrossRefGoogle Scholar
  52. Srikant R, Vu Q, Agrawal R (1997) Mining association rules with item constraints. In: International Conference on Knowledge Discovery and Data Mining (KDD). AASI, pp 67–73Google Scholar
  53. Tan PN, Kumar V, Srivastava J (2004) Selecting the right objective measure for association analysis. Inf Syst 29(4):293–313.  https://doi.org/10.1016/S0306-4379(03)00072-3 CrossRefGoogle Scholar
  54. Toivonen H, Klemettinen M, Ronkainen P, Hätönen K, Mannila H (1995) Pruning and grouping discovered association rules. In: Workshop on Statistics, Machine Learning, and Knowledge Discovery in Databases, pp 47–52Google Scholar
  55. Vaillant B, Lenca P, Lallich S (2004) A Clustering of Interestingness Measures. In: Lecture Notes in Artificial Intelligence (LNAI), vol 3245, pp 290–297.  https://doi.org/10.1007/978-3-540-30214-8_23
  56. Van Rijsbergen CJ (1979) Information retrieval. Butterworth-HeinemannGoogle Scholar
  57. Wu T, Chen Y, Han J (2010) Re-examination of interestingness measures in pattern mining: a unified framework. Data Min Knowl Disc 21(3):371–397.  https://doi.org/10.1007/s10618-009-0161-2 MathSciNetCrossRefGoogle Scholar
  58. Yao YY, Zhong N (1999) An analysis of quantitative measures associated with rules. In: Methodologies for Knowledge Discovery and Data Mining (LNCS 1574). Springer, pp 479–488.  https://doi.org/10.1007/3-540-48912-6_64
  59. Yazdanshenas AR, Moonen L (2011) Crossing the boundaries while analyzing heterogeneous component-based software systems. In: IEEE International conference on software maintenance (ICSM). IEEE, pp 193–202.  https://doi.org/10.1109/ICSM.2011.6080786
  60. Ying ATT, Murphy G, Ng RT, Chu-Carroll M (2004) Predicting source code changes by mining change history. IEEE Trans Softw Eng 30(9):574–586.  https://doi.org/10.1109/TSE.2004.52 CrossRefGoogle Scholar
  61. Yong SH, Horwitz S (2002) Reducing the overhead of dynamic analysis. Electron Notes Theor Comput Sci 70(4):158–178.  https://doi.org/10.1016/S1571-0661(04)80583-8 CrossRefGoogle Scholar
  62. Yule GU (1900) On the association of attributes in statistics. Philos Trans R Soc Lond 194:257–319CrossRefMATHGoogle Scholar
  63. Yule GU (1912) On the methods of measuring association between two attributes. J R Stat Soc LXXV:579–652.  https://doi.org/10.2307/2340126 CrossRefGoogle Scholar
  64. Zaki MJ (2000) Generating non-redundant association rules SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, pp 34–43.  https://doi.org/10.1145/347090.347101
  65. Zaki MJ, Hsiao CJ (1999) CHARM: an efficient algorithm for closed association rule mining. In: 2nd SIAM International Conference on Data Mining, pp 457–473.  https://doi.org/10.1137/1.9781611972726.27
  66. Zanjani MB, Swartzendruber G, Kagdi H (2014) Impact analysis of change requests on source code based on interaction and commit histories. In: International Working Conference on Mining Software Repositories (MSR), pp 162–171.  https://doi.org/10.1145/2597073.2597096
  67. Zhang T (2000) Association rules. In: Knowledge Discovery and Data Mining. Current Issues and New Applications, c, pp 245–256.  https://doi.org/10.1007/3-540-45571-X_31
  68. Zimmermann T, Zeller A, Weissgerber P, Diehl S (2005) Mining version histories to guide software changes. IEEE Trans Softw Eng 31(6):429–445.  https://doi.org/10.1109/TSE.2005.72 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Simula Research LaboratoryOsloNorway
  2. 2.Loyola University MarylandBaltimoreUSA

Personalised recommendations