Study of Ensemble Strategies in Discovering Linear Causal Models

  • Gang Li
  • Honghua Dai
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3614)


Determining the causal structure of a domain is frequently a key task in the area of Data Mining and Knowledge Discovery. This paper introduces ensemble learning into linear causal model discovery, then examines several algorithms based on different ensemble strategies including Bagging, Adaboost and GASEN. Experimental results show that (1) Ensemble discovery algorithm can achieve an improved result compared with individual causal discovery algorithm in terms of accuracy; (2) Among all examined ensemble discovery algorithms, BWV algorithm which uses a simple Bagging strategy works excellently compared to other more sophisticated ensemble strategies; (3) Ensemble method can also improve the stability of parameter estimation. In addition, Ensemble discovery algorithm is amenable to parallel and distributed processing, which is important for data mining in large data sets.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bollen, K.: Structural Equations with Latent Variables. Wiley, New York (1989)zbMATHGoogle Scholar
  2. 2.
    Wallace, C., Korb, K., Dai, H.: Causal discovery via MML. In: Proceedings of the 13th International Conference on Machine Learning (ICML 1996), pp. 516–524 (1996)Google Scholar
  3. 3.
    Wallace, C., Boulton, D.: An information measure for classification. Computer Journal 11, 185–194 (1968)zbMATHGoogle Scholar
  4. 4.
    Dai, H., Korb, K., Wallace, C., Wu, X.: A study of causal discovery with small samples and weak links. In: Proceedings of the 15th International Joint Conference On Artificial Intelligence IJCAI 1997, pp. 1304–1309. Morgan Kaufmann Publishers, Inc, San Francisco (1997)Google Scholar
  5. 5.
    Dai, H., Li, G.: An improved approach for the discovery of causal models via MML. In: Proceedings of The 6th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2002), Taiwan, pp. 304–315 (2002)Google Scholar
  6. 6.
    Li, G., Dai, H., Tu, Y.: Linear causal model discovery using MML criterion. In: Proceedings of 2002 IEEE International Conference on Data Mining, Maebashi City, Japan, pp. 274–281. IEEE Computer Society, Los Alamitos (2002)Google Scholar
  7. 7.
    Dai, H., Li, G., Tu, Y.: An empirical study of encoding schemes and search strategies in discovering causal networks. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, pp. 48–59. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)zbMATHMathSciNetGoogle Scholar
  9. 9.
    Efron, B., Tibshirani, R.: An introduction to the bootstrap. Chapman and Hall, New York (1993)zbMATHGoogle Scholar
  10. 10.
    Zhou, Z.H., Wu, J., Tang, W.: Enselbling neural networks: Many could be better than all. Artificial Intelligence 137, 993–1001 (2002)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Proceedings of EuroCOLT 1994, Barcelona, Spain, pp. 23–37 (1995)Google Scholar
  12. 12.
    Thollard, F., Sebban, M., Ezequel, P.: Boosting density function estimators. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS (LNAI), vol. 2430, pp. 431–443. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  13. 13.
    Murphy, K.: The Bayes Net Toolbox for Matlab. Computer Science and Statistics 33, 331–351 (2001)Google Scholar
  14. 14.
    Houck, C., Joines, J., Kay, M.: A genetic algorithm for function optimization: a matlab implementation. Technical Reports NCSU-IE-TR-95-09, North Carolina State University, Raleigh, NC (1995)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Gang Li
    • 1
  • Honghua Dai
    • 1
  1. 1.School of Information TechnologyDeakin UniversityAustralia

Personalised recommendations