Advertisement

Knowledge and Information Systems

, Volume 6, Issue 2, pp 164–187 | Cite as

Collective Mining of Bayesian Networks from Distributed Heterogeneous Data

  • R. Chen
  • K. SivakumarEmail author
  • H. Kargupta
Article

Abstract

We present a collective approach to learning a Bayesian network from distributed heterogeneous data. In this approach, we first learn a local Bayesian network at each site using the local data. Then each site identifies the observations that are most likely to be evidence of coupling between local and non-local variables and transmits a subset of these observations to a central site. Another Bayesian network is learnt at the central site using the data transmitted from the local site. The local and central Bayesian networks are combined to obtain a collective Bayesian network, which models the entire data. Experimental results and theoretical justification that demonstrate the feasibility of our approach are presented.

Keywords

Bayesian network Collective data mining Distributed data mining Heterogeneous data Web log mining 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    1991Abe:1991 Abe N, Takeuchi J, Warmuth M (1991) Polynomial learnability of probabilistic concepts with respect to the Kullback-Leibler divergence. In Proceedings of the 1991 Workshop on Computational Learning Theory, Santa Cruz, CA, pp 277–289Google Scholar
  2. 2.
    1997Bauer:1997 Bauer E, Koller D, Singer Y (1997) Update rules for parameter estimation in Bayesian networks. In Geiger D, Shanoy P (eds). Proceedings of the 13th conference on uncertainty in artificial intelligence. Morgan Kaufmann, Santa Mateo, CA, pp 3–13Google Scholar
  3. 3.
    1989Beinlich:1989 Beinlich I, Suermondt H, Chavez R, Cooper G (1989) The alarm monitoring system: a case study with two probabilistic inference techniques for belief networks. In Proceedings of the 2nd European conference on artificial intelligence in medical care. Springer, Berlin, pp 247–256Google Scholar
  4. 4.
    1997Binder:1997 Binder J, Koller D, Russel S, Kanazawa K (1997) Adaptive probabilistic networks with hidden variables. Machine Learning 29: 213–244CrossRefGoogle Scholar
  5. 5.
    1994Bouckaert:1994 Bouckaert RR (1994) Properties of Bayesian network learning algorithms. In de~Mantaras RL, Poole D (eds). Proceedings of the 10th conference on uncertainty in artificial intelligence. Morgan Kaufmann, Ssan Mateo, CA, pp 102–109Google Scholar
  6. 6.
    1998Breese:1998 Breese JS, Heckerman D, Kadie C, (1998) Empirical analysis of predictive algorithms for collaborative filtering. In Cooper GF, Moral S (eds). Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann, San Mateo, CA pp. 43–52Google Scholar
  7. 7.
    1991Buntine:1991 Buntine W (1991) Theory refinement on Bayesian networks. In D’Ambrosio BD, Bonissone PP, (eds) Proceedings of the 7th annual conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, CA, pp 52–60Google Scholar
  8. 8.
    1996Cheeseman:1995 Cheeseman P, Stutz J (1996) Bayesian classification (autoclass): theory and results. In Fayyad U, Shapiro GP, Smyth P, Uthurasamy RS (eds). Advances in knowledge discovery and data mining. AAAI Press, Menlo Park, CA. pp. 153–180Google Scholar
  9. 9.
    2002Chen:2002 Chen R, Sivakumar K (2002) A new algorithm for learning parameters of a Bayesian network from distributed data. In Proceedings of the 2002 IEEE international conference on data mining. Maebashi City, Japan. pp. 585–588Google Scholar
  10. 10.
    2001aChen:2001a Chen R, Sivakumar K, Kargupta H (2001a) An approach to online Bayesian learning from multiple data streams. In Kargupta H, Sivakumar K, Wirth R (eds). Proceedings of the workshop on ubiquitous data mining: technology for mobile and distributed KDD (5th European conference, PKDD 2001). Freiburg, Germany, pp 31–45Google Scholar
  11. 11.
    2001bChen:2001b Chen R, Sivakumar K, Kargupta H (2001b) Distributed web mining using Bayesian networks from multiple data streams. In Proceedings of the 2001 IEEE international conference on data mining. San Jose, CA. pp. 75–82Google Scholar
  12. 12.
    1997Cheng:1997 Cheng J, Bell DA, Liu W (1997) Learning belief networks from data: an information theory based approach. In proceedings of the 6th ACM international conference on information and knowledge management.Google Scholar
  13. 13.
    1996Chickering:1996 Chickering DM (1996) Learning equivalence classes of Bayesian network structure. In Horvitz E, Jensen F (eds). Proceedings of the 12th conference on uncertainty in artificial intelligence, Morgan Kaufmann, San Mateo, CAGoogle Scholar
  14. 14.
    1997Chickering:1997 Chickering DM, Heckerman D (1997) Efficient approximation for the marginal likelihood of incomplete data given a Bayesian network. Machine Learning 29:181–212CrossRefGoogle Scholar
  15. 15.
    1992Cooper:1992 Cooper GF, Herskovits E (1992) A Bayesian method for the induction of probabilistic networks from data. Machine Learning 9: 309–347CrossRefGoogle Scholar
  16. 16.
    1997Dasgupta:1997 Dasgupta S (1997) The sample complexity of learning fixed-structure Bayesian networks. Machine Learning 29: 165–180CrossRefGoogle Scholar
  17. 17.
    1995Ezawa:1995 Ezawa KJ, Schuermann T (1995) Fraud/uncollectable debt detection using Bayesian network based learning system: a rare binary outcome with mixed data structures. In Besnard P, Hanks S (eds). Proceedings of the 11th conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, CA, pp 157–166Google Scholar
  18. 18.
    1998Friedman:1998 Friedman N (1998) The Bayesian structural EM algorithm. In Cooper GF, Moral S, (eds). Proceedings of the 14th conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, CAGoogle Scholar
  19. 19.
    1997Friedman:1997a Friedman N, Geiger D, Goldszmidt M (1997), Bayesian network classifiers. Machine Learning 29:131–163CrossRefGoogle Scholar
  20. 20.
    1997Friedman:1997b Friedman N, Goldszmidt M (1997) Sequential update of Bayesian network structure. In Geiger D, Shanoy P (eds). Proceedings of the 13th conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, CAGoogle Scholar
  21. 21.
    1996Friedman:96 Friedman N, Yakhini Z (1996) On the sample complexity of learning Bayesian networks. In Proceedings of the twelfth conference on uncertainty in artificial intelligence.Google Scholar
  22. 22.
    1996Gilks:1996 Gilks W, Richardson S, Spiegelhalter D (1996), Markov chain Monte Carlo in practice. Chapman Hall London.Google Scholar
  23. 23.
    1998Heckerman:1998 Heckerman D (1998) A tutorial on learning with Bayesian networks. In Jordan MI (ed). Proceedings of the NATO advanced study institute on learning in graphical models. Kluwer, DordrechtGoogle Scholar
  24. 24.
    1995Heckerman:1995a Heckerman D, Geiger D, Chickering DM (1995) Learning Bayesian networks: the combination of knowledge and statistical data. Machine Learning 20:197–243CrossRefGoogle Scholar
  25. 25.
    1995Heckerman:1995 Heckerman D, Gieger D (1995) Learning Bayesian networks: a unification for discrete and Gaussian domains. In Besnard P, Hanks S (eds). Proceedings of the 11th conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, CA, pp 274–284Google Scholar
  26. 26.
    2001Hershberger:01 Hershberger D, Kargupta H (2001), Distributed multivariate regression using wavelet-based collective data mining. Journal of Parallel and Distributed Computing 61:372–400CrossRefGoogle Scholar
  27. 27.
    1996Jensen:1996 Jensen F (1996) An introduction to Bayesian networks, Springer, Berlin.Google Scholar
  28. 28.
    2001Kargupta:01 Kargupta H, Huang W, Sivakumar K, Johnson E (2001), Distributed clustering using collective principal component analysis. Knowledge and Information Systems Journal 3:422–448Google Scholar
  29. 29.
    2000Kargupta:00c Kargupta H, Park B, Hershberger D, Johnson E (2000) Collective data mining: a new perspective toward distributed data mining. In Kargupta H, Chan P (eds). Advances in distributed and parallel knowledge discovery. AAAI/ MIT Press, Menlo Park, CA, pp 133–184Google Scholar
  30. 30.
    1997Kenji:1997 Kenji Y (1997) Distributed cooperative Bayesian learning strategies. In Proceedings of the 10th annual conference on computational learning theory. ACM Press, Nashville, TN, pp 250–262Google Scholar
  31. 31.
    1994Lam:1994 Lam W, and Bacchus F, (1994) Learning Bayesian belief networks: an approach based on the MDL principle. Computational intelligence 10:262–293Google Scholar
  32. 32.
    1995Lauritzen:1995 Lauritzen SL (1995) The EM algorithm for graphical association models with missing data. Computational Statistics and Data Analysis 19:191–201CrossRefGoogle Scholar
  33. 33.
    1988Lauritzen:1988 Lauritzen SL, Spiegelhalter DJ (1988) Local computations with probabilities on graphical structures and their application to expert systems (with discussion). Journal of the Royal Statistical Society, series B 50:157–224Google Scholar
  34. 34.
    1994Madigan:1994 Madigan D, Raftery A (1994) Model selection and accounting for model uncertainty in graphical models using Occam’s window. Journal of the American Statistical Association 89:1535–1546Google Scholar
  35. 35.
    1998McLean:98 McLean B, Hawkins C, Spagna A, Lattanzi M, Lasker B, Jenkner H, White R (1998) New horizons from multi-wavelength sky surveys. IAU Symposium 179, Baltimore, MDGoogle Scholar
  36. 36.
    1998Meila:1998 Meila M, Jordan MI (1998) Estimating dependency structure as a hidden variable. In ‘NIPS’Google Scholar
  37. 37.
    2002Park:02 Park B, Kargupta H (2002) Distributed data mining: algorithms, systems, and applications. In Ye N (ed). Data mining handbook (To appear)Google Scholar
  38. 38.
    1993Pearl:1993 Pearl J (1993) Graphical models. causality and intervention. Statistical Science 8:266–273Google Scholar
  39. 39.
    1997Singh:1997 Singh M (1997) Learning Bayesian networks from incomplete data. In Proceedings of the national conference on artificial intelligence. AAAI Press Menlo Park, CA, pp 27–31Google Scholar
  40. 40.
    1995Singh:1995 Singh M, Provan GM (1995) A comparison of induction algorithms for selective and non-selective Bayesian classifiers. In Prieditis A, Russel S (eds). Proceedings of the 12th international conference on machine learning. Morgan Kaufmann, San Mateo, CA, pp 497–505Google Scholar
  41. 41.
    1990Spiegelhalter:1990 Spiegelhalter DJ, Lauritzen SL (1990) Sequential updating of conditional probabilities on directed graphical structures. Networks 20:570–605Google Scholar
  42. 42.
    1993Spirtes:1993 Spirtes P, Glymour C, Scheines R, (1993) Causation, prediction and search. In Lecture notes in statistics. no 81. Springer, BerlinGoogle Scholar
  43. 43.
    1993Suzuki:1993 Suzuki J (1993) A construction of Bayesian networks from databases based on an MDL scheme. In Heckerman D, Mamdani A (eds). Proceedings of the 9th conference on uncertainty in artificial intelligence. Morgam Kaufmann, San Mateo, CA, pp 266–273Google Scholar
  44. 44.
    1998Szalay:98 Szalay A (1998) The evolving universe. ASSL (231)Google Scholar
  45. 45.
    1995Thiesson:1995 Thiesson B (1995) Accelerated quantification of Bayesian networks with incomplete data. In Proceedings of the first international conference on knowledge discovery and data mining. AAAI Press, Menlo Park, CA, pp 306–311Google Scholar
  46. 46.
    1992Thomas:1992 Thomas A, Spiegelhalter D, Gilks W (1992) Bugs: a program to perform Bayesian inference using Gibbs sampling. In Bernardo J, Berger J, Dawid A, Smith A (eds). Bayesian statistics. Oxford University Press, Oxford, pp 837–842.Google Scholar
  47. 47.
    1998Zweig:1998 Zweig G, Russel SJ (1998) Speech recognition with dynamic Bayesian networks. In Proceedings of the 15th national conference on artificial intelligence, Madison, WIGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2004

Authors and Affiliations

  1. 1.School of Electrical Engineering and Computer ScienceWashington State UniversityUSA
  2. 2.Department of Computer Science and Electrical EngineeringUniversity of Maryland Baltimore CountyMDUSA

Personalised recommendations