Skip to main content

Advertisement

SpringerLink
Log in

Transfer learning for temporal nodes Bayesian networks

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Traditional machine learning algorithms depend heavily on the assumption that there is sufficient data to learn a reliable model. This is not always the case, and in situations where data is limited, transfer learning can be applied to compensate for the lack of information by learning from several sources. In this work, we present a novel methodology for inducing a Temporal Nodes Bayesian Network (TNBN) when training data is scarce by applying a transfer learning strategy. A TNBN is a probabilistic graphical model that offers a compact representation for dynamic domains by defining multiple time intervals in which events can occur. Learning a TNBN poses additional challenges to learning traditional Bayesian networks due to the incorporation of time intervals. Our proposal incorporates novel approaches to transfer knowledge from several TNBNs to learn the structure, parameters and intervals of a target TNBN. To evaluate our algorithm, we performed experiments with a synthetic network, where we created auxiliary models by altering the structure, parameters and temporal intervals of the original model. Results show that the proposed algorithm is capable of retrieving a reliable model even when few records are available for the target domain. We also performed experiments with a real-world data set belonging to the medical domain of HIV, where we were able to learn some documented mutational pathways and their temporal relations by applying transfer learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The decision to have the target data account for at least 25 % of the total was chosen empirically, and will not necessarily be the best value for other domains. To find a good percentage split, a cross validation procedure can be carried out, where the percentage assigned to the target data is increased with every new cross validation. The percentage split that averaged the best results is used.

  2. For our experiments, we approximate a data set of sufficient size to be that which holds at least 10 records per each probability value present in the model. A data set with fewer than this minimum of records is considered to have “scarce” records.

  3. Dr. Santiago Avila from the Research Center for Infectious Diseases (or CIENI) in Mexico city provided his assistance for these experiments.

References

  1. Arroyo-Figueroa G, Sucar LE (1999) A temporal Bayesian network for diagnosis and prediction. In: Proceedings of the fifteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc, pp 13–20

  2. Arroyo-Figueroa G, Sucar LE, Villavicencio A (1998) Probabilistic temporal reasoning and its application to fossil power plant operation. Expert Syst Appl 15(3):317–324

    Article  Google Scholar 

  3. Beerenwinkel N, Schmidt B, Walter H, Kaiser R, Lengauer T, Hoffmann D, Korn K, Selbig J (2002) Diversity and complexity of HIV-1 drug resistance: a bioinformatics approach to predicting phenotype from genotype. Proc National Acad Sci 99(12):8271–8276

    Article  Google Scholar 

  4. Brier GW (1950) Verification of forecasts expressed in terms of probability. Mon Weather Rev 78(1):1–3

    Article  Google Scholar 

  5. Caruana R (1997) Multitask learning. Mach Learn 28(1):41–75

    Article  MathSciNet  Google Scholar 

  6. Consortium E, et al (2002) Elvira: An environment for creating and using probabilistic graphical models. In: Proceedings of the first European workshop on probabilistic graphical models, pp. 222–230

  7. Cooper GF, Herskovits E (1992) A Bayesian method for the induction of probabilistic networks from data. Mach Learn 9(4):309–347

    MATH  Google Scholar 

  8. Dai W, Xue GR, Yang Q, Yu Y (2007) Transferring naive Bayes classifiers for text classification. In: Proceedings of the national conference on artificial intelligence, vol 22. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, p 540

  9. Drȧghici S, Potter RB (2003) Predicting HIV drug resistance with neural networks. Bioinformatics 19(1):98–107

    Article  Google Scholar 

  10. Dunn JC (1973) A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters. Cybern Syst 3(3):32–57

    MathSciNet  MATH  Google Scholar 

  11. Fiedler Cameras LJ, Sucar LE, Morales EF (2013) A transfer learning approach for learning temporal nodes Bayesian networks. In: The twenty-sixth international FLAIRS conference, pp 637–640

  12. Friedman N, Yakhini Z (1996) On the sample complexity of learning bayesian networks. In: Proceedings of the twelfth international conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc, pp 274–282

  13. Hanks S, Madigan D, Gavrin J (1995) Probabilistic temporal reasoning with endogenous change. In: Proceedings of the eleventh conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc, pp 245–254

  14. Hernandez-Leal P, Gonzalez JA, Morales EF, Enrique Sucar L (2013) Learning temporal nodes Bayesian networks. Int J Approx Reason 54(8):956–977

    Article  Google Scholar 

  15. Hernandez-Leal P, Rios-Flores A, Ávila-Rios S, Reyes-Terán G, Gonzalez JA, Fiedler-Cameras L, Orihuela-Espina F, Morales EF, Sucar LE (2013) Discovering human immunodeficiency virus mutational pathways using temporal Bayesian networks. Artif Intell Med 57(3):185–195

    Article  Google Scholar 

  16. Hernandez-Leal P, Sucar LE, Gonzalez JA, Morales EF, Ibarguengoytia PH (2011) Learning temporal bayesian networks for power plant diagnosis. In: Modern approaches in applied intelligence. Springer, pp 39–48

  17. Luis R, Sucar LE, Morales EF (2010) Inductive transfer for learning Bayesian networks. Mach Learn 79(1–2):227–255

    Article  MathSciNet  Google Scholar 

  18. Masquelier B, Breilh D, Neau D, Lawson-Ayayi S, Lavignolle V, Ragnaud JM, Dupon M, Morlat P, Dabis F, Fleury H et al (2002) Human immunodeficiency virus type 1 genotypic and pharmacokinetic determinants of the virological response to lopinavir-ritonavir-containing therapy in protease inhibitor-experienced patients. Antimicrob Agents Chemother 46(9):2926–2932

    Article  Google Scholar 

  19. Niculescu-Mizil A, Caruana R (2007) Inductive transfer for Bayesian network structure learning. J Mach Learn Res Proc Track 2:339–346

    Google Scholar 

  20. Oniśko A, Druzdzel MJ, Wasyluk H. (2001) Learning Bayesian network parameters from small data sets: application of noisy-or gates. Int J Approx Reason 27(2):165–182

    Article  MATH  Google Scholar 

  21. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowledge Data Eng 22(10):1345–1359

    Article  Google Scholar 

  22. Rhee SY, Gonzales MJ, Kantor R, Betts BJ, Ravela J, Shafer RW (2003) Human immunodeficiency virus reverse transcriptase and protease sequence database. Nucleic Acids Res 31(1):298–303

    Article  Google Scholar 

  23. Singh M, Valtorta M (1993) An algorithm for the construction of bayesian network structures from data. In: Proceedings of the ninth international conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc, pp 259–265

  24. Spirtes P, Glymour C, Scheines R (2000) Causation, prediction, and search, vol 81. The MIT Press

  25. Sun S, Xu Z, Yang M (2013) Transfer learning with part-based ensembles. In: Multiple classifier systems. Springer, pp 271– 282

  26. Tonda AP, Lutton E, Reuillon R, Squillero G, Wuillemin PH (2012) Bayesian network structure learning from limited datasets through graph evolution. In: EuroGP. Springer Verlag, pp 254– 265

  27. Xu Z, Sun S (2012) Multi-source transfer learning with multi-view adaboost. In: Neural information processing. Springer, pp 332–339

Download references

Acknowledgments

The first author is thankful to CONACyT for the financial support given to her through scholarship 261257.

Author information

Authors and Affiliations

  1. Instituto Nacional de Astrofísica, Óptica y Electrónica, Coordinación de Ciencias Computacionales, Luis Enrique Erro No. 1, Sta. Ma. Tonanzintla, Puebla, CP, 72840, México

    Lindsey J. Fiedler, L. Enrique Sucar & Eduardo F. Morales

Authors
  1. Lindsey J. Fiedler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. L. Enrique Sucar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eduardo F. Morales
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lindsey J. Fiedler.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fiedler, L.J., Sucar, L.E. & Morales, E.F. Transfer learning for temporal nodes Bayesian networks. Appl Intell 43, 578–597 (2015). https://doi.org/10.1007/s10489-015-0662-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-015-0662-1

Keywords

Search

Navigation