Advertisement

Incorporating Regular Vines in Estimation of Distribution Algorithms

  • Rogelio Salinas-GutiérrezEmail author
  • Arturo Hernández-Aguirre
  • Enrique R. Villa-Diharce
Part of the Studies in Computational Intelligence book series (SCI, volume 447)

Abstract

This chapter presents the incorporation and use of regular vines into Estimation of Distribution Algorithms for solving numerical optimization problems. Several kinds of statistical dependencies among continuous variables can be taken into account by using regular vines. This work presents a procedure for selecting the most important dependencies in EDAs by truncating regular vines. Moreover, this chapter also shows how the use of mutual information in the learning of graphical models implies a natural way of employing copula functions.

Keywords

Mutual Information Marginal Distribution Marginal Density Copula Function Distribution Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Aas, K., Czado, C., Frigessi, A., Bekken, H.: Pair-copula constructions of multiple dependence. Insurance: Mathematics and Economics 44(2), 182–198 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Arderí-García, R.J.: Algoritmo con Estimación de Distribuciones con Cópula Gaussiana. Universidad de La Habana, La Habana, Cuba (June 2007), Bachelor’s thesis. in Spanish Google Scholar
  3. 3.
    T. Bacigál and M. Komorníková. Fitting Archimedean copulas to bivariate geodetic data. In A. Rizzi and M. Vichi, editors, Compstat 2006 Proceedings in Computational Statistics, pages 649–656, Heidelberg, Germany, 2006. Physica-Verlag HD.Google Scholar
  4. 4.
    Baluja, S.: Population-Based Incremental Learning: A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning. Technical Report CMU-CS-94-163. Carnegie Mellon University, Pittsburgh, PA, USA (June 1994)Google Scholar
  5. 5.
    Baluja, S., Davies, S.: Using Optimal Dependency-Trees for Combinatorial Optimization: Learning the Structure of the Search Space. In: Fisher, D.H. (ed.) Proceedings of the Fourteenth International Conference on Machine Learning, pp. 30–38. Morgan Kaufmann (1997)Google Scholar
  6. 6.
    Barba-Moreno, S.E.: Una propuesta para EDAs no paramétricos. Master’s thesis, Centro de Investigación en Matemáticas, Guanajuato, México (December 2007) (in Spanish)Google Scholar
  7. 7.
    Bedford, T., Cooke, R.M.: Probability Density Decomposition for Conditionally Dependent Random Variables Modeled by Vines. Annals of Mathematics and Artificial Intelligence 32(1), 245–268 (2001)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Bedford, T., Cooke, R.M.: Vines – A New Graphical Model for Dependent Random Variables. The Annals of Statistics 30(4), 1031–1068 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Bosman, P.A.N., Grahl, J.: Matching Inductive Search Bias and Problem Structure in Continuous Estimation of Distribution Algorithms. Technical Report 03/2005. University of Mannheim, Mannheim, Germany (2005)Google Scholar
  10. 10.
    Bosman, P.A.N., Grahl, J., Thierens, D.: Enhancing the Performance of Maximum–Likelihood Gaussian EDAs Using Anticipated Mean Shift. In: Rudolph, G., Jansen, T., Lucas, S., Poloni, C., Beume, N. (eds.) PPSN 2008. LNCS, vol. 5199, pp. 133–143. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Brunel, N., Pieczynski, W., Derrode, S.: Copulas in vectorial hidden markov chains for multicomponent image segmentation. In: ICASSP 2005: Proceedings of the 2005 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 717–720 (2005)Google Scholar
  12. 12.
    Cherubini, U., Luciano, E., Vecchiato, W.: Copula Methods in Finance. Wiley, Chichester (2004)zbMATHGoogle Scholar
  13. 13.
    Cuesta-Infante, A., Santana, R., Hidalgo, J.I., Bielza, C., Larrañaga, P.: Bivariate empirical and n-variate Archimedean copulas in Estimation of Distribution Algorithms. In: WCCI 2010 IEEE World Congress on Computational Intelligence, pp. 1355–1362 (July 2010)Google Scholar
  14. 14.
    Davy, M., Doucet, A.: Copulas: a new insight into positive time-frequency distributions. IEEE Signal Processing Letters 10(7), 215–218 (2003)MathSciNetCrossRefGoogle Scholar
  15. 15.
    De Bonet, J.S., Isbell, C.L., Viola, P.: MIMIC: Finding Optima by Estimating Probability Densities. In: Advances in Neural Information Processing Systems, vol. 9, pp. 424–430. The MIT Press (1997)Google Scholar
  16. 16.
    De-Waal, D.J., Van-Gelder, P.H.A.J.M.: Modelling of extreme wave heights and periods through copulas. Extremes 8(4), 345–356 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Dowd, K.: Copulas in Macroeconomics. Journal of International and Global Economic Studies 1(1), 1–26 (2008)Google Scholar
  18. 18.
    Etxeberria, R., Larrañaga, P.: Global optimization with Bayesian networks. In: Ochoa, A., Soto, M., Santana, R. (eds.) Second International Symposium on Artificial Intelligence. Adaptive Systems, CIMAF 1999, La Habana, pp. 332–339. Academia (1999)Google Scholar
  19. 19.
    Flitti, F., Collet, C., Joannic-Chardin, A.: Unsupervised Multiband Image Segmentation using Hidden Markov Quadtree and Copulas. In: IEEE International Conference on Image Processing, Genova, Italy (September 2005)Google Scholar
  20. 20.
    Flores de la Fuente, E.: EDAs con Funciones de cópula. Master’s thesis, Centro de Investigación en Matemáticas, Guanajuato, México (September 2009) (in Spanish)Google Scholar
  21. 21.
    Frees, E.W., Valdez, E.A.: Understanding relationships using copulas. North American Actuarial Journal 2(1), 1–25 (1998)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Gao, Y.: Multivariate Estimation of Distribution Algorithm with Laplace Transform Archimedean Copula. In: Hu, W., Li, X. (eds.) 2009 International Conference on Information Engineering and Computer Science, ICIECS 2009, Wuhan, China (December 2009)Google Scholar
  23. 23.
    Genest, C., Favre, A.C.: Everything You Always Wanted to Know about Copula Modeling but Were Afraid to Ask. Journal of Hydrologic Engineering 12(4), 347–368 (2007)CrossRefGoogle Scholar
  24. 24.
    González, C.: Contributions on Theoretical Aspects of Estimation of Distribution Algorithms. PhD thesis. University of the Basque Country, Donostia-San Sebastián, Spain (November 2005)Google Scholar
  25. 25.
    Grahl, J., Minner, S., Bosman, P.A.N.: Learning structure illuminates black boxes – an introduction into Estimation of Distribution Algorithms. Technical Report 10/2006, University of Mannheim, Mannheim, Germany (2006)Google Scholar
  26. 26.
    Grigoriu, M.: Multivariate distributions with specified marginals: Applications to Wind Engineering. Journal of Engineering Mechanics 133(2), 174–184 (2007)CrossRefGoogle Scholar
  27. 27.
    Harik, G., Lobo, F.G., Goldberg, D.E.: The Compact Genetic Algorithm. In: Proceedings of the IEEE Conference on Evolutionary Computation, pp. 523–528 (1998)Google Scholar
  28. 28.
    Höns, R.: Estimation of Distribution Algorithm and Minimum Relative Entropy. PhD thesis. University of Bonn, Bonn, Germany (2005)Google Scholar
  29. 29.
    Igel, C., Suttorp, T., Hansen, N.: A computational efficient covariance matrix update and a (1+1)-CMA for evolution strategies. In: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, GECCO 2006, pp. 453–460. ACM (2006)Google Scholar
  30. 30.
    Jajuga, K., Papla, D.: Copula Functions in Model Based Clustering. In: Spiliopoulou, M., Kruse, R., Borgelt, C., Nürnberger, A., Gaul, W. (eds.) From Data and Information Analysis to Knowledge Engineering. Studies in Classification, Data Analysis, and Knowledge Organization, pp. 606–613. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  31. 31.
    Jenison, R.L., Reale, R.A.: The Shape of Neural Dependence. Neural Computation 16, 665–672 (2004)zbMATHCrossRefGoogle Scholar
  32. 32.
    Joe, H.: Multivariate models and dependence concepts. Chapman and Hall, Boca Raton (1997)zbMATHGoogle Scholar
  33. 33.
    Kurowicka, D., Cooke, R.: The vine copula method for representing high dimensional dependent distributions: application to continuous belief nets. In: Yücesan, E., Chen, C.-H., Snowdon, J.L., Charnes, J.M. (eds.) Proceedings of the 2002 Winter Simulation Conference, pp. 270–278 (2002)Google Scholar
  34. 34.
    Kurowicka, D., Cooke, R.: Uncertainty Analysis. Series in Probability and Statistics. Wiley. Wiley (2006)Google Scholar
  35. 35.
    Larrañaga, P., Etxeberria, R., Lozano, J.A., Peña, J.M.: Combinatorial optimization by learning and simulation of Bayesian networks. In: Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, pp. 343–352 (2000)Google Scholar
  36. 36.
    Larrañaga, P., Etxeberria, R., Lozano, J.A., Peña, J.M.: Optimization in continuous domains by learning and simulation of Gaussian networks. In: Proceedings of the Optimization by Building and Using Probabilistic Models OBUPM Workshop at the Genetic and Evolutionary Computation Conference GECCO 2000, pp. 201–204 (2000)Google Scholar
  37. 37.
    Larrañaga, P.: A Review on Estimation of Distribution Algorithms. In: Larrañaga and Lozano (eds.) [38], ch. 3, pp. 57–100 (2002)Google Scholar
  38. 38.
    Larrañaga, P., Lozano, J.A. (eds.): Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Genetic Algorithms and Evolutionary Computation. Kluwer Academic Publishers (2002)Google Scholar
  39. 39.
    Mercier, G., Bouchemakh, L., Smara, Y.: The Use of Multidimensional Copulas to Describe Amplitude Distribution of Polarimetric SAR Data. In: IGARSS 2007 (2007)Google Scholar
  40. 40.
    Monjardin, P.E.: Análisis de dependencia en tiempo de falla. Master’s thesis, Centro de Investigación en Matemáticas, Guanajuato, México (December 2007) (in Spanish)Google Scholar
  41. 41.
    Mühlenbein, H.: The Equation for Response to Selection and its Use for Prediction. Evolutionary Computation 5(3), 303–346 (1998)CrossRefGoogle Scholar
  42. 42.
    Mühlenbein, H., Mahnig, T., Ochoa-Rodriguez, A.: Schemata, distributions and graphical models in evolutionary optimization. Journal of Heuristics 5, 215–247 (1999)zbMATHCrossRefGoogle Scholar
  43. 43.
    Mühlenbein, H., Paaß, G.: From Recombination of Genes to the Estimation of Distributions I. Binary Parameters. In: Ebeling, W., Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 178–187. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  44. 44.
    Nelsen, R.B.: An Introduction to Copulas, 2nd edn. Springer Series in Statistics. Springer (2006)Google Scholar
  45. 45.
    Pelikan, M., Goldberg, D.E., Lobo, F.G.: A Survey of Optimization by Building and Using Probabilistic Models. Computational Optimization and Applications 21(1), 5–20 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Pelikan, M., Goldberg, D.E., Cantú-Paz, E.: BOA: The Bayesian Optimization Algorithm. In: Banzhaf, W., Daida, J., Eiben, A.E., Garzon, M.H., Honavar, V., Jakiela, M., Smith, R.E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 1999, vol. 1, pp. 525–532. Morgan Kaufmann Publishers (1999)Google Scholar
  47. 47.
    Pelikan, M., Mühlenbein, H.: The Bivariate Marginal Distribution Algorithm. In: Roy, R., Furuhashi, T., Chawdhry, P.K. (eds.) Advances in Soft Computing - Engineering Design and Manufacturing, pp. 521–535. Springer, London (1999)Google Scholar
  48. 48.
    Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.): Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications. SCI, vol. 33. Springer (2006)Google Scholar
  49. 49.
    Sakji-Nsibi, S., Benazza-Benyahia, A.: Multivariate indexing of multichannel images based on the copula theory. In: IPTA 2008 (2008)Google Scholar
  50. 50.
    Salinas-Gutiérrez, R., Hernández-Aguirre, A., Rivera-Meraz, M.J.J., Villa-Diharce, E.R.: Supervised Probabilistic Classification Based on Gaussian Copulas. In: Sidorov, G., Hernández Aguirre, A., Reyes García, C.A. (eds.) MICAI 2010, Part II. LNCS(LNAI), vol. 6438, pp. 104–115. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  51. 51.
    Salinas-Gutiérrez, R., Hernández-Aguirre, A., Rivera-Meraz, M.J.J., Villa-Diharce, E.R.: Using Gaussian Copulas in Supervised Probabilistic Classification. In: Castillo, O., Kacprzyk, J., Pedrycz, W. (eds.) Soft Computing for Intelligent Control and Mobile Robotics. SCI, vol. 318, pp. 355–372. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  52. 52.
    Salinas-Gutiérrez, R., Hernández-Aguirre, A., Villa-Diharce, E.R.: Using Copulas in Estimation of Distribution Algorithms. In: Aguirre, A.H., Borja, R.M., Garciá, C.A.R. (eds.) MICAI 2009. LNCS(LNAI), vol. 5845, pp. 658–668. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  53. 53.
    Salinas-Gutiérrez, R., Hernández-Aguirre, A., Villa-Diharce, E.R.: D-vine EDA: a new Estimation of Distribution Algorithm based on Regular Vines. In: GECCO 2010: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, pp. 359–366. ACM, New York (2010)CrossRefGoogle Scholar
  54. 54.
    Salinas-Gutiérrez, R., Hernández-Aguirre, A., Villa-Diharce, E.R.: Dependence Trees with Copula Selection for Continuous Estimation of Distribution Algorithms. In: GECCO 2011: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, pp. 585–592. ACM (2011); Estimation of Distribution Algorithms Track PapersGoogle Scholar
  55. 55.
    Schölzel, C., Friederichs, P.: Multivariate non-normally distributed random variables in climate research – introduction to the copula approach. Nonlinear Processes in Geophysics 15(5), 761–772 (2008)CrossRefGoogle Scholar
  56. 56.
    Sklar, A.: Fonctions de répartition à n dimensions et leurs marges. Publications de l’Institut de Statistique de l’Université de Paris 8, 229–231 (1959)MathSciNetGoogle Scholar
  57. 57.
    Soto, M., Ochoa, A., Acid, S., de Campos, L.M.: Introducing the polytree approximation of distribution algorithm. In: Ochoa, A., Soto, M., Santana, R. (eds.) Second International Symposium on Artificial Intelligence. Adaptive Systems, CIMAF 1999, pp. 360–367. Academia, La Habana (1999)Google Scholar
  58. 58.
    Stitou, Y., Lasmar, N., Berthoumieu, Y.: Copulas based multivariate gamma modeling for texture classification. In: ICASSP 2009: Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1045–1048. IEEE Computer Society, Washington, DC (2009)Google Scholar
  59. 59.
    Trivedi, P.K., Zimmer, D.M.: Copula Modeling: An Introduction for Practitioners. Foundations and Trends® in Econometrics. Now Publishers (2007)Google Scholar
  60. 60.
    Venter, G., Barnett, J., Kreps, R., Major, J.: Multivariate Copulas for Financial Modeling. Variance 1(1), 103–119 (2007)Google Scholar
  61. 61.
    Wang, L., Guo, X., Zeng, J., Hong, Y.: Using Gumbel Copula and Empirical Marginal Distribution in Estimation of Distribution Algorithm. In: Third International Workshop on Advanced Computational Intelligence, IWACI 2010, pp. 583–587. IEEE (August 2010)Google Scholar
  62. 62.
    Wang, L., Zeng, J., Hong, Y., Guo, X.: Copula Estimation of Distribution Algorithm Sampling from Clayton Copula. Journal of Computational Information Systems 6(7), 2431–2440 (2010)Google Scholar
  63. 63.
    Wang, L.F., Wang, Y.C., Zeng, J.C., Hong, Y.: An Estimation of Distribution Algorithm Based on Clayton Copula and Empirical Margins. In: Li, K., Li, X., Ma, S., Irwin, G.W. (eds.) LSMS 2010. CCIS, vol. 98, pp. 82–88. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  64. 64.
    Wang, L.F., Zeng, J.C.: Estimation of Distribution Algorithm Based on Copula Theory. In: Chen, Y.P. (ed.) Exploitation of Linkage Learning in Evolutionary Algorithms. Adaptation, Learning, and Optimization, vol. 3, pp. 139–162. Springer (2010)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2013

Authors and Affiliations

  • Rogelio Salinas-Gutiérrez
    • 1
    Email author
  • Arturo Hernández-Aguirre
    • 1
  • Enrique R. Villa-Diharce
    • 1
  1. 1.Center for Research in Mathematics (CIMAT)GuanajuatoMéxico

Personalised recommendations